Kong's AI Semantic Cache plugin allows us to perform differently formulated questions, with different semantics, but, nonetheless, the same meaning, and make use of a cache system, to keep the answers to those questions persisted. It allows to use a vector database, like for example the Redis database. In this video we are having a quick look at how to configure the AI Semantic Cache plugin and we will see how it works. We will also focus on how it can go wrong and what can we do to improve it. The point of this plugin is to avoid making "unnecessary" requests to our LLM model. The way this works is that the LLM model returns data in the form of embeddings that get vectorized in the Redis database. Using vectors, the Redis database can then leverage the plugin to be able to find similarities and determine if a certain question has already been made or not, based on how similar it is to other questions with the same response. The theory behind this is quite fascinating, but we are only looking at this plugin from a very technical perspective in this video. We will use as an examplel, questions about the color of items, like for the example, the color of the Sun. I hope you enjoy the video and have fun with it and until the next one, be sure to stay tech, keep programming, be kind and have a good one everyone! Cheers!
---
Chapters:
00:00:00 Start
00:00:42 Intro
00:03:42 Explaining Kong AI Semantic Cache Plugin
00:05:52 Referencing "What is a vector database ?" video
- https://www.youtube.com/watch?v=Yhv19le0sBw
00:07:51 Referencing the "Kong KONNECT, the first steps" video
- https://youtu.be/z3Y4NQgjGLE
00:08:12 Configuring Kong Konnect
00:09:48 Configuring the AI Semantic Cache Plugin in the example
00:13:12 Referencing the "How to and why configure the Kong AI-Proxy plugin in 10 minutes" video
- https://youtu.be/6Z8wWX-liBs
00:13:33 How to download the REDIS container
00:14:20 Explaining how to use environment variables with the prepared commands
00:18:52 Performing tests against the AI Semantic Cache plugin
00:24:56 End Notes
00:25:46 See you in the next video!
00:27:22 End credits
00:28:01 Disclaimer
---
Related videos:
- https://youtu.be/Kw5GZnMnVhw
- https://youtu.be/rJKbAzjb5lQ
- https://youtu.be/z3Y4NQgjGLE
- https://youtu.be/KE3VTYtLvnI
- https://youtu.be/6Z8wWX-liBs
- https://youtu.be/vRH4qLZ7tz8
- https://youtu.be/Yhv19le0sBw
--- Source code
- https://github.com/jesperancinha/kong-test-drives
- https://github.com/jesperancinha/jeorg-cloud-test-drives
---
Soundtrack:
- https://soundcloud.com/joaoesperancinha/slow-guitar-6-jesprotech
---
Sources:
- https://docs.konghq.com/hub/kong-inc/ai-semantic-cache/
---
As a short disclaimer, I'd like to mention that I'm not associated or affiliated with any of the brands eventually shown, displayed, or mentioned in this video.
---
