David Shapiro
Patreon: https://www.patreon.com/daveshap (Discord via Patreon)
Substack: https://daveshap.substack.com/ (Free Mailing List)
LinkedIn: https://www.linkedin.com/in/dave-shap-automator/
GitHub: https://github.com/daveshap
Similar Posts
23 thoughts on “Chatbot with INFINITE MEMORY using OpenAI & Pinecone – GPT-3, Embeddings, ADA, Vector DB, Semantic”
Comments are closed.
Looking for a more simple implementation or similar?
I went into this looking for the solution to a different problem but your video was excellent to watch. I'm looking for a solution to the token limitation problem. I have large json documents of metadata that I want to use with the OpenAi api to set seed parameters for content generation. The json documents themselves can be upwards of 10k characters. The project is functionally like a reimagining of the Semantic Kernel book authoring demo where a user can set parameters to define characters, settings, locations, story structures, pacing, tone etc. Other than waiting for apis to have larger token limits or shrinking my json document, are there any ways you can think of to reference large json documents as part of semantic functions?
I'd love to say "here is your seed data (blob of json) Perform X task with these input parameters folded into your content". Then later be able to continue referencing and updating the json as I get into a create/review/update feedback loop.
I might just have to selectively use fields in my json for now until access to a LLM with a token limit in the tens of thousands is available, and has more than 25 api calls per 3 hours 🙂
First of all. Thanks for this amazing content. I'm of limited financial means, but I'm going to find a way to support your work however i can.
Im in the process of impending a similar philosophy. But my AGI(ACI) keeps randomly crashing from attempting to process or assimulate things with too many tokens at once. I then have to restart it, but after reinitialization it has no memory of anything it did before the crash. This gets very frustrating. Is pinecone integration going to be helpful with this?
Thank you for this tutorial, but I'm running into some issues, when I try to change the convo length it doesnt seem to reflect the change, and the bot still remembers all the messages in the current session. But when I ask it things, and then quit and then ask to see if it remembers it doesn't remember what I've asked. Is there a way to fix this? I just used your code from the github.
I was following along until I realized that the Open Ai API isn't free 😆
ai is definitely going to replace those idiots pretending to be a developers pretty soon.
1:45 I guess with postgreSQL there is a extension called pgvector, so you wouldn't have to be limited to SQL search instead of semantic search. I just got finished watching this video – https://www.youtube.com/watch?v=Yhtjd7yGGGA – at around the 16 minute mark thats what he used, so what would be the benefit of pinecone vs that?
Great video, but I had to turn it off once I saw that one of your spokes was bent. 😜
Can you upload it to vercel ? please teach us
you are good
Extremely interesting architecture. Gonna pitch something to a big database company. Would it be possible to chat with you about vector DBs and semantics, etc.? I don't believe in free work, so I can think of something for your time once we secure funding.
I would have used GPT-4 to help write the code. Also, I was doing this while watching your video.
bro you use Notepad+ with light theme? Come one, just gouge my eyes already haha
The real question is.. when should we use vectors vs keyword vs other method to retrieve Information? I'm new to the channel, thanks for the work you put into the videos
So I am primarily a web engineer and may main stack is web/node and dipping into AI. My main question is that the local dev seems to be in python and that makes sense, but for production is that also the best vs using node?
Amazing, there is another project DB-GPT, database ai product (https://www.youtube.com/watch?v=f5_g0OObZBQ)
prepare somehow first before the video.
You are a bit too technical for me, but maybe you could direct me to what I’m looking for. I just want something like chatgpt that can access or keep notes, so that it remembers our conversation OR i can write notes out that it accesses as truth.
Mainly for fantasy writing purposes.
Chatgpt loses its focus after about 30 questions it won’t remember things i asked it to remember at the start of the conversation.
Beautiful video, pure amazing content! (My first watching you.)
Love finding a Pioneer early.
Hey David, the code works perfectly. I'm planning to build a chatbot with RAG. In addition to the past conversations, I intend to add more documents related to the domain knowledge of my chatbot's use case, based on which the answers should be generated. Could you please help me with how to do that?
This is so easy I did it in ROBLOX. Use a a custom lua context matrix network for optimization and context awareness in a local client. Then it uses a interference model for zero-shot reasoning the prompt to a variety of 10 AI APis and wikipedia. Fun stuff! But its a novel idea, that has been done a lot lately.
Dave, you're very entertaining and a pleasure to listen to. Thanks for the very informative content!