Why are we using Pinecone instead of a local vector database?

[DavidMCampbell]

I don't see any reason why Pinecone should be used. Some locally-running vector database would have lower latency, be free, and not require extra account creation. Why isn't a local vector database library the first choice, @Torantulino ?? Anything local like Milvus or Weaviate would be free, local, private, not require an account, and not require users to wait forever for pinecone to "initialize".

[Nnnsightnnn]

Scalability is what I would assume.

[GoMightyAlgorythmGo]

Complex tasks. (my project barely survives with pinecone) For quick er low latency stuff: Luckely my pinecone (region asia im in west europe) loads pretty much instantly it seems. But redis is already available way faster and also local cache which would be 0 installation required right:
python -m autogpt --use-memory local cache or smth.

[MrOakwine]

Yeah , same for OpenAi API, can't we use Alpaca or Vicuuna

[GoMightyAlgorythmGo]

No openAI is too good. But using VIcunna in addition would be useful Maybe multible instances or teams of conversational agents that act based on chain of thought promts or custom promts how they should act together to dubble and tribble check something might provide useful output. (And it might pavethe way to "freeMod" that does not cost much/any tokens while probably being less effective)

[marktellez]

I run local models with lm studio and text web chat and they have made me realize how FAR ahead the open AI models really are! :D

[barakplasma]

From what I see in scripts/memory/init.py it's already using local memory by default (rather than pinecone). There's also an option for Redis rather than Pinecone, which you can control with an env var according to https://github.com/Torantulino/Auto-GPT#redis-setup . https://github.com/Torantulino/Auto-GPT/blob/d8a7a811c841ae42952e859f3ee1e5754f0a79d8/scripts/memory/__init__.py#L15-L36

[tonihintikka]

Ok So I can choose this Redis and run it with docker without the Pinecode. It seems that Pinecode payed account is only available this time. Just want to get AutoGPT running.

[fire17]

I think this project needs some refactoring to be more generic. Like adding Data Access Layers (Dals) so the infra is easily swappable.

For this case we need to make the CRUD equivalent wrapper for embeddings. Needs connect to client, add docs, lookup, edit and delete.

Then the default should probably a local foss solution like chroma or one of the others, but anyone can easily choose pinecone instead while not needing to change any of the code. Just make a simple adapter, and set it to load from config.

[GoMightyAlgorythmGo]

Dont understand but if you mean more options for memory retrival then 100% this is insanely important for AutoGPT in long compelx projects and any projects at all. Human memory is a complex of many memory systems not just 1 form that is the best general one but also tonns of specialist and combinations almost like a democracy or super big network

[jamesoflol]

I agree. Not only do we not need Pinecone here, and it's not being used by default, but we don't need a vector DB at all.

Vector similarity search doesn't get computationally expensive until you hit 100k+ vectors, especially if you're just doing dot product. This should just be held in memory during run, with optionally storing to a local flat file if needed between executions.

Additionally, I don't see why we really need the OpenAI embeddings API. The embeddings here appear to just be used for a very basic similarity search, as we can't actually pass the vectors directly back to GPT3/4. Lots of research and analysis has shown that purpose-built sentence embedding models actually outperform huge language model embeddings for the sake of semantic similarity search. https://www.sbert.net/

[GoMightyAlgorythmGo]

Future improvement my project is insanely complex for example (even simple tasks can get very complex if they are lage enougth, eventho the components are infinitly simple, not recalling a previous system may make it useless just like 1 false number or missing operation can derail a very long chain of a calculation) and not everyone only wants AutoGPT for simple things. Those will be doable when the compelx things work anyway and even more smoothly.

[yonnic]

the reason is that pinecone does some heavy marketing and is investing lot of money into content creation like tutorials and sponsored content. You can see on youtube how much pinecone content there is. VCs currently invest heavily into startups in the AI area so they can easily afford such heavy marketing

[GoMightyAlgorythmGo]

I don't see any reason why Pinecone should be used. Some locally-running vector database would have lower latency, be free, and not require extra account creation. Why isn't a local vector database library the first choice, @Torantulino ?? Anything local like Milvus or Weaviate would be free, local, private, not require an account, and not require users to wait forever for pinecone to "initialize".

There is already redis docker option or local cache. My project/application benefits massively from pinecone though and its not overkill its barely enought actually a combination of all integrated memory systems or a more airtight system for making progress along the lines of past plans and resulting progress would be very helpful. So you dont have to worry just use redis if that is what you want or local cache (for that no installation is required i believe just add the "python -m autogpt --use-memory local cache" or something like that. 👍

[lucasgadams]

How many vectors are you storing honestly?

[LarryStewart2022]

We signed up for pinecone in June with the $70 per month plan, we created an index of our products, 3,000 of them. We made a few test calls; 8 or 9 after we uploaded the data and that was it, we did nothing with it since. And today, July 6th, I received a bill for $123.31. WHAT THE HELL!

Jun 1st - Jun 30th 2023
Total Cost $123.31
Daily Average $4.11

Copied from the usage page. This was from 1 index with 3000 products and 9 queries. Thank God we didn't use this beyond the few test calls. SMH! so, to everyone out there, the greed is real. Go local. We are being nickeled and dimed to death with all these subscriptions for everything.

[pramitchoudhary]

sqlite-vss could be another option for default, maybe --> https://github.com/asg017/sqlite-vss
Planning to use that for one of the initiatives. Any other re-ranking strategy could be added on top maybe.

[LarryStewart2022]

We installed MIlvus on our local Docker server. Also, a good option.

[pramitchoudhary]

Ya, I suggested sqlite-vss since it's embedded, so there is no additional server overhead for quick experimenting.
Weaviate may have an embedded version soon as well i think.