Could RAG API work with other vector databases?

[jinzishuai]

From what I can read from https://github.com/danny-avila/rag_api, only PostgreSQL/pgvector is supported.

I wonder if we could support other vector databases. Particularly, since LibreChat already uses MongoDB and there are many recommendations to use the Atlas managed MongoDB such as https://github.com/danny-avila/LibreChat/blob/main/docs/install/configuration/mongodb.md. I recently found out that Altas has a vector search feature https://www.mongodb.com/products/platform/atlas-vector-search and we've been using it for a recommendation system nicely. The appeal here is to have a single database system rather than two in order to run LibreChat. What do you guys think?

Additionally, I am very intereted in the Cassandra vector features for very serious workloads: https://cassandra.apache.org/doc/latest/cassandra/vector-search/concepts.html.

[danny-avila]

Yes it could. To truly have one database, you would have to use Atlas in the first place, and to benefit from vectors, some configurations need to be applied that I'm not sure could be applied programmatically. The local MongoDB Atlas features seem to be only for development purposes: https://www.mongodb.com/blog/post/introducing-local-development-experience-atlas-search-vector-search-atlas-cli

Meilisearch also supports vectors but would require a bit of a refactor since it works quite differently from psql/pgvector

Cassandra seems great, too.

But in my opinion, for most users especially those using a local mongodb setup, this is always going to be a double database integration, no matter the additional support we add.

I'm also much more keen to add PGVecto.rs and supabase support before anything else, and adding additional vector stores is not a priority at the moment.

[danny-avila]

So I guess based on your response, other vector DBs are possible but not a priority at the moment. If I manage to find time, I might experiment creating a fork of rag_api to work with other vector dbs. It looks like a much smaller code space than LibreChat and could be a fun project to learn more about Langchain and RAG.

@jinzishuai

Absolutely, I appreciate your interest in this. It is much smaller and will come easier if you are more familiar with python.

One thing to note about this, I'm using some "hacks" over langchain to ensure the processes run asynchronously, something we take for granted in JavaScript but is not natural to python.

Most of the inspiration was from this video guide, so it is worth watching before starting: https://www.youtube.com/watch?v=Arf7UwWjGyc

[jinzishuai]

Hi @danny-avila, could you please take a look at danny-avila/rag_api#21?
It is not meant to be merged yet but I want to show you the foundational work and please let me know if they make sense or not.
My plan is to fix APIs one by one. Do you have a list of apis/use-cases sorted by priorities? At least point me to some high level directions.
Also, we might want to discuss how to make it easy for users to configure which vector database to use.

[danny-avila]

Great work, thanks for doing that. I made a note about the collections potentially be shared with existing File collection, I think Mongo allows for this.

Ideally these work too, along with query/embed, in order of importance.
/ids
/documents (post/delete)
/documents/{id}/context
/query_multiple

What do you think, is env variable good enough to switch off vector stores? it doesn't seem necessary to allow multiple at once.

[danny-avila]

but it should be easy switch at any time, and migration between the two is not necessary, maybe that can be done through standalone scripts eventually

[gaocegege]

Hi

pgvecto.rs offers a compatibility mode with pgvector, as mentioned in the documentation available at https://docs.pgvecto.rs/usage/compatibility.html. This compatibility mode allows pgvecto.rs to work seamlessly with pgvector if it is supported.

[jinzishuai]

@danny-avila as I look into implementing the /ids API, I had some trouble understanding the use of custom_id in the pgvector database. Quick search indicates it is related to /query_multiple API but I haven't seen any call to this API in the LibreChat code. Interestingly, I got the embed/query APIs to work without using custom_id at all, ie, the MongoDB vector search collection doesn't even have this field:

My questions are

1. Do we need this custom_id field for MongoDB?
2. If yes, what is the expected pattern to generate this field?
3. Is the custom_id in pgvector equivalent to the file_id field in the MongoDB vector collection?

[danny-avila]

Thanks for looking into this.

1. no if it works without it
2. n/a
3. yes

custom_id allows for non-semantic querying via PSQL

[danny-avila]

I will add that as long as custom_id/file_id are interchangeable in the final outcome. You probably already know this but we shouldn't have separate routes for mongo/pgvector. The routes I do have specific to PSQL were only for testing and are not intended to be exposed.

[DanBtt]

Would using Ferretdb be an alternative to consolidate the databases somewhat? They say it "allows you to use MongoDB drivers seamlessly with PostgreSQL as the database backend". I think it's pretty fast and has a decent reputation too.

[jinzishuai]

@danny-avila danny-avila/rag_api#21 is now ready. Please take a look.