-
Notifications
You must be signed in to change notification settings - Fork 211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Roadmap (2024) #1121
Labels
enhancement
New feature or request
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
SDK
Rust
Description
The following is a rough roadmap of things we would like to tackle this year. There are no fixed dates and the roadmap is flexible, but it is a general indication of what people are working on. The roadmap is in rough priority order (higher priority items at the top) but outside the top items (which are actively being worked on) this is pretty fluid.
Unified API
Currently the rust, python, and node APIs are all completely independent. As a result, there is considerable divergence in available features. Python is the most complete, with support for things like hybrid search, and rust is the least complete, as it doesn't even have embedding support. We would like to migrate all SDKs onto one common (rust) base.
New Functionality
ML workloads need unique operations that aren't part of traditional database workflows. We have added support for those in the past (embeddings, hybrid search) and want to continue exploring what capabilities we can offer.
Client / server model
LanceDb currently runs embedded in the user's application. This lends itself to simpler architectures and solutions. However, there are some cases where a client / server model can be useful. For example, if the client is outside the datacenter or to develop ad-hoc clients in new languages that don't have rust support.
Easier Ingestion
Adding data into LanceDb properly can be complicated. Indices need to be updated, compaction needs to be run, inputs need to be batched. The default settings for LanceDb should make this easier.
Query API
Currently it is possible to plug LanceDb into an existing query engine (duckdb / datafusion / etc.) However, we can also pretty easily expose the underlying datafusion query engine as an SQL frontend to give users the ability to run SQL statements without needing another dependency.
The text was updated successfully, but these errors were encountered: