New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create an open source data layer #793
Comments
Only existing version (using chainlit version 0.7.0) can be found here Will Chainlit Devs assist in this effort or will this be 100% community-driven? |
@hayescode thanks for sharing this repo. They decided to rebuild a graphql API. A custom data layer needs:
This removes the need to stay compatible with the graphql API, the need to maintain a server reachable via your chainlit app. And it enables a direct connection to your DB. I'm looking into other chainlit tasks at the moment, but I'll be around to review PRs and help! |
I'm currently working on my own Data Layer which is based on the BaseDataLayer. Could you please specify which of the features would be mandatory in order to create a valid PR? |
Hi |
Hi, I'm implementing this PR: #796 |
@AndreasMarcec The best would be a complete implementation that overrides all methods from the |
@Rajatkhanna801 the best would be to start with the Authentication callback. This is a configuration on a per-app basis. No need for custom data layer if you just need MSAL Authentication. |
@tpatel |
@tpatel I need one help the SQLite database needs to create a user table is there is already predefined model structure for user table? |
@sandangel the update_thread function is an upsert. I agree it's weird everything else has a create/update/delete but not for threads.. |
@tpatel if you could share the DDL for the backend tables I think it would speed up each of our developments. Thanks! |
@hayescode I updated the code to mimic literalai client instead. It's working now, just need a few update on filter. |
@sandangel why did you do that? |
@hayescode I explained in the PR. |
@willydouhard @tpatel @constantinidan implementing a custom data layer is proving difficult with the intertwining of literalai in Will chainlit be refactored to natively support Chainlit functionality? |
I just opened a PR to add a Postgres custom data layer with ADLS support -> #825 |
Just opened a PR to add a simple file-based SQLite data layer -> #832 No need to set up extra database. By default, the data is persisted in chainlit.db in working dir. The user can use any SQLite database tool to view the DB. |
@tpatel @willydouhard Here's my PR for adding a dialect agnostic SQLAlchemy custom data layer. We've been getting more community contributions on this lately, can we move this to 'In Progress' or 'In Review'? |
Chainlit has a feature that enables you to store, analyze and persist your data with Literal.
You can also create your own custom data layer to store the data in your database.
The goal of this ticket is to create an implementation of a custom data layer using a database like postgres or redis.
You can see an example of a custom data layer in this test, although it doesn't connect to a database.
The text was updated successfully, but these errors were encountered: