-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce Duckdb memory consumption #1127
Comments
it should also speedup opening new datasets |
https://duckdb.org/2024/01/26/multi-database-support-in-duckdb.html as per this recent blog post, a solution may be to construct a postgres cache table to take advantage of postgres's stronger on-disk index support, instead of relying on duckdb's indexing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the short term, we should have a single global db connection that can make unique view names for each dataset, and read multiple database files using
ATTACH
: https://duckdb.org/docs/sql/statements/attach.htmlThis will hopefully help with minimizing any caching that duckdb does for each different connection.
The text was updated successfully, but these errors were encountered: