Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON / buffer performance #111

Open
tap2k opened this issue Dec 12, 2023 · 1 comment
Open

JSON / buffer performance #111

tap2k opened this issue Dec 12, 2023 · 1 comment

Comments

@tap2k
Copy link

tap2k commented Dec 12, 2023

Are there any performance differences between using buffers and JSON strings to store embeddings?

FYI on node I am only able to get node-sqlite3 (and not better-sqlite) to work with JSON storage.

@asg017
Copy link
Owner

asg017 commented Dec 12, 2023

Whether you insert vectors as buffers or JSON, it'll be stored in the same way, so there's no storage-related benefit either way. There may be some small CPU perf improvement if you use buffers instead (since it doesn't need to serialize/deserialize JSON), but you probably won't notice much of a difference.

Insert data with JSON in better-sqlite should work the same - do you mind sharing some sample code? Be sure to call JSON.stringify() before passing in vectors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants