Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate with rust-bert #111

Open
brainless opened this issue May 8, 2024 · 1 comment
Open

Integrate with rust-bert #111

brainless opened this issue May 8, 2024 · 1 comment
Assignees
@brainless
Labels
enhancement New feature or request integration Integration with AI, database or data providers rust Pull requests that update Rust code

Comments

@brainless
Copy link
Owner

As a user I would like to use local libraries to generate embedding. On their website rust-bert is introduced as:

Rust-native state-of-the-art Natural Language Processing models and pipelines. Port of Hugging Face’s Transformers library, using tch-rs or onnxruntime bindings and pre-processing from rust-tokenizers.

We can use this for classification, summarization or other utilities as well in the future.
@brainless brainless added enhancement New feature or request rust Pull requests that update Rust code labels May 8, 2024
@brainless brainless self-assigned this May 8, 2024
@brainless
Copy link
Owner Author

It would be nice to have this done for our MVP, as part of the work for #105.

@brainless brainless added the integration Integration with AI, database or data providers label May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brainless brainless

Labels
enhancement New feature or request integration Integration with AI, database or data providers rust Pull requests that update Rust code
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@brainless