Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Are there any tools that can serve AQLM quantized models? #44

Closed
Mayorc1978 opened this issue Mar 9, 2024 · 4 comments
Closed

Are there any tools that can serve AQLM quantized models? #44

Mayorc1978 opened this issue Mar 9, 2024 · 4 comments
Labels
stale

Comments

@Mayorc1978
Copy link

Are Ollama or other frameworks or tools capable of serving AQLM quantized models?
@BlackSamorez
Copy link
Collaborator

text-generation-webui should support it in the dev branch.
vllm integration is on it's way as well.

@remiconnesson
Copy link

lorax on it's way too

@Mayorc1978 Mayorc1978 mentioned this issue Apr 5, 2024
Closed
@github-actions GitHub Actions
Copy link

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale label Apr 22, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented May 6, 2024

This issue was closed because it has been inactive for 14 days since being marked as stale.

@github-actions github-actions bot closed this as completed May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@BlackSamorez @remiconnesson @Mayorc1978