-
Notifications
You must be signed in to change notification settings - Fork 189
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Quantization via MLIR #1452
Comments
There are quantized tensor types in StableHLO spec, but looks like there are still discussions around it, for example openxla/stablehlo#1491. Also, that's only the specification, based on openxla/xla#9291 I infer it's not implemented in XLA yet. So on this level the best bet is to wait. There is also google/aqt, which is actually the only quantization reference/project I found for Jax/Flax. It lifts quantization one level up, by introducing a quantized tensor/config and reimplementing some operations with quantization in mind. Finally, it's worth noting that quantization is more broad, there are many approaches and algorithms, and both the use case and hardware plays a role. hf/transformers has a matrix of different techniques. |
No description provided.
The text was updated successfully, but these errors were encountered: