-
Notifications
You must be signed in to change notification settings - Fork 244
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Quantized Neural compress model not generating expected results in AMD processor #1531
Comments
Hi @Bhuvaneswaran-R, thank you for using the tool. We actually didn't commit to support quantize on AMD processor. But we once inferenced some quantized models on AMD CPU which is generated on Intel Processor, and the accuracy is similar. https://github.com/intel/neural-compressor/blob/master/docs/source/validated_model_list.md#validated-onnx-qdq-int8-models-on-multiple-hardware-through-onnx-runtime |
Hi chensuyue, |
Oh, I see, thanks for your reply, I will track this issue. But fix the inference issue on AMD processor is not a high priority task for my team, we welcome the contribution from community to fix this issue. |
Hi Team,
I have converted a norma t5 small model to Onnx using onnxruntime 1.15.1, python =3.10.12 in Intel Processor and AMD processor but received different response! Please let me know how to use the same for AMD processors
Input Seq: translate English to German: The house is wonderful.
Results from Intel based processors:
###T5 Small Output
###Actual: Das Haus ist wunderbar.
ONNX Result: Das Haus ist wunderbar.
###Quantized: Das Haus ist wunderbar.
###Neural Compressor: Intel: Das Haus ist wunderbar.
Results from AMD based processors:
##AMD
###ONNNX: Das Haus ist wunderbar.
###Quantized: Das Haus ist wunderbar.
###Neural Compressor: DOMIEIE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE IE
Code used for conversion:
Quantization code with Calibration Data Loader/ Data Set
from neural_compressor import quantization
from neural_compressor.config import PostTrainingQuantConfig,TuningCriterion
from transformers import AutoModelForSeq2SeqLM
model="/t5-smallonnx"
conf = (
PostTrainingQuantConfig(approach="dynamic"
)) # default approach is "auto", you can set "dynamic":PostTrainingQuantConfig(approach="dynamic")
q_model = quantization.fit(
model=model,
conf=conf
)
q_model.save("/t5-small-neural-compressor.onnx")
Neural compressor version: 2.4.1
The text was updated successfully, but these errors were encountered: