[Bug]: Optimizer Fails on AMD Radeon Optimizing RX 6500 XT Unet on Stable Diffusion

[mousetail]

What happened?

I'm getting this error trying to run the stable diffusion example:

Optimizing vae_decoder
[2023-08-24 18:28:07,493] [INFO] [footprint.py:168:get_pareto_frontier] pareto frontier points: 3_OrtTransformersOptimization-2-b1a42f24996a64c47333128369a9eb21-gpu-dml {'latency-avg': 3116.94146}
[2023-08-24 18:28:07,494] [INFO] [engine.py:475:run_search] Output all 1 models
[2023-08-24 18:28:07,494] [INFO] [engine.py:318:run] No packaging config provided, skip packaging artifacts
Unoptimized Model : C:\Users\Admin\pypro\Olive\examples\directml\stable_diffusion\cache\models\2_OnnxConversion-bed9f6d6008fff0395cbb86a8e7378c9-53e9f77e1a62ba2aa899ddb5369d16c1-gpu-dml\model.onnx
Optimized Model   : C:\Users\Admin\pypro\Olive\examples\directml\stable_diffusion\cache\models\3_OrtTransformersOptimization-2-b1a42f24996a64c47333128369a9eb21-gpu-dml\model.onnx

Optimizing unet
[2023-08-24 18:28:17,346] [ERROR] [engine.py:763:_run_passes] Evaluation failed: [ONNXRuntimeError] : 1 : FAIL : D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\ExecutionProvider.cpp(896)\onnxruntime_pybind11_state.pyd!00007FFC766F0201: (caller: 00007FFC766F0C2F) Exception(2) tid(5a84) 887A0006 De GPU reageert niet op meer opdrachten, waarschijnlijk vanwege een ongeldige opdracht van de aanroepende toepassing.

[2023-08-24 18:28:17,347] [WARNING] [engine.py:307:run] Failed to run Olive on gpu-dml: [ONNXRuntimeError] : 1 : FAIL : D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\ExecutionProvider.cpp(896)\onnxruntime_pybind11_state.pyd!00007FFC766F0201: (caller: 00007FFC766F0C2F) Exception(2) tid(5a84) 887A0006 De GPU reageert niet op meer opdrachten, waarschijnlijk vanwege een ongeldige opdracht van de aanroepende toepassing.

Seems to be the same issue as this: #301 But can provide any diagnostic info if needed.

Version?

Main branch as of 24-08-2023

[PatriceVignola]

Hi @mousetail,

How much dedicated RAM does your card have? There exists 2 versions of RX 6500 XT (4GB and 8GB), and I'm afraid 4GB is not enough to run stable diffusion on the GPU.

[mousetail]

I have the 8 GB variant