Fix the bug of output_type=np or latent.

[yuanwu2017]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[regisss]

@yuanwu2017 Please provide a command or a script to reproduce the bug

[yuanwu2017]

@regisss
Please use the test script
Cuda:
python diffusers_test_output_type.py --device cuda
output log:

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:05<00:00,  1.75s/it]
if output_type is np, the output image should be = <class 'numpy.ndarray'>
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  4.93it/s]
if output_type is latent, the output image should be = <class 'torch.Tensor'>
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  5.22it/s]
if output_type is pil, the output image should be = <class 'list'>
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  5.12it/s]
if output_type is pt, the output image should be = <class 'torch.Tensor'>

HPU
python diffusers_test_output_type.py --device hpu
output log:

[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:24:03,698 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:24:03,698 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:53<00:00, 26.68s/it]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:24:57,073 >> Speed metrics: {'generation_runtime': 53.3576, 'generation_samples_per_second': 0.037, 'generation_steps_per_second': 0.037}
if output_type is np, the output image should be = <class 'list'>
[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:24:57,154 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:24:57,154 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  7.38it/s]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:24:57,446 >> Speed metrics: {'generation_runtime': 0.2714, 'generation_samples_per_second': 7.37, 'generation_steps_per_second': 7.37}
if output_type is latent, the output image should be = <class 'list'>
[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:24:57,448 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:24:57,448 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  7.08it/s]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:24:57,744 >> Speed metrics: {'generation_runtime': 0.2827, 'generation_samples_per_second': 7.074, 'generation_steps_per_second': 7.074}
if output_type is pil, the output image should be = <class 'list'>
[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:24:57,797 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:24:57,797 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  7.19it/s]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:24:58,090 >> Speed metrics: {'generation_runtime': 0.2786, 'generation_samples_per_second': 7.178, 'generation_steps_per_second': 7.178}
if output_type is pt, the output image should be = <class 'list'>

You can see that our image type always is list, but it is list for pil, tensor for pt and latent, numpy.ndarray for np, when running with Cuda device. It will make some pytest failed. For example,
diffusers testcase
The following error will happen.
AttributeError: 'list' object has no attribute 'shape'

HPU After applying the patch.
Output log:

[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:38:06,543 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:38:06,544 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:55<00:00, 27.60s/it]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:39:01,765 >> Speed metrics: {'generation_runtime': 55.1951, 'generation_samples_per_second': 0.036, 'generation_steps_per_second': 0.036}
if output_type is np, the output image should be = <class 'numpy.ndarray'>
[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:39:01,902 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:39:01,902 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  6.00it/s]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:39:02,255 >> Speed metrics: {'generation_runtime': 0.3339, 'generation_samples_per_second': 5.99, 'generation_steps_per_second': 5.99}
if output_type is latent, the output image should be = <class 'torch.Tensor'>
[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:39:02,256 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:39:02,256 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:12<00:00,  6.34s/it]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:39:14,958 >> Speed metrics: {'generation_runtime': 12.6856, 'generation_samples_per_second': 0.158, 'generation_steps_per_second': 0.158}
if output_type is pil, the output image should be = <class 'list'>
[INFO|pipeline_stable_diffusion.py:410] 2024-05-27 02:39:14,999 >> 1 prompt(s) received, 2 generation(s) per prompt, 1 sample(s) per batch, 2 total batch(es).
[WARNING|pipeline_stable_diffusion.py:415] 2024-05-27 02:39:14,999 >> The first two iterations are slower so it is recommended to feed more batches.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  6.09it/s]
[INFO|pipeline_stable_diffusion.py:591] 2024-05-27 02:39:15,342 >> Speed metrics: {'generation_runtime': 0.3287, 'generation_samples_per_second': 6.085, 'generation_steps_per_second': 6.085}
if output_type is pt, the output image should be = <class 'torch.Tensor'>

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[regisss]

LGTM!
Can you also run

pip install -U ruff
make style

please?

[yuanwu2017]

LGTM! Can you also run

pip install -U ruff
make style

please?

Done.