Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Sampling: Performing several samples per epoch only results in one sampled image #284

Open
Seith opened this issue May 5, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@Seith
Copy link

Seith commented May 5, 2024

What happened?

In the Sampling tab you can set up several samples to be generated per one epoch. However only the first sampled image appears in the Samples directory.

According to the console the other sample images seem to get generated (progress bar fills up normally) but they're never actually copied to the Samples directory. Only the first one is.

So for example at the end of a training with 2 samples per epoch and 10 epochs in total there should be 20 sampled images in the Samples folder, yet there are only 10.

What did you expect would happen?

A generated sampled image should appear in the Samples directory for each sample being performed.

Relevant log output

No response

Output of pip freeze

No response

@Seith Seith added the bug Something isn't working label May 5, 2024
@yggdrasil75
Copy link

there should be subdirectories created for each sample request, and the samples are named based on time of generation. commas are stripped as well as longer sample prompts, so if the difference between 2 samples is only after a few tags or only commas, then they get lumped together. if tensorflow only shows 1 image, that is probably why (it also only shows the latest for each training cycle for each sample)

@Seith
Copy link
Author

Seith commented May 6, 2024

Thank you for your reply!

The decision to have the samples work this way is rather confusing and tends to defeat the purpose of sampling in the first place, which is to quickly check at a glance how the training is going by seeing all the images next to each other (I'm not running a tensorboard).

Also what you're explaining about about the images sometimes getting lumped together in the same folder when the prompt gets longer (and cut-off in a similar way) adds to the confusion and seems to clearly highlight a flaw in that logic. It would be nice to address that at some point...

@Nerogar
Copy link
Owner

Nerogar commented May 6, 2024

Each sample should have a number as a prefix. Did this not work for you? Even if you have 10 samples with the same prompt, all of them should have a unique prefix.

@Seith
Copy link
Author

Seith commented May 6, 2024

If you mean a "0 - " and "1 - " yes, I do see them.

For example Kohya_SS gives that prefix to the images themselves so they can appear next to their siblings within the same sample folder. It just seems much more useful to me that way, but I'll have to adapt. Win some, lose some. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants