Retry decorator hides exception types/info

[pipe2null]

Please adjust retry function decorator (src/ytdl_sub/utils/retry.py) and/or other error reporting to include the type of exception that is thrown.

Instead of:
"Exception thrown when attempting to run %s, attempt %d of %d",

something like:
try:
someFunction()
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).name, ex.args)
print message

etc...

I am debugging my setup/environment and consistently get exceptions after every single download completes while attempting to get avatar/banner, but without knowing what the actual exception is, it has been difficult to get to the bottom of the problem. For example:

[ytdl-sub] Exception thrown when attempting to run download_and_convert_url_thumbnail, attempt 1 of 3
[ytdl-sub] Exception thrown when attempting to run download_and_convert_url_thumbnail, attempt 2 of 3
[ytdl-sub] Exception thrown when attempting to run download_and_convert_url_thumbnail, attempt 3 of 3
[ytdl-sub:downloader] Failed to download thumbnail id 'avatar_uncropped'
[ytdl-sub] Exception thrown when attempting to run download_and_convert_url_thumbnail, attempt 1 of 3
[ytdl-sub] Exception thrown when attempting to run download_and_convert_url_thumbnail, attempt 2 of 3
[ytdl-sub] Exception thrown when attempting to run download_and_convert_url_thumbnail, attempt 3 of 3
[ytdl-sub:downloader] Failed to download thumbnail id 'banner_uncropped'

There are many possible sources of the exception, both within ytdl-sub as well as the FFMPEG installed in my environment. There are also some odd files getting generated in the /tmp directory (not my working directory). Not sure what is throwing the exception or what the exception actually is, but with the "wait_sec" time and consistent exceptions, it's adding 30 seconds to every download which adds up fast, and still not getting channel jpg's/metadata.

Thanks

[jmbannon]

The thumb URL is failing to download, and like you said, repeats on every vid. I will add something to only try once every 100 vids or something

[pipe2null]

Not sure what would be easier, your mitigation, or perhaps just exposing "times" and "wait_sec" ? Or just cut it down to 1 try at 2 sec timeout or something for the short term?

If it's the underlying ffmpeg or yt-dlp that has a bug causing exceptions, then thats out of your/our direct control, so let them keep throwing, and hopefully someday an update will fix it from underneath. It is the "(video count) * retrytimes * wait_sec * [banner+avatar [+ banner+avatar]]" that is the time killer. Sometimes it's 30 sec/vid attempting to fail to get avatar+banner, some channels for whatever reason take 60 sec/vid to fail to get avatar+banner+avatar+banner. And it's on every downloaded video.

I'm also seeing unexplained use of the /tmp directory for bits and pieces including a sanitized-string-name version of the working directory path that is a file and not a directory (thought everything was supposed to be stored in "working_directory"). Not sure if that is expected, or possibly related to some issue with my environment that could contribute to the avatar download problem?

Either way, including at least the string name of exceptions in messages/logs would be a good thing, verifies it's not a FileNotFound or permissions or weird network issue.

FYI, it took a bit for me to wrap my head around it, but using YAML to produce a layered config for yt-dlp is a pretty cool concept.