Errors not propagated when upload_file fails

[fidel-perez]

When s3.py -> upload_file fails, the errors are not propagated because of concurrent execution.
After reviewing the last repository changes, I believe this bug was introduced in the latest PR that was merged.

How to reproduce

• Install following the readme instructions
• Configure pointing towards an S3 bucket using wrong credentials or wrong URL or any other wrong configuration
• Run the target using a dummy input tap

You should get no errors, despite no file was uploaded to S3 (because of the miss configuration done on purpose).

I found the problem was in the upload_file method after trying adding this try/except to the code and finally being able to see where it was failing by using the print statement:

try:
    result = config['client'].upload_file(
        file_metadata['absolute_path'].as_posix(),
        config.get('s3_bucket'),
        file_metadata['relative_path'],
        **encryption_args)
except Exception as e:
    print(e)

[fidel-perez]

I just made some tests locally using the commit previous to the mentioned PR and can confirm the bug was introduced on that merge.

[ome9ax]

Hey @fidel-perez I've added a little change in the main branch to disable the ThreadPoolExecutor with the config option "thread_pool": false. Please let me know if that resolves the issue?

[fidel-perez]

Thank you for this. We stopped focusing on this repo in my job so I think we can close the issue with your recent fix!
I would recommend adding one line or two about this in the readme though :)