TimeOutError in s3 download

[brettc]

Hi. I'm getting a TimeoutError when downloading a large file from s3 like this:

from aiobotocore.session import get_session
from pathlib import Path

session = get_session()
bucket = "some-bucket-name"
key = "some/key/name"
to_file = Path('some.file')

async with session.create_client("s3") as cl:
    obj = await cl.get_object(Bucket=bucket, Key=key)
    body = obj["Body"]
    with to_file.open("wb") as fd:
        async for chunk in body.iter_chunks(chunk_size=2**20):
            print(f'wrote {len(chunk)} bytes')
            fd.write(chunk)

The timeout happens at exactly 5 minutes, which is the default aiohttp timeout.
I can monkey patch my way around this by adding this code first:

from aiohttp import client
# Double the default timeout to 10 minutes
client.DEFAULT_TIMEOUT = client.ClientTimeout(total=10 * 60)

But there must be a nicer way to do it? I looked around in the code and cannot see how
to pass through a timeout to session creation. Did I miss something?

[thehesiod]

Via the botocore config object there's a timeout you can set

[brettc]

Thanks for the response. I take it you mean something like this:

    from botocore.client import Config
    config = Config(read_timeout=5)
    # then ...
    async with session.create_client("s3", config=config) as cl:
       # etc...

That doesn't have any effect. To clarify, I'm getting an asyncio.TimeoutError, not anything from boto. Here is the relevant bit of the stack trace:

  File "/Users/brett/Documents/Dragonfly/GISAID/moxie/src/moxie/build/_assets.py", line 100, in download_from_url
    async for chunk in resp.content.iter_chunked(DOWNLOAD_CHUNK_SIZE):

  File "/Users/brett/Library/Caches/pypoetry/virtualenvs/moxie-Ch9kj1Ys-py3.10/lib/python3.10/site-packages/aiohttp/streams.py", line 35, in __anext__
    rv = await self.read_func()

  File "/Users/brett/Library/Caches/pypoetry/virtualenvs/moxie-Ch9kj1Ys-py3.10/lib/python3.10/site-packages/aiohttp/streams.py", line 385, in read
    await self._wait("read")

  File "/Users/brett/Library/Caches/pypoetry/virtualenvs/moxie-Ch9kj1Ys-py3.10/lib/python3.10/site-packages/aiohttp/streams.py", line 303, in _wait
    with self._timer:

  File "/Users/brett/Library/Caches/pypoetry/virtualenvs/moxie-Ch9kj1Ys-py3.10/lib/python3.10/site-packages/aiohttp/helpers.py", line 720, in __exit__
    raise asyncio.TimeoutError from None

asyncio.exceptions.TimeoutError

[thehesiod]

ah, that's because when you're reading from the body that comes directly from the response of the underlying http session, which in this case is aiohttp. We do configure the aiohttp timeouts here: https://github.com/aio-libs/aiobotocore/blob/master/aiobotocore/httpsession.py#L126 which gets set from config here: https://github.com/aio-libs/aiobotocore/blob/master/aiobotocore/args.py#L78 btw I don't think you can retry a timed out aiohttp body request. Note that it should be read_timeout=5 * 60, 5 = 5 seconds.

can you compare against using botocore? You'll probably get a urllib3 read timeout...if you don't let me know as we may be missing an exception mapping layer.