Need to run boto3 requests in parallel

[shantanugupta]

I am trying to run below code where each block pulls data from different AWS resources like ec2, rds etc. Here I have to perform some calculations using cloudwatch metrics. So I need to run each request in parallel. However, it seems like each request is running sequention.
I am not sure if boto3.Sessions allow requests to be sent in parallel or will it be sequential only.

await asyncio.gather(
asyncio.create_task(prepare_ec2_data(profile_name)),
asyncio.create_task(prepare_rds_data(profile_name)),
asyncio.create_task(prepare_ecs_data(profile_name))
)

[aBurmeseDev]

Hi @shantanugupta, thanks for reaching out.
Per Multithreading/Multiprocessing doc, it’s recommended to create a new session object for each process. Also, here’s a stackoverflow post that might be helpful to check out. Please let me know if you have any other questions!

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.