aws s3 sync local -> s3bucket eats up swap while enough physical free memory is left

[KinWaiCheung]

Hi,
has anyone face issued with aws s3 sync from local to s3bucket and eats up swap memory?

Even though there is enough physical memory left?

aws-cli/2.11.23

every week I schedule a job to sync backup files from local to s3 bucket and the swap memory is decreased a couple hundred of MB each week.

s3 is configured with 20 concurrent requests
file size ranges from 10GB to 70GB each and total amount are in the 4TB

free -m
total used free shared buff/cache available
Mem: 515215 424992 75334 3634 14888 82955
Swap: 3999 3482 517

[tim-finnigan]

Linking related issue: #7937. We probably want to close either the issue or this discussion in order to consolidate discussion in one place.

[KinWaiCheung]

closed and check the related issue

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.