Can awscli-sync ignore deleted files at the destination

[PackiarajSakkananGW]

We need to sync local directory (src) to s3 (dest). But we do not want to copy deleted files at the destination (s3, in this case) again, otherwise it would cause duplicate processing again. (may sound like primitive queue, due to that nature of the existing implementation we cannot use queue or other mechanism )

Is there a way to ignore deleted files at the destination like using delete-marker or something like that (assume destination is version enabled s3 bucket)

We could not find any option which does that, so we thought using custom sync strategies but we end up with our own version of awscli. so here are my few questions

1. Is there a way to ignore deleted files at the destination like using delete-marker or similar?
2. Does aws cli V2 support custom sync strategies?
3. if so, can custom sync strategies be used without having us to maintain our own version of cli?

Thanks a lot for your time & help!

[RyanFitzSimmonsAK]

Hi @PackiarajSakkananGW, thanks for reaching out. To clarify your ask, when you say "deleted files", do you mean files that you've already uploaded and then deleted from your S3 bucket, or something else? Thanks!

[PackiarajSakkananGW]

Yes @RyanFitzSimmonsAK, By "deleted file" I mean, I have already uploaded (as part of previous sync operation) and then deleted from destination S3 bucket

think as sync being run as job

[RyanFitzSimmonsAK]

If the current version of an object is a delete marker when you make a request, then the header x-amz-delete-marker: true is included in the response. You could try including / excluding files to upload using that header to determine if a file was deleted. Here is some relevant documentation that might be helpful.

I wasn't able to find any reference to custom sync strategies. Could you elaborate more on that?