Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adopt polling API for uploading data in PutObject requests #874

Merged
merged 6 commits into from May 14, 2024

Conversation

passaro
Copy link
Contributor

@passaro passaro commented May 10, 2024

Description of change

Adopt the new polling API (poll_write()) to send data to the CRT for upload, resulting in:

  • improved write throughput, reverting the regression observed after Adopt new async write API for PutObject requests #832. The CRT stores data directly on buffers from its pool and avoids an additional copy.
  • simplified data ownership model. Contrary to the previous approach, poll_write() guarantees to never access the data slice after returning, so our wrapper can retain ownership and does not have to worry about cancelling the meta request.

Does this change impact existing behavior?

No


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

@passaro passaro temporarily deployed to PR integration tests May 10, 2024 08:25 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 10, 2024 08:25 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 10, 2024 08:25 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 10, 2024 08:25 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 10, 2024 08:25 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 10, 2024 08:25 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 10, 2024 08:25 — with GitHub Actions Inactive
@passaro passaro added the performance PRs to run benchmarks on label May 10, 2024
Copy link
Contributor

@arsh arsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

asking some questions

mountpoint-s3-client/src/s3_crt_client/put_object.rs Outdated Show resolved Hide resolved
mountpoint-s3-crt/src/s3/client.rs Outdated Show resolved Hide resolved
Signed-off-by: Alessandro Passaro <[email protected]>
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 10:30 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 10:30 — with GitHub Actions Inactive
@passaro passaro deployed to PR benchmarks May 14, 2024 10:30 — with GitHub Actions Active
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 10:30 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 10:31 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 10:31 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 10:31 — with GitHub Actions Inactive
@passaro passaro enabled auto-merge May 14, 2024 14:25
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 19:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 19:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 19:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 19:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 19:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 19:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests May 14, 2024 19:04 — with GitHub Actions Inactive
@passaro passaro added this pull request to the merge queue May 14, 2024
Merged via the queue into awslabs:main with commit 2a3a06f May 14, 2024
26 checks passed
@passaro passaro deleted the poll-write branch May 14, 2024 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance PRs to run benchmarks on
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants