Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vcpkg-tool] Parallelise binary caching operations #38404

Open
Crzyrndm opened this issue Apr 25, 2024 · 0 comments
Open

[vcpkg-tool] Parallelise binary caching operations #38404

Crzyrndm opened this issue Apr 25, 2024 · 0 comments
Assignees
Labels
category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly

Comments

@Crzyrndm
Copy link

Crzyrndm commented Apr 25, 2024

With a remote binary cache, the current implementations can take a notable period of time for each individual package operation. As an example, in my experience, the AWS S3 backend typically requires approx. 2-5 seconds per package. A solid chunk of this time is networking latency and other delays that can be effectively reduced by performing multiple operations in parallel (tasks/threads).

This would be a fairly simple transformation (or at least as simple as anything involving threading gets) to using parallel_for_each or parallel_transform in place of the current for loops for the caching tools/operations which support parallelisation

@JonLiu1993 JonLiu1993 self-assigned this Apr 25, 2024
@JonLiu1993 JonLiu1993 added the category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly label Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly
Projects
None yet
Development

No branches or pull requests

2 participants