Parallel download of packages

[vitorgil]

I am trying out vcpkg to check whether its suitable for building a huge dependency list we have.
As a test, I ran:
vcpkg install collada-dom
which goes and installs about 40 packages in total, many of them from boost.

It works beautifully. But is there a way of doing parallelization of package downloading and eventually proceed with the build?
Also, is each of these packages being compiled with all cores? Can I configure the number of cores I want to use for building packages?

[MikeGitb]

How a package is build depends on the individual port, but in general, the preferred mechanism is ninja, which runs compilation in parallel. Not sure if there is a way to override that behavior.

Downloading one package, while a different one is processed is -as far as I know - not supported at the moment. The difficulty I see there is that downloading, extracting, building and installing all happen as part of a single cmake file, so vcpkg probably doesn't have a mechanism to say "start downloading but wait until I give the signal to proceed with compilation".

[florin-crisan]

While ninja generally uses all available cores, when installing a large number of packages there are a lot of times when only one core is used. It would be nice if unrelated packages could be installed in parallel.

[mrange]

Hi. I watched a twitch streamer trying to use vcpkg and boost (https://www.twitch.tv/garethhubball). Boost wanted to install 294 packages. Each package took about 2 secs meaning it took him 600 seconds or 10 minutes to install the dependencies. IMHO, that would be too slow for my taste.

[MikeGitb]

Well, most likely, the streamer didn't actually need all of boost. Still, it might be beneficial to have the ability to install all of boost in once go instead of by library.

[MikeGitb]

Which reminds me: Do any non-boost packages actually make use of the modular boost structure in their dependency list?

[tosone]

Does vcpkg now support parallel downloading and compiling packages?

[PhoebeHui]

AFAIK, vcpkg doesn't support parallel downloading, for one package, vcpkg builds it in parallel, however, it can't build several packages togather.

[MikeGitb]

As this has been moved to "discussions". Can I create a feature request for this feature (parallel download/unzipping of multiple packages) under "issues"?

[PhoebeHui]

@MikeGitb, please don't create a 'feature request' under 'issues', we would move all valid vcpkg feature requests and questions to disscusion.

[MikeGitb]

So issues are only for bug reports?

[Thomas1664]

Maybe it's possible to move information that is needed for downloading a package to the manifest file.
This would vcpkg enable to separate download from extraction/ build.

[dg0yt]

There is already a --only-downloads option. It does download, extraction and patching. But it doesn't enable parallel download and build.