Add make target to download sources

[voltagex]

Having a make target that only downloads (and probably unpacks) sources would make caching builds in CI easier.

This is probably possible with the current Makefile but I am not familiar enough with makefile syntax

[jacobsvante]

Also looking for this!

Same here though with Makefile familiarity...

[ryanwoodsmall]

Easiest way to get everything you need is to run make extract_all TARGET=... ; make clean and tar up the sources/ directory, then untar during your build pipeline.

The file extraction is slow and noisy but will grab everything - you don't actually need to do this though, the sources can be pulled directly given a filename under sources/. If you use environment variables for versions, you can reference them in your config.mak or via make arguments.

musl-cross-make # musl-cross-make $ make sources/gcc-9.4.0.tar.xz TARGET=x86_64-linux-musl
mkdir -p sources
mkdir -p sources/gcc-9.4.0.tar.xz.tmp
cd sources/gcc-9.4.0.tar.xz.tmp && wget -c -O gcc-9.4.0.tar.xz https://ftpmirror.gnu.org/gnu/gcc/gcc-9.4.0/gcc-9.4.0.tar.xz
Connecting to ftpmirror.gnu.org (209.51.188.200:443)
Connecting to mirror.us-midwest-1.nexcess.net (208.69.120.125:443)
saving to 'gcc-9.4.0.tar.xz'
gcc-9.4.0.tar.xz     100% |*******************************************************************************************************************************************************************| 69.0M  0:00:00 ETA
'gcc-9.4.0.tar.xz' saved
cd sources/gcc-9.4.0.tar.xz.tmp && touch gcc-9.4.0.tar.xz
cd sources/gcc-9.4.0.tar.xz.tmp && sha1sum -c /usr/local/crosware/tmp/musl-cross-make/hashes/gcc-9.4.0.tar.xz.sha1
gcc-9.4.0.tar.xz: OK
mv sources/gcc-9.4.0.tar.xz.tmp/gcc-9.4.0.tar.xz sources/gcc-9.4.0.tar.xz
rm -rf sources/gcc-9.4.0.tar.xz.tmp

If you're doing CI, use environment variables; you generate a config.mak in that case as well; e.g., set your target, GCC version, etc. via the environment:

musl-cross-make # ( export GCC_VER=11.2.0 TARGET=x86_64-linux-musl ; make sources/gcc-${GCC_VER}.tar.xz )

This will pull an individual file, checksum, and drop it in the sources/ directory.

extract_all output:

musl-cross-make # test -e sources && find sources/ -type f ; make extract_all TARGET=x86_64-linux-musl &>/dev/null ; make clean ; ls -lA sources/
rm -rf gcc-* binutils-* musl-* gmp-* mpc-* mpfr-* isl-* build build-* linux-*
total 98492
-rw-r--r--    1 root     root      21490848 Apr 19 01:28 binutils-2.33.1.tar.xz
-rw-r--r--    1 root     root         36159 Apr 19 01:28 config.sub
-rw-r--r--    1 root     root      72411232 Apr 19 01:27 gcc-9.4.0.tar.xz
-rw-r--r--    1 root     root       2386766 Apr 19 01:28 gmp-6.1.2.tar.bz2
-rw-r--r--    1 root     root       1078912 Apr 19 01:28 linux-headers-4.19.88-1.tar.xz
-rw-r--r--    1 root     root        701263 Apr 19 01:28 mpc-1.1.0.tar.gz
-rw-r--r--    1 root     root       1652074 Apr 19 01:28 mpfr-4.0.2.tar.bz2
-rw-r--r--    1 root     root       1080786 Apr 19 01:28 musl-1.2.5.tar.gz

musl-cross-make # git status --ignored
On branch master
Your branch is up to date with 'origin/master'.

Ignored files:
  (use "git add -f <file>..." to include in what will be committed)
        sources/

nothing to commit, working tree clean

sources/ should survive a make clean but will deleted with a make distclean or git clean -fdx

• source download : https://github.com/richfelker/musl-cross-make/blob/master/Makefile#L56-L93
• extract_all : https://github.com/richfelker/musl-cross-make/blob/master/Makefile#L148

[jacobsvante]

Thanks for the thorough writeout @ryanwoodsmall, much appreciated!

I went with your suggestion to spell out each package version as environment variables, then running make sources/...tar.... for each. Works great!