golang no longer building on linux

[tundratim]

When attempting to install go to a non-default location in linux, I see this error:

fatal: unable to access 'https://go.googlesource.com/tools.git/': error setting certificate verify locations: CAfile: /home/linuxbrew/.linuxbrew/etc/[email protected]/cert.pem CApath: /home/linuxbrew/.linuxbrew/etc/[email protected]/certs

I can work around this with:

git config --global http.sslVerify false
However, this was never necessary in the past. It feels like the git build logic has the default cert location wired into it.

UPDATE: This is specifically and only a problem with https based clones. ssh clones seems to work fine. So, it's the https certs that are causing the noise.

In this case, git was built from source with brew. If I uninstall it and use system git (in this case ubuntu) there is no problem - system git is finding whatever cert authority info it needs. So ... I'd surmise that this has something do do with how git gets built on linux.

[tundratim]

The more I poke at this, the more this looks like an [email protected] problem, but I have no clue how to fix

[gromgit]

What does brew info [email protected] output?

[tundratim]

`[test] dock4 ~>brew info [email protected]
Error: Please update your system cURL.
Minimum required version: 7.41.0
Your cURL version: 7.29.0
Your cURL executable: /usr/bin/curl
[email protected]: stable 1.1.1i
Cryptography and SSL/TLS Toolkit
https://openssl.org/
/opt/TundraWare/tools/Cellar/[email protected]/1.1.1i (8,403 files, 24.7MB) *
Built from source on 2020-12-21 at 00:13:19
/opt/TundraWare/tools/Library/Homebrew/shims/scm/git --version
From: https://github.com/Homebrew/linuxbrew-core/blob/HEAD/Formula/[email protected]
License: OpenSSL
==> Caveats
A CA file has been bootstrapped using certificates from the system
keychain. To add additional certificates, place .pem files in
/opt/TundraWare/tools/etc/[email protected]/certs

and run
/opt/TundraWare/tools/opt/[email protected]/bin/c_rehash
/usr/bin/curl --disable --globoff --show-error --user-agent Linuxbrew/2.6.2-214-g2ddb71a\ (Linux;\ x86_64\ 5.4.0-58-generic)\ curl/7.29.0 --header Accept-Language:\ en --retry 3 --max-time 5 https://formulae.brew.sh/api/formula-linux/[email protected]
==> Analytics
==> install (30 days)

Index	Name (with options)	Count	Percent
1	[email protected]	23,819	100.00%
==> install (90 days)
Index	Name (with options)	Count	Percent
-----:	--------------------------------------------------------------------------------------------------------------	-------:	--------:
1	[email protected]	75,485	100.00%
==> install (365 days)
Index	Name (with options)	Count	Percent
-----:	-------------------------------------------------------------------------------------------------------------	--------:	--------:
1	[email protected]	292,683	100.00%
==> install-on-request (30 days)
Index	Name (with options)	Count	Percent
-----:	---------------------------------------------------------------------------------------------------------------	------:	--------:
1	[email protected]	647	100.00%
==> install-on-request (90 days)
Index	Name (with options)	Count	Percent
-----:	---------------------------------------------------------------------------------------------------------------	------:	--------:
1	[email protected]	1,772	100.00%
==> install-on-request (365 days)
Index	Name (with options)	Count	Percent
-----:	---------------------------------------------------------------------------------------------------------------	------:	--------:
1	[email protected]	6,121	100.00%
==> build-error (30 days)
Index	Name (with options)	Count	Percent
-----:	---------------------------------------------------------------------------------------------------------------	------:	--------:
1	[email protected]
`

[gromgit]

And what do brew config and brew doctor output?

[jonchang]

My guess is that brew postinstall [email protected] failed, but you didn't notice the error. Try running that.

[tundratim]

Per @danielnachun brew reinstall -s curl seemed to resolve the issue for the moment, but I am not sure why that works.

[danielnachun]

I'm fairly sure this error is happening because curl is currently set to cellar :any, but it is not relocatable without some configuration by the user: Linuxbrew/legacy-linuxbrew#380 (comment). For now I think brew reinstall -s curl will fix your problem, but we should figure out how to deal with the cellar status of curl to avoid this type of issue.

[tundratim]

So upon further digging, I found the mechanism of failure. Even if you do brew install -s curl, it immediately goes about installing git as one of its dependencies. But git itself lists curl as a dependency and when IT install curl, it does not do so from source. So, merely adding -s to all install commands won't work. Sequencing is required:

brew -s install git
brew -s reinstall curl

Is there some sort of ENVVAR I could set to force brew to always install everything from source so that even dependent packages get properly built?

[carlocab]

Not at the moment, but this should be fixed in a recent PR. Let try to track it down no. (I misremembered regarding the PR I was thinking about.)

[danielnachun]

In my experience on a non-standard HOMEBREW_PREFIX on Linux I've not had any issues with relocatable bottles except this recent issue with curl. If you try to build a curl bottle locally it will tell you it is not relocatable, and also says the same in CI. I don't know why curl currently has cellar :any but that only showed up with 7.74. Up until then, curl was always built from source with a non-standard HOMEBREW_PREFIX. I think this recent change may been an accidental oversight. I already have a PR open for curl for an unrelated issue and will bring this up there.

[tundratim]

I've also found that after just pulling the brew repo and doing brew install gcc that a ton of stuff is suddenly getting built. For some reason, brew is not seeing that system curl and git are on its path and proceeds to build curl with the problems noted above. It's been a while since I built this project from scratch but I do not recall a simple gcc install - even at a nonstandard location invoking all this extra machinery.

[danielnachun]

It's probably doing that because your system curl is too old (see the warning in the output of brew info you posted above).

[tundratim]

RESOLVED: While the process above did expose curl not being properly installed from source, the underlying problem that exposed this also had to be addressed. The docker container being used for builds was using CentOS7. As @danielnachun pointed out, the versions of git and curl there were so far backleveled, brew kept trying to install its own which is when all the serial dependencies and failure to build from source consistently showed up.

The fix was to use a craftier approach to get around the broken curl and all is now well.