"ELF: x: Error" on CentOS 7

[tueda]

I am trying to fresh-install Homebrew on CentOS 7 (with sufficiently up-to-date system cURL and Git). Currently, the post-install step of brew install gcc fails as

$ brew postinstall -vd gcc
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gcc.rb
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/shims/scm/git --version
==> Postinstalling gcc
Warning: The post-install step did not complete successfully
You can try again using:
  brew postinstall gcc
==> An exception occurred within a child process:
  ErrorDuringExecution: Failure while executing; `/home/linuxbrew/.linuxbrew/Cellar/gcc/10.2.0_4/bin/gcc-10 -print-libgcc-file-name` exited with 127. Here's the output:


/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils/popen.rb:12:in `popen_read'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils/popen.rb:16:in `safe_popen_read'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gcc.rb:184:in `post_install'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:1076:in `block (2 levels) in run_post_install'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:918:in `with_logging'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:1075:in `block in run_post_install'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils.rb:558:in `with_env'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:1064:in `run_post_install'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/postinstall.rb:22:in `<main>'

Actually, if I directly run gcc-10 then I get:

$ /home/linuxbrew/.linuxbrew/Cellar/gcc/10.2.0_4/bin/gcc-10
/home/linuxbrew/.linuxbrew/Cellar/gcc/10.2.0_4/bin/gcc-10: �o7��: ELF: x: Error 18446744072955217047

Is there any chance to avoid this error?

[maxim-belkin]

Hi @tueda. What does the following command return to you?

patchelf --print-interpreter /home/linuxbrew/.linuxbrew/Cellar/gcc/10.2.0_4/bin/gcc-10

[danielnachun]

Thanks @tueda for this additional info. At the very least we can rule out more obvious issues like linking to the wrong libraries or using the wrong linker. What is the setup for the this CentOS system? Virtual machine, desktop, server, etc? Is SELinux enabled or disabled?

[tueda]

The machine is used as a computation server (not a virtual machine). I have the root privilege and currently I am the main user, but people in our group used it in the past and may use it in future, so I installed Homebrew at /home/linuxbrew/.linuxbrew (rather than ~/.linuxbrew). SELinux is disabled:

$ sestatus
SELinux status:                 disabled

CentOS 7 was pre-installed when we bought the machine.

The old installation of Homebrew worked well on the machine except that it gave many warning messages about xorg like

Warning: Use libx11 instead of deprecated linuxbrew/xorg/libx11

Recently it has started to warn that the system curl and git are old. So, I thought it may be a good idea to try a clean installation (I have a backup of /home/linuxbrew/.linuxbrew so that I can switch back to the old installation). To install decent versions of curl and git, I used the city-fan.org repo and the IUS repo, respectively. The clean installation seemed to work until I found the post-install step in brew install gcc (and actually the same for perl, python, ruby) failed due to the aforementioned error.

Just in case, the kernel version is:

$ uname -a
Linux ************** 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[danielnachun]

You can ignore the xorg, curl and git warnings as they shouldn't have any bearing on this.

The clean installation seemed to work until I found the post-install step in brew install gcc (and actually the same for perl, python, ruby) failed due to the aforementioned error.

So you're seeing ELF errors with all binaries installed by Homebrew, not just GCC? That would make me suspect the brewed glibc installation was doing something weird. When you installed to /home/linuxbrew/.linuxbrew, glibc was probably installed by bottle instead of by source as it would have been when you installed to $HOME/.linuxbrew.

If you don't mind potentially breaking your system, could you try reinstalling glibc with brew reinstall -s -v glibc? To my knowledge the vast majority of Linux users who use brewed glibc are doing so because they don't have admin rights to upgrade their system to a newer distro with glibc > 2.23, so it's possible that glibc is rarely if every poured from a bottle (since doing so requires admin rights).

I'm very interested in trying to eventually make glibc a relocatable bottle so I'd like to understand what issues we may have with using from a bottle.

[tueda]

So you're seeing ELF errors with all binaries installed by Homebrew, not just GCC?

Indeed, for many binaries. But somehow patchelf doesn't give the error, though I see it is "Poured from bottle" and it's an "ELF 64-bit LSB executable".

If you don't mind potentially breaking your system, could you try reinstalling glibc with brew reinstall -s -v glibc?

I will try, but maybe a bit later.

Here are what file says for patchelf and gcc-5, respectively. (I'm not sure if the difference between GNU/Linux and SYSV has a problem.)

$ file /home/linuxbrew/.linuxbrew/Cellar/patchelf/0.12_4/bin/patchelf
/home/linuxbrew/.linuxbrew/Cellar/patchelf/0.12_4/bin/patchelf: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[sha1]=05c9bcce97c53ca1da46b742a5cb1f963f503e3c, not stripped

$ file /home/linuxbrew/.linuxbrew/Cellar/gcc\@5/5.5.0_6/bin/gcc-5
/home/linuxbrew/.linuxbrew/Cellar/gcc@5/5.5.0_6/bin/gcc-5: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, stripped

[tueda]

Here I write a test log using a Docker container (the Docker runs on CentOS 7, but I expect it is reproducible on other host OSes).

docker run -it --rm centos:7

yum update -y
yum groupinstall -y 'Development Tools'
yum install -y procps-ng curl file git

useradd test
mkdir -p /home/linuxbrew/.linuxbrew
chown -R test:test /home/linuxbrew
su -l test

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Because the system curl is very old, it seems that Homebrew tries to install the latest curl, but this doesn't work because of "ELF: x: Error". Log excerpt:

/bin/bash: line 268: which: command not found

/bin/bash: line 121: /usr/bin/sudo: No such file or directory

Error: Please update your system curl.
Minimum required version: 7.41.0
Your curl version: 7.29.0
Your curl executable: /usr/bin/curl
Error: Please update your system Git.
Minimum required version: 2.7.0
Your Git version: 1.8.3.1
Your Git executable: /usr/bin/git

Warning: [email protected] has been deprecated because it is a versioned formula!

Warning: The post-install step did not complete successfully
You can try again using:
  brew postinstall gcc@5

Error: 'curl' must be installed and in your PATH!
Failed during: /home/linuxbrew/.linuxbrew/bin/brew update --force --quiet

The bottle of gcc@5 is downloaded during the installation, but the executable doesn't run:

$ /home/linuxbrew/.linuxbrew/Cellar/gcc\@5/5.5.0_6/bin/gcc-5
/home/linuxbrew/.linuxbrew/Cellar/gcc@5/5.5.0_6/bin/gcc-5:  -��: ELF: x: Error 1655813271

[rohith-subramanyam]

Check #1357 (comment). I hit the same issue. I was able to figure out why. Answered it in the above discussion. I have also raised an issue which I have linked in the discussion.

[rohith-subramanyam]

Below is the answer from that discussion:
This was because the interpreter of binaries like git, gcc is /home/linuxbrew/.linuxbrew/lib/ld.so which is symlinked to system's ld, /lib64/ld-linux-x86-64.so.2 which is coming from system's glibc which is a different version (2.16) compared to the one within the linuxbrew environment (2.23).
Where as the interpreter for the other shared libraries is /home/linuxbrew/.linuxbrew/Cellar/glibc/2.23/lib/ld-linux-x86-64.so.2, which is glibc 2.23 version. Due to the different versions of glibc, it throws an ELF error when running the executable.
I was doing a fresh and clean install of linuxbrew on my CentOS machine and the installation of gcc@5 probably sets up these paths of the so files in lib directory. But, why is it linked against the system's ld?, which can be a of a different version and cause discrepancies. So, this needs to be fixed.

~% ldd /home/linuxbrew/.linuxbrew/bin/git
	linux-vdso.so.1 =>  (0x00007ffced196000)
	libpcre2-8.so.0 => /home/linuxbrew/.linuxbrew/opt/pcre2/lib/libpcre2-8.so.0 (0x00007f4570a68000)
	libz.so.1 => /home/linuxbrew/.linuxbrew/lib/libz.so.1 (0x00007f4570853000)
	libpthread.so.0 => /home/linuxbrew/.linuxbrew/lib/libpthread.so.0 (0x00007f4570636000)
	librt.so.1 => /home/linuxbrew/.linuxbrew/lib/librt.so.1 (0x00007f457042e000)
	libc.so.6 => /home/linuxbrew/.linuxbrew/lib/libc.so.6 (0x00007f457008f000)
	/home/linuxbrew/.linuxbrew/lib/ld.so => /lib64/ld-linux-x86-64.so.2 (0x00007f4570cf8000)
~% ldd /home/linuxbrew/.linuxbrew/lib/libc.so.6
	/home/linuxbrew/.linuxbrew/Cellar/glibc/2.23/lib/ld-linux-x86-64.so.2 (0x00007f9bf08cb000)
	linux-vdso.so.1 (0x00007fffa5b65000)

[rohith-subramanyam]

~% unlink $(brew --prefix)/lib/ld.so`
~% ln -s $(brew --prefix)/opt/glibc/lib/ld-linux-x86-64.so.2 $(brew --prefix)/lib/ld.so

should fix this.

[jaredsampson]

FWIW the unlink/ln commands above just worked for me to rescue functionality from an old linuxbrew installation that hadn't been used or updated in a while and suddenly started giving similar ELF errors.