Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"undefined reference to `archive_read_new" #2113

Open
mickey298 opened this issue Feb 22, 2024 · 16 comments
Open

"undefined reference to `archive_read_new" #2113

mickey298 opened this issue Feb 22, 2024 · 16 comments

Comments

@mickey298
Copy link

mickey298 commented Feb 22, 2024

os

fresh new vm

uname -a
Linux archlinuxtest 6.7.5-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 17 Feb 2024 14:02:33 +0000 x86_64 GNU/Linux

seastar commit

f0b2971


configure and build

./configure.py --mode=release --enable-dpdk --prefix=/usr

ninja -C build/release

error:

ninja: Entering directory `build/release'
[104/351] Linking CXX executable tests/perf/fstream_perf
FAILED: tests/perf/fstream_perf 
: && /usr/bin/g++ -O2 -g -DNDEBUG  tests/perf/CMakeFiles/test_perf_fstream.dir/fstream_perf.cc.o -o tests/perf/fstream_perf  libseastar.a  /usr/lib/libboost_program_options.so  /usr/lib/libboost_thread.so  /usr/lib/libboost_chrono.so  /usr/lib/libboost_atomic.so  /usr/lib/libcares.so  /usr/lib/libfmt.so.10.2.0  /usr/lib/liblz4.so  -ldl  /usr/lib/libgnutls.so  -latomic  /usr/lib/libsctp.so  /usr/lib/libprotobuf.so  /usr/lib/libyaml-cpp.so  /usr/lib/libhwloc.so  /usr/lib/liburing.so  /usr/lib/libnuma.so && :
/usr/bin/ld: libseastar.a(dpdk.o): in function `firmware_read':
eal_firmware.c:(.text+0x2c6f2): undefined reference to `archive_read_new'
/usr/bin/ld: eal_firmware.c:(.text+0x2c706): undefined reference to `archive_read_support_format_raw'
/usr/bin/ld: eal_firmware.c:(.text+0x2c716): undefined reference to `archive_read_support_filter_xz'
/usr/bin/ld: eal_firmware.c:(.text+0x2c72e): undefined reference to `archive_read_open_filename'
/usr/bin/ld: eal_firmware.c:(.text+0x2c741): undefined reference to `archive_read_next_header'
/usr/bin/ld: eal_firmware.c:(.text+0x2c795): undefined reference to `archive_read_data'
/usr/bin/ld: eal_firmware.c:(.text+0x2c7c2): undefined reference to `archive_read_free'
/usr/bin/ld: eal_firmware.c:(.text+0x2c7e8): undefined reference to `archive_read_free'
collect2: error: ld returned 1 exit status
[106/351] Building CXX object tests/perf/CMakeFiles/test_perf_shared_token_bucket.dir/shared_token_bucket.cc.o
ninja: build stopped: subcommand failed.

issue still exists after installing libarchive.

@tchaikov

@tchaikov
Copy link
Contributor

if we enable dpdk when building seastar, dpdk pulls libarchive in if the latter is installed in the system. and we do not use libdpdk.pc for calculating the ldflags exposed by seastar library. because we want to statically link against libdpdk, we just pull the .a archives manually with a hardwired name list, but we missed the libraries linked by it.

@tchaikov
Copy link
Contributor

@mickey298 please add the information on how you configure seastar, and how dpdk is involved in the build.

@mickey298
Copy link
Author

@mickey298 please add the information on how you configure seastar, and how dpdk is involved in the build.
modified.

@mickey298
Copy link
Author

mickey298 commented Mar 1, 2024

I solved the problem temporarily by add ” target_link_libraries(seastar_private INTERFACE -larchive)" in the CMakeLists.txt. and continue building. an new issues appeared.

[1/16] Linking CXX executable apps/httpd/httpd
FAILED: apps/httpd/httpd
: && /usr/bin/g++ -O2 -g -DNDEBUG  apps/httpd/CMakeFiles/app_httpd.dir/demo.json.cc.o apps/httpd/CMakeFiles/app_httpd.dir/main.cc.o -o apps/httpd/httpd  libseastar.a  /usr/lib/libboost_program_options.so  /usr/lib/libboost_thread.so  /usr/lib/libboost_chrono.so  /usr/lib/libboost_atomic.so  /usr/lib/libcares.so  /usr/lib/libfmt.so.10.2.0  /usr/lib/liblz4.so  -ldl  /usr/lib/libgnutls.so  -latomic  /usr/lib/libsctp.so  /usr/lib/libprotobuf.so  /usr/lib/libyaml-cpp.so  /usr/lib/libhwloc.so  /usr/lib/liburing.so  /usr/lib/libnuma.so  -larchive && :
/usr/bin/ld: libseastar.a(prometheus.cc.o): undefined reference to symbol '_ZN4absl12lts_2023080212log_internal17MakeCheckOpStringIllEEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEET_T0_PKc'
/usr/bin/ld: /usr/lib/libabsl_log_internal_check_op.so.2308.0.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
[3/16] Linking CXX executable apps/io_tester/io_tester
ninja: build stopped: subcommand failed.

confirm the file the symbol:

❯ ls /usr/lib/libabsl_log_internal_check_op.so -alh                                                       
lrwxrwxrwx 1 root root 41 Sep 19 04:50 /usr/lib/libabsl_log_internal_check_op.so -> libabsl_log_internal_check_op.so.2308.0.0


❯ nm -D /usr/lib/libabsl_log_internal_check_op.so |rg "_ZN4absl12lts_2023080212log_internal17MakeCheckOpStringIllEEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEET_T0_PKc"
0000000000003b50 T _ZN4absl12lts_2023080212log_internal17MakeCheckOpStringIllEEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEET_T0_PKc

it seems a protobuf version problem from these information. i didn't sovled this issue eventually .maybe i need to wait for the official update? if anyone solved this issue,pls let me know about it.
FindProtobuf does not look for abseil dependency
ONNX 1.15.0: missing symbols (absl) onnx_gtests #5740
Unable to link absl log internal

Thank you for your help. @tchaikov
Are you going to add this ( target_link_libraries(seastar_private INTERFACE -larchive)) into master branch? maybe a better way?
I hope close this issue soon If there are no other questions.

@tchaikov
Copy link
Contributor

tchaikov commented Mar 5, 2024

so there are two issues:

  1. libarchive linkage pulled in by dpdk
  2. absl libraries linkage pulled in by protobuf >= 22.0

regarding the 2nd one, before CMake upstream fixes it, we need to find a workaround.

tchaikov added a commit to tchaikov/seastar that referenced this issue Mar 5, 2024
protobuf v22.0 is packaged by in modern distros, but the FindProtobuf
module shipped along with CMake fails to add the new dependency of
abseil libraries to the linkage. so Seastar fail to link if compiled
with protobuf v22.0 and up.

so, in this change, we try to find protobuf using its CMake package
configuration first, and fall back to the Findprotobuf module. because
find_package() runs in module mode, and fall back to the config mode,
and the protobuf's CMake configuration adds the abseil libraries
dependency correctly, we need to use config mode first.

Refs scylladb#2113
Signed-off-by: Kefu Chai <[email protected]>
avikivity pushed a commit that referenced this issue Mar 5, 2024
protobuf v22.0 is packaged by in modern distros, but the FindProtobuf
module shipped along with CMake fails to add the new dependency of
abseil libraries to the linkage. so Seastar fail to link if compiled
with protobuf v22.0 and up.

so, in this change, we try to find protobuf using its CMake package
configuration first, and fall back to the Findprotobuf module. because
find_package() runs in module mode, and fall back to the config mode,
and the protobuf's CMake configuration adds the abseil libraries
dependency correctly, we need to use config mode first.

Refs #2113
Signed-off-by: Kefu Chai <[email protected]>
@mickey298
Copy link
Author

@tchaikov Still had issues.
After building with same error message,i delete QUIET keyword and got this warning:

CMake Warning at cmake/SeastarDependencies.cmake:162 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version "2.5.0".

  The following configuration files were considered but not accepted:

    /usr/lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0
    /lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0

Call Stack (most recent call first):
  CMakeLists.txt:395 (seastar_find_dependencies)


-- Found Protobuf: /usr/lib/libprotobuf.so (found suitable version "4.25.3", minimum required is "2.5.0") 

At first,i thougt it's version problems. So i delete the version constraint(2.5.0),and continue build with this command:

./configure.py --mode=release --enable-dpdk --prefix=/usr

the configure works fine. and i got this error after building:

$ ninja -C build/release -j3
ninja: Entering directory `build/release'
[1/346] Generating gen/src/proto/metrics2.pb.h, gen/src/proto/metrics2.pb.cc
FAILED: gen/src/proto/metrics2.pb.h gen/src/proto/metrics2.pb.cc /home/warren/seastar/build/release/gen/src/proto/metrics2.pb.h /home/warren/seastar/build/release/gen/src/proto/metrics2.pb.cc 
cd /home/warren/seastar/build/release && /usr/bin/cmake -E make_directory /home/warren/seastar/build/release/gen/src/proto && --cpp_out=/home/warren/seastar/build/release/gen/src/proto -I/home/warren/seastar/src/proto /home/warren/seastar/src/proto/metrics2.proto
/bin/sh: line 1: --cpp_out=/home/warren/seastar/build/release/gen/src/proto: No such file or directory
[3/346] Generating catest.key
ninja: build stopped: subcommand failed.

PS.
provide my envrionment in case someone try to reproduce the issue:

$uname -a
Linux archlinuxtest 6.7.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 03 Mar 2024 00:30:36 +0000 x86_64 GNU/Linux

$gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/13.2.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --enable-languages=ada,c,c++,d,fortran,go,lto,m2,objc,obj-c++ --enable-bootstrap --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --with-build-config=bootstrap-lto --with-linker-hash-style=gnu --with-system-zlib --enable-__cxa_atexit --enable-cet=auto --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-default-ssp --enable-gnu-indirect-function --enable-gnu-unique-object --enable-libstdcxx-backtrace --enable-link-serialization=1 --enable-linker-build-id --enable-lto --enable-multilib --enable-plugin --enable-shared --enable-threads=posix --disable-libssp --disable-libstdcxx-pch --disable-werror
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 13.2.1 20230801 (GCC)

seastar version: commit e6799055c8cacd2dc15c3037c7181831b2e48d79

tchaikov added a commit to tchaikov/seastar that referenced this issue Mar 6, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we have to specify a range of versions which is acceptable
by Seastar. please note Protobuf is using an interesting versioning:
the library is versioned with v21.x, etc. while the C++ APIs are
versioned like v3.21.x, v4.22, etc. if we find the package using
config mode, we search with its library version, otherwise we
search using its C++ API version.

Refs scylladb#2113
Signed-off-by: Kefu Chai <[email protected]>
@mickey298
Copy link
Author

Warining @ddef7a2
-- Found yaml-cpp: /usr/lib/libyaml-cpp.so (Required is at least version "0.5.1")
CMake Warning (dev) at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:447 (message):
find_package() specify a version range but the module Protobuf does not
support this capability. Only the lower endpoint of the range will be
used.
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindProtobuf.cmake:749 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
cmake/SeastarDependencies.cmake:169 (find_package)
CMakeLists.txt:395 (seastar_find_dependencies)
This warning is for project developers. Use -Wno-dev to suppress it

tchaikov added a commit to tchaikov/seastar that referenced this issue Mar 6, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we have to specify a range of versions which is acceptable
by Seastar. please note Protobuf is using an interesting versioning:
the library is versioned with v21.x, etc. while the C++ APIs are
versioned like v3.21.x, v4.22, etc. if we find the package using
config mode, we search with its library version, otherwise we
search using its C++ API version.

Refs scylladb#2113
Signed-off-by: Kefu Chai <[email protected]>
tchaikov added a commit to tchaikov/seastar that referenced this issue Mar 6, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we have to specify a range of versions which is acceptable
by Seastar. please note Protobuf is using an interesting versioning:
the library is versioned with v21.x, etc. while the C++ APIs are
versioned like v3.21.x, v4.22, etc. if we find the package using
config mode, we search with its library version, otherwise we
search using its C++ API version. see also
https://protobuf.dev/support/version-support/

Refs scylladb#2113
Signed-off-by: Kefu Chai <[email protected]>
@tchaikov
Copy link
Contributor

tchaikov commented Mar 6, 2024

@mickey298 while you were testing. i just sent a fix for it.

@mickey298
Copy link
Author

@mickey298 while you were testing. i just sent a fix for it.

Ok. I was in a hurry and i will testing it again after a few minutes.

@tchaikov
Copy link
Contributor

tchaikov commented Mar 6, 2024

@mickey298 thank you! i don't have an environment for testing the protobuf cmake configuration file at this moment. neither of my linux boxes have protobuf packaged with the configuration file installed: debian sid and f38. would be great if you could test again after removing QUIET if the latest revision does not work for you.

@mickey298
Copy link
Author

@mickey298 thank you! i don't have an environment for testing the protobuf cmake configuration file at this moment. neither of my linux boxes have protobuf packaged with the configuration file installed: debian sid and f38. would be great if you could test again after removing QUIET if the latest revision does not work for you.

My pleasure. Have you finished?

@tchaikov
Copy link
Contributor

tchaikov commented Mar 6, 2024

@mickey298 yup, i've pushed the latest revision to #2132.

@mickey298
Copy link
Author

mickey298 commented Mar 6, 2024

@mickey298 yup, i've pushed the latest revision to #2132.

I got this.

CMake Warning at cmake/SeastarDependencies.cmake:162 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version range "22.0...25.3".

  The following configuration files were considered but not accepted:

    /usr/lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0
    /lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0

Call Stack (most recent call first):
  CMakeLists.txt:395 (seastar_find_dependencies)


-- Found Protobuf: /usr/lib/libprotobuf.so (found suitable version "4.25.3", minimum required is "2.5.0")

So i modified to 22.0...25.3.0,and i got this:

CMake Warning at cmake/SeastarDependencies.cmake:162 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version range "22.0...25.3.0".

  The following configuration files were considered but not accepted:

    /usr/lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0
    /lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0

Call Stack (most recent call first):
  CMakeLists.txt:395 (seastar_find_dependencies)


-- Found Protobuf: /usr/lib/libprotobuf.so (found suitable version "4.25.3", minimum required is "2.5.0")

The same error with 22.0.0...25.3.0.
But
Only use 25.3.0, compile succeesed.

Do you feel this is too strict? What if the version of protobuf upgrade to 25.4.0? Can we have a better way? @tchaikov

tchaikov added a commit to tchaikov/seastar that referenced this issue Mar 6, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we should not specify a version number which is acceptable
by Seastar as a parameter of `find_package()`. and instead, we just
check the version number after calling `find_package()` in config mode.
if we find the package using config mode, we search it again, for
printing out the version number to the stdout, otherwise we search
using its C++ API version.

Refs #scylladb#2113

Signed-off-by: Kefu Chai <[email protected]>
tchaikov added a commit to tchaikov/seastar that referenced this issue Mar 6, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we should not specify a version number which is acceptable
by Seastar as a parameter of `find_package()`. and instead, we just
check the version number after calling `find_package()` in config mode.
if we find the package using config mode, we search it again, for
printing out the version number to the stdout, otherwise we search
using its C++ API version.

Refs #scylladb#2113

Signed-off-by: Kefu Chai <[email protected]>
@tchaikov
Copy link
Contributor

tchaikov commented Mar 6, 2024

@mickey298 thanks. now i understand the problem better. and yeah, it's too strict. so i revised the patch again. see #2132 . could you take another look?

@mickey298
Copy link
Author

from 0d4499
No warning.No error. Perfect result. Thank you for your hard work. @tchaikov

@tchaikov
Copy link
Contributor

tchaikov commented Mar 6, 2024

@mickey298 thank you for testing and the great analysis! without your help, i would not be able to come up with this fix =)

avikivity added a commit that referenced this issue Mar 13, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we should not specify a version number which is acceptable
by Seastar as a parameter of `find_package()`. and instead, we just
check the version number after calling `find_package()` in config mode.
if we find the package using config mode, we search it again, for
printing out the version number to the stdout, otherwise we search
using its C++ API version.

Refs ##2113

Closes #2132

* github.com:scylladb/seastar:
  build: relax the version constraint for Protobuf
  build: use protobuf::protoc for the path to protoc compiler
graphcareful pushed a commit to graphcareful/seastar that referenced this issue Mar 20, 2024
protobuf v22.0 is packaged by in modern distros, but the FindProtobuf
module shipped along with CMake fails to add the new dependency of
abseil libraries to the linkage. so Seastar fail to link if compiled
with protobuf v22.0 and up.

so, in this change, we try to find protobuf using its CMake package
configuration first, and fall back to the Findprotobuf module. because
find_package() runs in module mode, and fall back to the config mode,
and the protobuf's CMake configuration adds the abseil libraries
dependency correctly, we need to use config mode first.

Refs scylladb#2113
Signed-off-by: Kefu Chai <[email protected]>
graphcareful pushed a commit to graphcareful/seastar that referenced this issue Mar 20, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we should not specify a version number which is acceptable
by Seastar as a parameter of `find_package()`. and instead, we just
check the version number after calling `find_package()` in config mode.
if we find the package using config mode, we search it again, for
printing out the version number to the stdout, otherwise we search
using its C++ API version.

Refs #scylladb#2113

Signed-off-by: Kefu Chai <[email protected]>
graphcareful pushed a commit to graphcareful/seastar that referenced this issue Mar 20, 2024
as higher versions of Protobuf does not claim to be compatible with
v2.5.0, so we should not specify a version number which is acceptable
by Seastar as a parameter of `find_package()`. and instead, we just
check the version number after calling `find_package()` in config mode.
if we find the package using config mode, we search it again, for
printing out the version number to the stdout, otherwise we search
using its C++ API version.

Refs #scylladb#2113

Closes scylladb#2132

* github.com:scylladb/seastar:
  build: relax the version constraint for Protobuf
  build: use protobuf::protoc for the path to protoc compiler
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants