[python-package] Error in installation in github MacOS runners

[shreyasvinaya]

Description

I get the below error

error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [93 lines of output]
      INFO:root:running bdist_wheel
      /private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-qp939adj/lightgbm_50f3e107466e4082958a581d9539c590/setup.py:220: SetuptoolsDeprecationWarning: setup.py install is deprecated.
      !!
      
              ********************************************************************************
              Please avoid running ``setup.py`` directly.
              Instead, use pypa/build, pypa/installer or other
              standards-based tools.
      
              See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
              ********************************************************************************
      
      !!
        install.initialize_options(self)
      INFO:root:running build
      INFO:root:running build_py
      INFO:root:creating build
      INFO:root:creating build/lib
      INFO:root:creating build/lib/lightgbm
      INFO:root:copying lightgbm/callback.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/compat.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/plotting.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/__init__.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/engine.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/dask.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/basic.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/libpath.py -> build/lib/lightgbm
      INFO:root:copying lightgbm/sklearn.py -> build/lib/lightgbm
      INFO:root:running egg_info
      INFO:root:writing lightgbm.egg-info/PKG-INFO
      INFO:root:writing dependency_links to lightgbm.egg-info/dependency_links.txt
      INFO:root:writing requirements to lightgbm.egg-info/requires.txt
      INFO:root:writing top-level names to lightgbm.egg-info/top_level.txt
      INFO:root:reading manifest file 'lightgbm.egg-info/SOURCES.txt'
      INFO:root:reading manifest template 'MANIFEST.in'
      WARNING:root:no previously-included directories found matching 'build'
      WARNING:root:warning: no files found matching '*.so' under directory 'lightgbm'
      WARNING:root:warning: no files found matching '*.so' under directory 'compile'
      WARNING:root:warning: no files found matching '*.dll' under directory 'compile/Release'
      WARNING:root:warning: no files found matching '*.dll' under directory 'compile/windows/x64/DLL'
      WARNING:root:warning: no previously-included files matching '*.py[co]' found anywhere in distribution
      WARNING:root:warning: no previously-included files found matching 'compile/external_libs/compute/.git'
      INFO:root:adding license file 'LICENSE'
      INFO:root:writing manifest file 'lightgbm.egg-info/SOURCES.txt'
      INFO:root:copying lightgbm/VERSION.txt -> build/lib/lightgbm
      INFO:wheel:installing to build/bdist.macosx-11.0-arm64/wheel
      INFO:root:running install
      INFO:LightGBM:Starting to compile the library.
      INFO:LightGBM:Starting to compile with CMake.
      Traceback (most recent call last):
        File "/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-qp939adj/lightgbm_50f3e107466e4082958a581d9539c590/setup.py", line 95, in silent_call
          subprocess.check_call(cmd, stderr=log, stdout=log)
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['make', '_lightgbm', '-I/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-qp939adj/lightgbm_50f3e107466e4082958a581d9539c590/build_cpp', '-j4']' returned non-zero exit status 2.
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-qp939adj/lightgbm_50f3e107466e4082958a581d9539c590/setup.py", line 334, in <module>
          setup(name='lightgbm',
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 399, in run
          self.run_command("install")
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/Users/runner/miniconda3/envs/deepchem/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-qp939adj/lightgbm_50f3e107466e4082958a581d9539c590/setup.py", line 248, in run
          compile_cpp(use_mingw=self.mingw, use_gpu=self.gpu, use_cuda=self.cuda, use_mpi=self.mpi,
        File "/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-qp939adj/lightgbm_50f3e107466e4082958a581d9539c590/setup.py", line 199, in compile_cpp
          silent_call(["make", "_lightgbm", f"-I{build_dir}", "-j4"], raise_error=True,
        File "/private/var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/pip-install-qp939adj/lightgbm_50f3e107466e4082958a581d9539c590/setup.py", line 99, in silent_call
          raise Exception("\n".join((error_msg, LOG_NOTICE)))
      Exception: An error has occurred while building lightgbm library file
      The full version of error log was saved into /Users/runner/LightGBM_compilation.log
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for lightgbm
ERROR: Could not build wheels for lightgbm, which is required to install pyproject.toml-based projects

Reproducible example

This error occurs when i try to install lightgbm using pip on github runner macos-large
scripts used:
https://github.com/deepchem/deepchem/blob/master/.github/workflows/build.yml
https://github.com/deepchem/deepchem/blob/master/scripts/install_deepchem_conda.sh

Environment info

LightGBM version or commit hash:

Command(s) you used to install LightGBM

pip install lightgbm

[jameslamb]

Thanks for using LightGBM.

I strongly suspect that pip is trying to install an old version of lightgbm, based on that setup.py warning in the logs you provided. lightgbm dropped its setup.py in v4.0.0 (#5837).

Please try forcing pip to install a newer version, and follow the suggestions from #5328 (comment) to build from source (as lightgbm does not yet publish wheels for arm64 macs).

pip install \
  --no-binary lightgbm \
  'lightgbm>=4.3.0'