Unable to get HCP FC datalad data in a pyenv with Python 3.12.2 ["you need 'boto' dependency which seems to be missing"]

[lilikapa]

What is the problem?

Hi Datalad community,
I am unable to get HCP FC files using datalad in pyenv with python 3.12.2 and the error messages point to an issue with the boto dependency, although boto is installed:

[Failed to download from any of 1 locations ["For using <class 'datalad.downloaders.s3.S3Downloader'> downloader, you need 'boto' dependency which seems to be missing"]

What steps will reproduce the problem?

pyenv virtualenv 3.12.2 datalad_test
pyenv activate datalad_test
pip install datalad
datalad clone https://github.com/datalad-datasets/hcp-functional-connectivity.git hcp_fc
cd hcp_fc
datalad get /path/to/hcp_fc/101309/MNINonLinear/Results/rfMRI_REST1_RL/rfMRI_REST1_RL_hp2000_clean.nii.gz

I actually intend to use the datalad.api within Python script, but this won't work either.

DataLad information

datalad --version: 1.02
git-annex version: 10.20230126

Additional context

When running pip list, I can see that boto is installed (version 2.49.0)

pip list
Package            Version     Editable project location
------------------ ----------- ---------------------------------------------------------------
annexremote        1.6.5
boto               2.49.0
certifi            2024.2.2
cffi               1.16.0
chardet            5.2.0
charset-normalizer 3.3.2
cryptography       42.0.6
datalad            1.0.2

But boto does not appear under the dependencies when running datalad wtf:

# WTF
## configuration <SENSITIVE, report disabled by configuration>
## credentials 
  - keyring: 
    - active_backends: 
      - PlaintextKeyring with no encyption v.1.0 at /home/lpaas/.local/share/python_keyring/keyring_pass.cfg
    - config_file: /home/lpaas/.config/python_keyring/keyringrc.cfg
    - data_root: /home/lpaas/.local/share/python_keyring
## datalad 
  - version: 1.0.2
## dataset 
  - branches: 
    - git-annex@dd8b9d5
    - master@238c274
  - id: 58f187ae-892d-11ea-b9e0-a0369f287950
  - path: /data/project/COIL/3_Data/hcp_fc
  - repo: AnnexRepo
## dependencies 
  - annexremote: 1.6.5
  - cmd:7z: 16.02
  - cmd:annex: 10.20230126
  - cmd:bundled-git: UNKNOWN
  - cmd:git: 2.39.2
  - cmd:ssh: 9.2p1
  - cmd:system-git: 2.39.2
  - cmd:system-ssh: 9.2p1
  - humanize: 4.9.0
  - iso8601: 2.1.0
  - keyring: 25.2.0
  - keyrings.alt: 5.0.1
  - msgpack: 1.0.8
  - platformdirs: 4.2.1
  - requests: 2.31.0
  [...]

Have you had any success using DataLad before?

Yes, I am able to get the files outside any virtual environment.

[yarikoptic]

yes, sorry about that -- we need to reincarnate/finish up

• RF: Drop boto for boto3 #7575
  which I apparently managed to miss to review etc. Will get to it now.

[yarikoptic]

overall it is a duplicate of

• downloaders/s3: migrate from boto (2.x) to boto3 #5597

but I will keep it open so you get alerted when "fixed" etc.

[lilikapa]

Hi Yarik, thank you so much for the work put on solving this issue! For my planning, do you maybe have a rough estimate of when this could be "fixed"? I am using a toolbox that for now works with python >= 3.12 and that is why I am a bit dependent on this version.