Could fpm evolve into an "open-ecosystem" build tool?

[jalvesz]

Description

Today I can only use fpm for testing or small projects because of its closed-ecosystem approach which on one hand produces an internal cascade of dependencies and on the other hand does not propose an "easy" solution for integrating externally pre-compiled libraries.

I had one example here trying to compile h5fortran with a locally compiled HDF5 library (not installed globally on purpose) https://fortran-lang.discourse.group/t/how-to-use-fpm-with-external-pre-compiled-modules-and-libraries/2744/5?u=hkvzjal

And recently another similar question from another user here https://fortran-lang.discourse.group/t/fpm-does-not-find-an-external-module/5509/9

If fpm enables using libraries that were compiled locally but in an external folder it would open the door for a more broad adoption.

Possible Solution

No response

Additional Information

No response

[urbanjost]

It might be a flaw, it might be a feature but a general concept in the early versions was that anything not portable to another platform should not appear in the fpm.toml file except user-defined profiles ( which are still no implemented) . Perhaps an include or conditional processing of the fpm.toml file; or an option to specify different manifest files would be an improvement, but you can access local external libraries. Probably the best way is to add the library names in the fpm.toml file and then either include the directory where the library resides on as an fpm compiler option, or use options on your platform such as setting the LD_LIBRARY_PATH variable would let the loader find the file. How to do that depends on your system.

Perhaps there is a complication I am not aware of. I will try to look at your references shortly to see if I am missing something or if something was missed in the responses. In case it is not there, what platform (MacOS, MSWIndows, Linux, OpenBSD, ...) do you need this to work on?

[jalvesz]

what platform (MacOS, MSWIndows, Linux, OpenBSD, ...) do you need this to work on?

Regarding the platforms I work on Windows and Linux.

I tested already the local path dependencies to import another fpm project, that works like a charm. And when thinking about it, I have the feeling that it would be healthier that the default behavior of fpm would be to place dependencies not within build/dependencies folder, but in parallel to the current project folder. Just like when one installs a python package with PyPi or conda, all packages are installed within the site-packages folder. This is just regarding the install hierarchy.

Then comes the issue with non-fpm projects such as HDF5, as mentioned, I avoided installing it globally or making it visible through the system path because in general, I will handle other non-fpm dependencies in a similar way within local builds.

[urbanjost]

You can create a directory, lets call it "LINKS". install all your dependency packages there. Now in your other projects make a softlink to the LINKS directory at the top of the project, and use names in the fpm.toml file for local dependencies like LINKS/name
and it eliminates duplicating and pulling down projects. I have thinking something like that should be standard where you can specify a local "repository" of packages. Ends up that is very similar to what happens with registered packages. A directory is created local copies of your dependencies are placed, which can then be shared by all your projects. So I do not think it is a great leap to be able to specify a directory for unregistered packages used as dependencies as well. Perhaps the same location as the package registry uses. I think you should be able to use multiple fpm.toml files; one with local customizations. I think that would help with things like how to locate HDF files. user-defined profiles would help too. Some of those are very likely to become fpm features. Currently, I use response files when a lot of command options are required. So hopefully a multi-pronged solution will produce a better solution.