gleam publish: Get all .gleam files from src, do not using gitignore

[lpil]

Discussed in #3144

^{Originally posted by RudolfVonKrugstein May 16, 2024}
Hi,

I have a problem with publishing and then using it on hex.

• I created this project: https://github.com/RudolfVonKrugstein/gleam_processgroups
• I tested to import the package using "path" form another package, it worked!
• I run gleam publish and successfully published it: https://hex.pm/packages/processgroups
• I added it to another project: gleam add processgroups
• I tried to use it: import processgroups as pg
• gleam does not accept it:

error: Unknown module
  ┌─ /home/nathan/projects/private/pubsub/src/pubsub.gleam:5:1
  │
5 │ import processgroups as pg
  │ ^^^^^^^^^^^^^^^^^^^^^^^^^^ Did you mean `process`?

No module has been found with the name `processgroups`.

I have no Idea why.

When I publish, I notice that it only publishes build files, not the source files:

Generated files:
  - include/processgroups_GroupMonitor.hrl
  - include/processgroups_ProcessJoined.hrl
  - include/processgroups_ProcessLeft.hrl
  - src/processgroups.app.src
  - src/processgroups.erl

Source files:
  - LICENSE
  - README.md
  - gleam.toml

That might be correct, but looking at other packages it seemed to me that hey also publish the .gleam files ...

Hi @RudolfVonKrugstein! Opening an issue for the bug you found.

Do you know how to reproduce this problem? What version of Gleam are you using?

[RudolfVonKrugstein]

Hi,
Thanks for opening the ticket. My gleam version is:

❯ gleam --version
gleam 1.1.0

The steps I described worked for me twice to produce the issue. I feel a littel hesitant to try again, since the packages are not deleteable from hex.pm. But if there something else I should try, let me know!

Should gleam have outputed the gleam source file in the under "Source files" when I run gleam publish? If so, that is easily reproducablea. I tried to run gleam publish on https://github.com/massivefermion/radish and it also did not list me the source file (I stopped the process after gleam listed the files).

[lpil]

Thank you.

since the packages are not deleteable from hex.pm.

Hex isn't immutable for the first 24 hours so you can use the nightly version of Gleam with the command gleam hex revert --package packagename --version 1.2.3 to remove a version.

Should gleam have outputed the gleam source file in the under "Source files" when I run gleam publish? If so, that is easily reproducablea. I tried to run gleam publish on https://github.com/massivefermion/radish and it also did not list me the source file (I stopped the process after gleam listed the files).

It should, yes. Are you finding this happens for all packages for you?

[RudolfVonKrugstein]

Yes, I did try it for different packages:

radish

> gleam publish
...
Generated files:                                                                                                                                                                                                                                              - src/radish.app.src                                                                                                                                                                                                                                        - src/radish.erl                                                                                                                                                                                                                                            - src/[email protected]                                                                                                                                                                                                                                     - src/[email protected]                                                                                                                                                                                                                                    - src/radish@[email protected]                                                                                                                                                                                                                               - src/radish@[email protected]                                                                                                                                                                                                                               - src/radish@[email protected]                                                                                                                                                                                                                                - src/radish@command@sorted_set.erl                                                                                                                                                                                                                         - src/[email protected]                                                                                                                                                                                                                                    - src/[email protected]                                                                                                                                                                                                                                    - src/[email protected]                                                                                                                                                                                                                                      - src/[email protected]                                                                                                                                                                                                                                       - src/[email protected]                                                                                                                                                                                                                                       - src/[email protected]                                                                                                                                                                                                                                       - src/[email protected]                                                                                                                                                                                                                                        - src/radish@sorted_set.erl                                                                                                                                                                                                                                 - src/[email protected]                                                                                                                                                                                                                                        - src/[email protected]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Source files:                                                                                                                                                                                                                                                 - LICENSE                                                                                                                                                                                                                                                   - README.md                                                                                                                                                                                                                                                 - gleam.toml                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Name: radish                                                                                                                                                                                                                                                Version: 0.15.0  

aw4_request:

Generated files:
  - src/radish.app.src
  - src/radish.erl
  - src/[email protected]
  - src/[email protected]
  - src/radish@[email protected]
  - src/radish@[email protected]
  - src/radish@[email protected]
  - src/radish@command@sorted_set.erl
  - src/[email protected]
  - src/[email protected]
  - src/[email protected]
  - src/[email protected]
  - src/[email protected]
  - src/[email protected]
  - src/[email protected]
  - src/radish@sorted_set.erl
  - src/[email protected]
  - src/[email protected]

Source files:
  - LICENSE
  - README.md
  - gleam.toml

Name: radish
Version: 0.15.0

httpc:

Generated files:
  - include/gleam@httpc_Builder.hrl
  - src/[email protected]
  - src/gleam_httpc.app.src

Source files:
  - LICENSE
  - README.md
  - gleam.toml

Name: gleam_httpc
Version: 2.1.2

I also tried it with gleam 1.1.0, nightly and 1.0.0.

So I guess it must have to do something with my environment?

I am on WSL2, I don't know if that might be related ...

[RudolfVonKrugstein]

I discovered something.

I compiled gleam locally, trying to find the source of the behavior.

I I remove the require_git here it works.

I don't get why, and will investigate further :)

[RudolfVonKrugstein]

Ok, I get it and it has to do with my specific setup.

I use a git project manger (gws) to mange my projects and as a result my directory structure look looks like this:

projects:

• .gitignore
• .git/...
• processgroups/
  • .gitignore
  • .git/...
  • src/...

So in short, I have a git project above all my projects, which has a .gitignore that basically ignores all files.

For every project I have a git project in the specific directory.

For most tools, that is no problem, as they only care about the first git project they find when going up.

But gleam (or the rust function require_git) seems to not only look at the first git project it finds when walking up, but also my "project managing" git project. Removing the .gitignore up there fixes the problem.

Now, in my opinion that is a bug, as gleam publish should not care about any git repository above the projects git repository (and also because it breaks my workflow :) ). But I might not see something important ...

[lpil]

I'm not sure I agree that this is a bug, that is how gitignore works and I don't think I want to invent our own semantics for that.

That said I don't know why we respect gitignore here in any case. We don't when compiling, so we shouldn't for publishing.

[RudolfVonKrugstein]

I agree, that if that is the way .gitignore works, gleam should work that way with gitignore, but is it really?

If I run git command in the processgroups directory (see my dir structure above), git completly ignores any .gitignore that is above the repository root directory.

And if git behaves this way, should a tool like gleam not also behave that way?

[lpil]

I thought that was how it worked, but apparently not. In that case please open an issue with that particular dependency.

[RudolfVonKrugstein]

Here: BurntSushi/ripgrep#2812

[RudolfVonKrugstein]

Ok, ripgrep says this behavior is intended. The options I see are:

1. Dont't use require_git(false) but require_git(true). In this case, .gitignores are not used when there is no git repository.
2. Call my workflow exotic and do nothing :).

I still think, gleam should treat .gitignore files the same way as git, and that is way I am suggestion option 1.

But I understand, that there might be different opinions :).