Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependentTasksOutputFiles : different behavior with NX_NATIVE_TASK_HASHER=false #22253

Open
1 of 4 tasks
sloonz opened this issue Mar 10, 2024 · 5 comments · May be fixed by #23179
Open
1 of 4 tasks

dependentTasksOutputFiles : different behavior with NX_NATIVE_TASK_HASHER=false #22253

sloonz opened this issue Mar 10, 2024 · 5 comments · May be fixed by #23179
Assignees
Labels
community This is a good first issue for contributing scope: core core nx functionality type: bug

Comments

@sloonz
Copy link

sloonz commented Mar 10, 2024

Current Behavior

Overall setup : in my monorepo, I have a code-generator tool (let’s call it codegen). Some other projects use it to generate some code. Obviously, I want changes in codegen to result in the "codegen" task in the projects to be re-run. To achieve that behavior, I’m defining this in targetDefaults :

        "codegen": {
            "cache": true,
            "dependsOn": [{"projects": ["codegen"], "target": "build"}],
            "inputs": [{"dependentTasksOutputFiles": "**/*.js", "transitive": false}],
            "outputs": ["{projectRoot}/gen/**/*"]
        }

(build is standard, takes src/**/*.ts as input, output is dist/**/*.js)

If I run nx run project:codegen, then do some chage to codegen, then re-run nx run project:codegen, the behavior depends on the value of NX_NATIVE_TASK_HASHER :

  • If unset, the second codegen run will reuse cache
  • If set to false, the second codegen run will NOT reuse cache

Expected Behavior

Unless I did a mistake, it should not reuse cache in both cases. Even if I made a mistake somewhere, the two behaviors should match.

GitHub Repo

https://github.com/sloonz/nx-hasher-issue-repo

Steps to Reproduce

In the example repository, run nx run project:codegen, then do some change to codegen/src/main.ts (just increment the counter), the re-run nx run project:codegen.

Nx Report

NX   Report complete - copy this into the issue template

Node   : 21.7.0
OS     : linux-x64
npm    : 10.5.0

nx             : 18.0.8
@nx/workspace  : 18.0.8
@nx/devkit     : 18.0.8
@nrwl/tao      : 18.0.8
typescript     : 5.4.2

Failure Logs

No response

Package Manager Version

No response

Operating System

  • macOS
  • Linux
  • Windows
  • Other (Please specify)

Additional Information

No response

@sloonz
Copy link
Author

sloonz commented Mar 10, 2024

Either I’m drunk or some magic happened, but the instant I pressed "send issue", the bug disappeared, both in my reproduction repository and in my original one.

@sloonz sloonz closed this as completed Mar 10, 2024
@sloonz
Copy link
Author

sloonz commented Mar 15, 2024

I’m getting crazy, the issue is back.

Where is the native hasher implementation, so I can try to troubleshoot it ?

@sloonz sloonz reopened this Mar 15, 2024
@sloonz
Copy link
Author

sloonz commented Mar 15, 2024

Added some debug traces to hash_task_output.rs, got this :

output_files=["codegen/dist/main.d.ts", "codegen/dist/main.js", "codegen/dist/main.js.map"] glob="**/*.js"
hashes=[]

I’m confused, shouldn’t that glob (**/*.js) match codegen/dist/main.js ?

@sloonz
Copy link
Author

sloonz commented Mar 15, 2024

I finally found the issue and why it seemed to come and go.

In my example repository, codegen/dist/main.js indeed matches **/*.js as expected. The issue lies here : https://github.com/nrwl/nx/blob/master/packages/nx/src/native/tasks/hashers/hash_task_output.rs#L15

Since this is not resolved relative to workspace root, if I run nx codegen while in project/, hash_file does not find the file, and the resulting hashes is empty. If I run nx run project:codegen while in workspace root, hash_file works as intended and the bug disappears.

As a check, I can confirm that replacing

.filter_map(hash_file)

with

.filter_map(|file| hash_file(Path::new(workspace_root).join(file).to_str().expect("path contains invalid utf-8").to_owned()))

fixes the issue.

@FrozenPandaz FrozenPandaz added the scope: core core nx functionality label Mar 15, 2024
@FrozenPandaz
Copy link
Collaborator

Great debugging! It seems like you have a fix. Would you like to submit a PR with the fix?

@FrozenPandaz FrozenPandaz added the community This is a good first issue for contributing label Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community This is a good first issue for contributing scope: core core nx functionality type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants