needs.<job_calling_reusable_workflow>.result is success when it actually failed due to error in parsing its input via fromJson

[akladiev]

Select Topic Area

Bug

Body

It seems that a specific kind of errors in inputs of reusable workflows are getting ignored in "needs" context, causing a wrong result reported here.

Example setup that can trigger the issue: one job produces an output in json format (key-value), and another job (that calls a reusable workflow), takes as an input a specific value from the output of the first job by retrieving it via fromJson (e.g. fromJson(needs.job1.outputs.output_name).key), like in the example below. The last job checks status of all previous jobs by checking needs.*.result values.

The issue occur when the value under fromJson is empty, e.g. due to a typo in the output name - in this case:

• the job calling a reusable workflow fails to be executed and is not displayed in GUI
• the workflow itself is marked as failed with an error in summary: The template is not valid. Error parsing fromJson ... Error reading JToken from JsonReader. Path '', line 0, position 0.
• but when we try to read the status of all jobs in the workflow from the "needs" context of the last job (which is always executed, regardless of the previous failures), the result of the problematic job is displayed as success.

Example workflow that triggers the issue
Here I purposefully provide the wrong output name in an input, and I expect it to fail the Overall_Status check:

jobs:
  job1:
    outputs:
      output: "${{ steps.set_output.outputs.output }}"
    runs-on: ubuntu-latest
    steps:
      - name: Set output
        id: set_output
        shell: bash
        run: |
          echo "output={'key': 'value'}" >> $GITHUB_OUTPUT

  job2:
    needs: [job1]
    uses: ./.github/workflows/job_reusable.yml
    with:
      example_input: ${{ fromJson(needs.job1.outputs.output_name_with_typo).key }}

  Overall_Status:
    needs: [job1, job2]
    if: ${{ always() }}
    runs-on: ubuntu-latest
    steps:
      - name: Show needs
        run: |
          echo "${{ toJson(needs) }}"

      - name: Check status of all jobs
        if: >-
          ${{
            contains(needs.*.result, 'failure') ||
            contains(needs.*.result, 'cancelled')
          }}
        run: exit 1

How it looks in GUI (note that job2 is not even displayed among executed, but still successful according to "needs" context):

Expected behavior

I'd expect the result of the job that is specified in "needs" block but failed to be executed for any reason to be "failure" or empty (definitely not "success"). This may look like a corner case (or maybe I'm doing something wrong?), and there may be workarounds to take this issue into account when checking the overall status, but still it doesn't look right