Unexpected log entries regarding local variables

[mymasse]

We have just noticed the following entries in the concourse logs for which we never noticed before. Doesn't seem like this should happen since these are local variables and would only be known at build runtime.

concourse-web {"timestamp":"2021-11-16T17:41:59.533350427Z","level":"info","source":"atc","message":"atc.tracker-imb.run.finish.errored","data":{"build":"check","error":"resource config creds evaluation: undefined vars: .:velero-azure-plugin_version","pipeline":"orchestra","pre_build_id":4989289,"resource":"velero-azure-plugin-prod-update-repo","session":"23.4989287.4","team":"main"}}

We've also noticed on our Vault instance a lot more logs of request coming from Concourse since we upgraded to 7.6.0, has something changed?

[mymasse]

So reading up on the resource checker I wouldn't have expected these entries in the log since these are resource only used for put so not an input in my pipeline and the documentation mentions only resources used as input are checked. Could the behavior changed in 7.6.0 with this PR: #7208

[mymasse]

So I did some more tests comparing 7.5.0 and 7.6.0 with a simple pipeline with concourse running with docker-compose.

Pipeline:

---
resource_types:
  - name: registry-tag
    type: registry-image
    source:
      repository: ghcr.io/tlwr/registry-tag-resource
      tag: a711814d6f5b1c41e21e41ac90fd43550fc86f06

resources:
  - name: pipeline-tasks
    type: git
    icon: bitbucket
    source:
      uri: [email protected]:trecnoc/concourse.git
      branch: master
      private_key: ((bitbucket.private_key))
  - name: docker-repo
    type: git
    icon: bitbucket
    source:
      uri: [email protected]:trecnoc/docker.git
      branch: master
      private_key: ((bitbucket.private_key))
  - name: debian-release
    type: registry-tag
    icon: tag
    source:
      uri: https://hub.docker.com/v2/repositories/library/debian
      regexp: ^buster-\d+-slim$
      pages: 10
      tags_per_page: 10
      sort:
        method: alphabetical
  - name: debian-update-repo
    type: git
    icon: bitbucket
    source:
      uri: [email protected]:trecnoc/docker.git
      branch: azure-cli-image-update-debian-to-((.:debian_version))
      private_key: ((bitbucket.private_key))
jobs:
  - name: update-debian
    build_log_retention:
      builds: 2
      minimum_succeeded_builds: 1
    plan:
      - in_parallel:
          - get: pipeline-tasks
          - get: docker-repo
          - get: debian-release
      - load_var: debian_version
        file: debian-release/tag
      - task: update-dockerfile
        file: pipeline-tasks/tasks/search-and-replace-file.yml
        input_mapping:
          input-repo: docker-repo
        output_mapping:
          output-repo: debian-updated-repo
        params:
          FILE: azure-cli/Dockerfile
          SEARCH: ARG DEBIAN_VERSION=.*
          REPLACEMENT: ARG DEBIAN_VERSION=((.:debian_version))
          COMMIT_MESSAGE: "Update azure-cli image with debian version ((.:debian_version))"
          COMMIT_USER: ((bitbucket.user))
          COMMIT_EMAIL: ((bitbucket.email))
      - put: debian-update-repo
        params:
          repository: debian-updated-repo

Logs from 7.5.0 only checks input resources.

{"timestamp":"2021-11-17T15:31:52.199359921Z","level":"info","source":"atc","message":"atc.scanner.notify.created-build","data":{"build":"check","build_id":29,"pipeline":"test-new","resource":"pipeline-tasks","session":"22.11","team":"main"}}
{"timestamp":"2021-11-17T15:31:52.199578770Z","level":"info","source":"atc","message":"atc.scanner.notify.created-build","data":{"build":"check","build_id":30,"pipeline":"test-new","resource":"docker-repo","session":"22.11","team":"main"}}
{"timestamp":"2021-11-17T15:31:52.200338637Z","level":"info","source":"atc","message":"atc.scanner.notify.created-build","data":{"build":"check","build_id":31,"pipeline":"test-new","resource":"debian-release","session":"22.11","team":"main"}}

Logs from 7.6.0 check all resources

{"timestamp":"2021-11-17T15:33:26.347687918Z","level":"info","source":"atc","message":"atc.tracker-imb.run.running","data":{"build":"check","pipeline":"test-new","pre_build_id":1,"resource":"debian-update-repo","session":"22.1","team":"main"}}
{"timestamp":"2021-11-17T15:33:26.348573490Z","level":"info","source":"atc","message":"atc.tracker-imb.run.running","data":{"build":"check","pipeline":"test-new","pre_build_id":2,"resource":"docker-repo","session":"22.2","team":"main"}}
{"timestamp":"2021-11-17T15:33:26.356414847Z","level":"info","source":"atc","message":"atc.tracker-imb.run.running","data":{"build":"check","pipeline":"test-new","pre_build_id":3,"resource":"debian-release","session":"22.3","team":"main"}}
{"timestamp":"2021-11-17T15:33:26.364576602Z","level":"info","source":"atc","message":"atc.tracker-imb.run.running","data":{"build":"check","pipeline":"test-new","pre_build_id":4,"resource":"pipeline-tasks","session":"22.4","team":"main"}}

And will fail on the local variable ((.:debian_version)) which is used in the branch name

We've also noticed a lot more load on our Vault server since we upgraded to 7.6.0 and we think it will be all the calls from our output resources variable checks.

Should I raise a bug about this, tagging @evanchaoli since he authored the PR which I think made these changes

[taylorsilva]

Open an issue and tag it with the bug label. Great info here, thanks for digging into it!

[mymasse]

Issue raised: #7831