Concourse web node restarting with exit code 2

[skreddy6673]

Web node configs:

web
    State:          Running
      Started:      Wed, 13 May 2020 01:32:41 -0400
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Mon, 11 May 2020 06:04:15 -0400
      Finished:     Wed, 13 May 2020 01:32:40 -0400
    Ready:          True
    Restart Count:  2
    Limits:
      cpu:     7
      memory:  30Gi
    Requests:
      cpu:      6
      memory:   20Gi
    Liveness:   http-get http://:atc/api/v1/info delay=180s timeout=3s period=15s #success=1 #failure=5
    Readiness:  http-get http://:atc/api/v1/info delay=180s timeout=3s period=15s #success=1 #failure=5

And this is what i see in logs

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0xdf5d53]

goroutine 48395912 [running]:
github.com/concourse/concourse/atc/metric/emitter.(*NewRelicEmitter).emitBatch(0xc0005e4750, 0x28966c0, 0xc0075c4ba0, 0xc000784000, 0x7d0, 0x7d0)
        /tmp/build/1c3187db/concourse/atc/metric/emitter/newrelic.go:218 +0x283
created by github.com/concourse/concourse/atc/metric/emitter.(*NewRelicEmitter).submitBatch
        /tmp/build/1c3187db/concourse/atc/metric/emitter/newrelic.go:195 +0x154

Any idea what might be broken?
Concourse version: 5.7.1
Helm Chart: 8.2.13

[cirocosta]

Hi @skreddy6673 ,

Could you please let us know (with secrets/credentials redacted) what configuration you had? e.g., how values.yaml looks like?

It'd be great to be able to tell how we can reproduce to help you out 😁

thank you!

[skreddy6673]

Thanks @cirocosta for your quick response.
Here is my values.yaml file https://github.com/skreddy6673/concourse-chart/blob/master/values.yaml

Thank you!

[gowrisankar22]

@skreddy6673 can you try switching off newrelic ?if works can you check, new relic details are correct ?

[skreddy6673]

@gowrisankar I have the same new relic configuration for over 1 year and never had any issues. And moreover metrics is being emitted to New Relic without any issues. So i don't think it's the issue with New Relic details. Probably there might be some other dependent configs in my helm chart that are causing this.
The Web Pods were fine for like 3 days and then started seeing this issue.