Postgres node always reaches connection timeout if worker is in a "bad state"

[syllabusadmin]

Bug Description

Certain workers, seem to enter a "bad state", which makes postgres nodes always reach Connection terminated due to connection timeout.

This always happens if the executions runs on these workers, and it will always timeout on the first postgres node in the workflow. Automatic retries fail.

I believe the "bad state" comes from Python nodes, as the workers that run this Postgres node have had python ran on them previously, and memory on the instance has bubbled up. I've not been able to verify this.

{ "errorMessage": "Connection terminated due to connection timeout", "errorDescription": "Failed query: SELECT * FROM **", "errorDetails": {}, "n8nDetails": { "nodeName": "Postgres", "nodeType": "n8n-nodes-base.postgres", "nodeVersion": 2.3, "resource": "database", "operation": "executeQuery", "time": "5/1/2024, 8:03:13 AM", "n8nVersion": "1.37.3 (Self Hosted)", "binaryDataMode": "default", "stackTrace": [ "NodeOperationError: Connection terminated due to connection timeout", " at parsePostgresError (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Postgres/v2/helpers/utils.js:68:12)", " at /usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Postgres/v2/helpers/utils.js:175:31", " at processTicksAndRejections (node:internal/process/task_queues:95:5)", " at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Postgres/v2/actions/database/executeQuery.operation.js:79:12)", " at Object.router (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Postgres/v2/actions/router.js:49:30)", " at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Postgres/v2/PostgresV2.node.js:16:16)", " at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/Workflow.js:728:19)", " at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/WorkflowExecute.js:660:53", " at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/WorkflowExecute.js:1062:20" ] } }

To Reproduce

1. Run some Python on a worker
2. Attempt to run a remote postgres query on that same worker
3. Note that the connection timeout always occurs

Expected behavior

Postgres node should not always timeout

Operating System

Built from docker n8nio/n8n:1.37.3

n8n Version

1.37.3

Node.js Version

18.19.1

Database

PostgreSQL

Execution mode

queue

[Joffcom]

Hey @syllabusadmin,

It would be unusual for the code node to cause a connection timeout but as the code nodes do consume more memory until the workflow has finished I can see a resource issue causing it.

At the moment though there is not really enough here to confirm it as a bug and it just looks like a standard networking issue which could be why it only impacts some workers.

Have you looked into the networking side of things from the workers to Postgres and checked any logs there?

It could also be worth increasing the amount of memory available to the workers if you are using the code node just to allow for things like this.