-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Application crashes on outgoing DNS requests/traffic #2271
Comments
I started digging into why it works for one team but not the other. My hypotesis about the websocket does not hold water. I did not know, but the other team is now actually using WebSockets more extensively than the one having the issues. What stands out is that on the problematic page, one graphql query is done that returns a 6MB response which takes more than 5 seconds to complete and in general it makes more grapqhl queries than should be needed. |
More digging; as can be seen from the logs, the error is related to DNS query. In our case it can be completely avoided if we configure the db connection with the db server IP instead of the "db" name, thus avoiding the DNS query. I'm open for screen-sharing session to show you reproduction, but I'll need time to set up an environment. |
Thank you! we're still investigating the logs. |
Fixed with #2308 |
Observed again |
Bug Description
Our Nest.js application crashes seemingly a bit randomly.
We think it is related to having an active WebSocket connection from frontend -> backend where backend is running in mirrord.Looks like it is triggered if the DNS query is done while there is a long running >5 seconds stolen request being processed in the local process.
Steps to Reproduce
Backtrace
Tested on 3.90.0
Relevant Logs
internal_proxy.log
Your operating system and version
macOS Sonoma 14.2.1 (23C71)
Local process
nodejs
Local process version
... node/v18.17.1/bin/node: Mach-O 64-bit executable arm64
Additional Info
It most often seems to crash when it tries to connect to the database.
The database is run in a separate namespace in the k8s cluster. An ExternalName with name "db" is created in APP's namespace, which is used in app config.
The text was updated successfully, but these errors were encountered: