-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
http: TLS handshake error from <promxy IP>:port: EOF #638
Comments
When I limit the number of the concurrencies to 1 by setting |
and there is no issue at all if we send queries in parallel directly to reverse proxy, bypass the |
I have tried with some another setups to narrow down the scope that could cause the problem:
But TLS error messages still show up in the logs. |
@jacksontj Do you have any feedback/comments on this issue? Do you think there is race condition there in Promxy? |
First off, thanks for reaching out! I did some initial digging but your configuration seems incomplete (maybe just not included in the issue?). Specifically its missing the So in my local testing I have And I was able to get data working correctly and use a variation on your curl to test parallel usage:
I have used promxy in front of HTTPs downstreams before without issue; so I don't expect you'll run into issues (other than the config; which is a bit odd because the prometheus scrape_config is a bit odd). Hopefully that helps? |
Thanks @jacksontj for the reply. Yes, we do have
The scheme
The issue is not always showed up if the traffics towards promxy is low; it happens more frequently if we add more traffics like running the same |
Hi @jacksontj Do you have a chance to reproducing the issue using the way I mentioned above? |
We have a deployment as below:
and here is http_client we passed to Promxy:
When I perform multiple PromQL queries in parallel towards Promxy via curl command like this:
seq 1 200 | xargs -n1 -P10 curl --cert tls.crt --key tls.key --cacert ca.crt "https://promxy-endpoint:9091/api/v1/query?query=up"
I got lots of tls error messages
http: TLS handshake error from <promxy IP>:port: EOF
in our reverse proxy. It does not happen when the queries are sent in sequence.I have decoded certs of both sides, client and server, they are all valid certificates.
Checking Promxy logs, there is no error messages; logs shows queries with returned successful code (200).
I would like to know if Promxy supports queries in parallel?
I have tried to test with
query.max-concurrency=20
and the default onequery.max-concurrency=-1
; it does not help.The text was updated successfully, but these errors were encountered: