-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MaxTime is set to a too large number when doing larger latency requests? #7319
Comments
It must be something other then the maxtime of the range request being messed with. That logging is the String representation of the endpoint ref and corresponds to the time range of the store, see thanos/pkg/query/endpointset.go Line 860 in 1e745af
Edit: The max time comes probably from here if its a sidecar: Line 375 in 1e745af
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm having a vague problem, but I will try my best to explain what is happening.
From time to time, I'm getting this error:
Basically I'm seeing a bunch of http2 connection resets.
When debugging, I'm using the Thanos Querier directly and I have been able to reproduce this behaviour under a very specific set of variables.
I have been somewhat starring blind on this, while eventually it hit me.
Looking deeper on an error query like, it actually shows more details. It's a
422 Unprocessable Entity
response code.This I can explain, as due to the error message.
Let me show you the query in question:
Response:
Which make sense, as the
MaxTime
is9223372036854775807
which isn't a legit value. However, as you can see, I make the query with aend: 1714564667.884
argument.This only seems to happen when the query reaches a certain latency response. If I would only query the last 2 hours instead of 6 hours, it's "fine". Like-wise, if I move my querier closer to the Prometheus in question, I can query the last 6 hours just fine.
Somehow, and I don't know why/how, when the latency/response becomes too long, it magically ruins my maxtime to an invalid value?
The text was updated successfully, but these errors were encountered: