-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
stale read QPS is jitter #974
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I found this issue in #970
Topology
Version: nightly
Inject network latency
Inject 1000ms network delay in all TiDB nodes to
192.168.80.2
, which deploy TiKV192.168.80.2:23360
and192.168.80.2:23361
192.168.78.3
and192.168.78.4
are same with upper too.Test
Load: stale-read request
tidb_kv_read_timeout: 100
max_execution_time: 400
And run more load in TiDB
192.168.78.2
:The following are related metrics, some of metrics here are not as expected:
DataNotReady
backoff.DataNotReady
is cause by lock. Following is TiKV log:444257405948395529
from upper log to search in TiDB slow query, which is a TiDB internal query, from statistic.the related slow log is following:
The slow query execution plan is following:
Then I add some metrics ( #973 ) in batch-client, this is the root cause to make Insert statement execute too slowly.
In the following metric, since I inject around 1 second network latency between
192.168.78.2
and192.168.80.2
, so the maxbatch conn send duration
is 15s.In the following metric, the max
got-resp
is 1.14min, and maxwait-to-send
is 16s, this is unexpected.The text was updated successfully, but these errors were encountered: