-
Notifications
You must be signed in to change notification settings - Fork 288
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Opportunities to batch / avoid truncate and delete shards operations, triggered via chitchat. #4908
Comments
Delete shard only happens via chitchat. Nothing is done if the shard is not still part of the control plane model, so there should not be any redundancy in this operation. Truncation can happen via chitchat or via reception of a grpc request. Because the gRPC does not update the shardpositions model, we actually often truncate twice, and the check is helping. The check (and the truncation) however, require acquiring the write lock. Maybe we could make this operation cheaper by either doing the check using the partial lock, by updating the shardpositions model on grpc, or possibly by somehow batching truncations. |
The delete operation seems to be using the index correctly.
|
upon deletion of an indexer, I see 317 "deleting shards" logs. Investigating specific shard id does show a unique line. The second half is probably "rebalancing" kicking in. |
No description provided.
The text was updated successfully, but these errors were encountered: