-
Notifications
You must be signed in to change notification settings - Fork 535
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement roaring_bitmap_internal_validate #658
Comments
I have written the validation according to C version of Roaring bitmaps. Some checks were removed, e.g. it is impossible to have negative capacity as it is derived from array length, which is always positive. One TODO left - is it somehow possible to have run container with Do you have some suggestion how to create broken serialized bitmap to make some tests? |
Important work!
It should not be. We don’t include empty containers. |
When deserializing a bitmap, it is possible that the result might be invalid. This could happen because there was data corruption. The deserialization could still generate a bitmap without failure, but the result could be otherwise unusable.
You can avoid such problems by hashing your saved data (e.g., md5sum). But we could could also directly, at some expense, validate the deserialized data.
The C version of Roaring has an interesting function that can be called after deserializing a bitmap, to make sure it is proper:
https://github.com/RoaringBitmap/CRoaring/blob/a103d3811702b9389c538881c9974e9a7a7552af/src/roaring.c#L435
It is not very difficult to implement and could help users who have production data.
The text was updated successfully, but these errors were encountered: