Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve mbtiles diff validation, add new metadata field, usability #1244

Open
nyurik opened this issue Mar 11, 2024 · 0 comments
Open

Improve mbtiles diff validation, add new metadata field, usability #1244

nyurik opened this issue Mar 11, 2024 · 0 comments
Labels
cli Related to Command Line Interface mbtiles Anything related to Martin's MBTiles support

Comments

@nyurik
Copy link
Member

nyurik commented Mar 11, 2024

Current state

  • Files generated by martin's mbtiles tool contain agg_tiles_hash metadata field. The field represent an aggregate hash of all tiles table (z/x/y/data). See docs.
  • When diffing two mbtiles files - file A and file B, the generated delta file will include agg_tiles_hash_after_apply value - the hash of the file B.

Proposal

Current design has a few shortcomings that need to be addressed. The biggest issues is that any mbtiles can be applied to any other mbtiles - which is different than "merge-copying" (simply adding content to the same file from multiple other files, where tiles with the same x/y/z override existing ones.

  • Add a new metadata field agg_tiles_hash_before_apply (name?) to the delta files -- this corresponds to the agg_tiles_hash of thefile A.
  • Improve logic and default behavior

Creating a diff file between A and B

  • Refuse to work if either A or B files contain agg_tiles_hash_before_apply or agg_tiles_hash_after_apply (unless there is a --force flag)
  • If either files A or B do not contain agg_tiles_hash, compute it (but don't save it?)
  • Store agg_tiles_hash as agg_tiles_hash_before_apply and agg_tiles_hash_after_apply values in the delta file

Patch file A with a diff, or apply on copy

  • Refuse to work if file A contain agg_tiles_hash_before_apply or agg_tiles_hash_after_apply
  • Refuse to work if the diff file does NOT contain agg_tiles_hash, agg_tiles_hash_before_apply or agg_tiles_hash_after_apply
  • Unless some --validate flag is set, accept agg_tiles_hash stored in file A as valid. If it does not exist, compute it.
  • Refuse to work if the stored/computed agg_tiles_hash does not match agg_tiles_hash_before_apply
  • After patching file A, validate the file against the agg_tiles_hash_after_apply value
@nyurik nyurik added mbtiles Anything related to Martin's MBTiles support cli Related to Command Line Interface labels Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cli Related to Command Line Interface mbtiles Anything related to Martin's MBTiles support
Projects
None yet
Development

No branches or pull requests

1 participant