db-debugger: new commands #13340

msmouse · 2024-05-19T22:36:37Z

Description

get-leaf to return leaf info for given snapshot version and leaf index
get-value to query the kv table by key
scan-snapshot to go through a state snapshot

Type of Change

New feature

Which Components or Systems Does This Change Impact?

Other (specify) -- db-debugger

How Has This Been Tested?

manual runs

Key Areas to Review

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-05-19T22:36:40Z

⏱️ 15h 38m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
rust-targeted-unit-tests	2h 14m	🟩 ⬜ 🟩 🟩 🟩 (+13 more)
rust-smoke-tests	1h 37m	🟩 ⬜ 🟩
rust-lints	1h 36m	🟥 ⬜ 🟩 🟩 🟩 (+13 more)
windows-build	1h 30m	🟩 🟩
execution-performance / single-node-performance	1h 17m	🟩 🟩 🟩
run-tests-main-branch	1h 16m	🟩 🟩 🟩 🟩 🟥 (+14 more)
rust-move-tests	50m	🟩 ⬜ 🟩 🟩 🟩 (+14 more)
forge-e2e-test / forge	46m	🟩 ⬜ 🟩
rust-images / rust-all	40m	🟩 🟩 🟩
forge-compat-test / forge	37m	🟩 ⬜ 🟩
forge-framework-upgrade-test / forge	33m	⬜ 🟩
general-lints	31m	🟩 🟩 🟩 🟩 🟩 (+13 more)
cli-e2e-tests / run-cli-tests	29m	🟥 🟥 🟥
check-dynamic-deps	23m	🟩 🟩 🟩 🟩 🟩 (+14 more)
rust-build-cached-packages	17m	🟩 🟩 🟩
execution-performance / test-target-determinator	13m	🟩 🟩 🟩
test-target-determinator	12m	🟩 🟩 🟩
check	12m	🟩 🟩 🟩
semgrep/ci	8m	🟩 🟩 🟩 🟩 🟩 (+14 more)
file_change_determinator	3m	🟩 🟩 🟩 🟩 🟩 (+14 more)
permission-check	3m	🟩 🟩 🟩 🟩 🟩 (+14 more)
file_change_determinator	3m	🟩 🟩 🟩 🟩 🟩 (+13 more)
node-api-compatibility-tests / node-api-compatibility-tests	3m	🟩 🟩 🟩
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose	2m	🟩
permission-check	54s	🟩 🟩 🟩 🟩 🟩 (+14 more)
permission-check	49s	🟩 🟩 🟩 🟩 🟩 (+14 more)
permission-check	48s	🟩 🟩 🟩 🟩 🟩 (+14 more)
file_change_determinator	30s	🟩 🟩 🟩
permission-check	11s	🟩 🟩 🟩
determine-docker-build-metadata	6s	🟩 🟩 🟩

🚨 5 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg
cli-e2e-tests / run-cli-tests	9m	6m
forge-e2e-test / forge	19m	15m
execution-performance / single-node-performance	26m	22m
rust-targeted-unit-tests	11m	19m
rust-move-tests	3m	9m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

msmouse · 2024-05-20T03:02:11Z

db-debugger: new commands #13340 👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @msmouse and the rest of your teammates on Graphite

github-actions · 2024-05-21T22:18:34Z

✅ Forge suite `compat` success on `3ffe0986b5fe4acb76544ae7ae85d73b91a6a411` ==> `daa84423c54f09c7455c65d217568e394fc5fdae`

Compatibility test results for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> daa84423c54f09c7455c65d217568e394fc5fdae (PR)
1. Check liveness of validators at old version: 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411
compatibility::simple-validator-upgrade::liveness-check : committed: 6723.731472042241 txn/s, latency: 4932.391682192206 ms, (p50: 4800 ms, p90: 8100 ms, p99: 8700 ms), latency samples: 257640
2. Upgrading first Validator to new version: daa84423c54f09c7455c65d217568e394fc5fdae
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1775.6309088295588 txn/s, latency: 16213.727741573033 ms, (p50: 18300 ms, p90: 21900 ms, p99: 22300 ms), latency samples: 89000
3. Upgrading rest of first batch to new version: daa84423c54f09c7455c65d217568e394fc5fdae
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1846.0772219683465 txn/s, latency: 15406.041174554928 ms, (p50: 19000 ms, p90: 21700 ms, p99: 22300 ms), latency samples: 92120
4. upgrading second batch to new version: daa84423c54f09c7455c65d217568e394fc5fdae
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3593.371478566849 txn/s, latency: 8707.779158526375 ms, (p50: 9800 ms, p90: 12400 ms, p99: 12700 ms), latency samples: 143320
5. check swarm health
Compatibility test for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> daa84423c54f09c7455c65d217568e394fc5fdae passed
Test Ok

github-actions · 2024-05-21T22:25:43Z

✅ Forge suite `realistic_env_max_load` success on `daa84423c54f09c7455c65d217568e394fc5fdae`

two traffics test: inner traffic : committed: 8359.015873577484 txn/s, latency: 4690.911882694663 ms, (p50: 4500 ms, p90: 5400 ms, p99: 10200 ms), latency samples: 3608020
two traffics test : committed: 100.08604379691607 txn/s, latency: 1792.5113636363637 ms, (p50: 1800 ms, p90: 2100 ms, p99: 2300 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.211, avg: 0.202", "QsPosToProposal: max: 0.236, avg: 0.207", "ConsensusProposalToOrdered: max: 0.488, avg: 0.431", "ConsensusOrderedToCommit: max: 0.343, avg: 0.327", "ConsensusProposalToCommit: max: 0.769, avg: 0.758"]
Max round gap was 1 [limit 4] at version 1344266. Max no progress secs was 4.752398 [limit 15] at version 1344266.
Test Ok

github-actions · 2024-05-21T22:26:34Z

✅ Forge suite `framework_upgrade` success on `3ffe0986b5fe4acb76544ae7ae85d73b91a6a411` ==> `daa84423c54f09c7455c65d217568e394fc5fdae`

Compatibility test results for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> daa84423c54f09c7455c65d217568e394fc5fdae (PR)
Upgrade the nodes to version: daa84423c54f09c7455c65d217568e394fc5fdae
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1198.4848453352406 txn/s, submitted: 1199.837282625444 txn/s, failed submission: 1.3524372902033936 txn/s, expired: 1.3524372902033936 txn/s, latency: 2573.701175474892 ms, (p50: 2100 ms, p90: 4200 ms, p99: 6600 ms), latency samples: 106340
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1061.1511581363206 txn/s, submitted: 1064.022716224533 txn/s, failed submission: 2.871558088212358 txn/s, expired: 2.871558088212358 txn/s, latency: 2851.9448272273107 ms, (p50: 2100 ms, p90: 5400 ms, p99: 7700 ms), latency samples: 96080
5. check swarm health
Compatibility test for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> daa84423c54f09c7455c65d217568e394fc5fdae passed
Upgrade the remaining nodes to version: daa84423c54f09c7455c65d217568e394fc5fdae
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1175.0591065788276 txn/s, submitted: 1177.3882623896875 txn/s, failed submission: 2.3291558108599157 txn/s, expired: 2.3291558108599157 txn/s, latency: 2747.218721506442 ms, (p50: 2200 ms, p90: 4800 ms, p99: 7200 ms), latency samples: 100900
Test Ok

msmouse force-pushed the 0519-alden-debugger branch 5 times, most recently from c7a1688 to 7753138 Compare May 20, 2024 03:02

msmouse changed the base branch from main to 0519-alden-lazy-iter May 20, 2024 03:02

msmouse mentioned this pull request May 20, 2024

schemadb: make iterator lazy #13341

Merged

8 tasks

msmouse added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label May 20, 2024

This comment has been minimized.

Sign in to view

msmouse removed the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label May 20, 2024

This comment has been minimized.

Sign in to view

msmouse force-pushed the 0519-alden-debugger branch from 7753138 to 41c5e8c Compare May 20, 2024 15:55

msmouse changed the base branch from 0519-alden-lazy-iter to main May 20, 2024 15:55

msmouse force-pushed the 0519-alden-debugger branch 3 times, most recently from 027dc38 to 4d2ef03 Compare May 20, 2024 16:11

msmouse changed the title ~~db-debugger: get-by-leaf-index~~ db-debugger: new commands May 20, 2024

msmouse force-pushed the 0519-alden-debugger branch 3 times, most recently from c359532 to 9371751 Compare May 20, 2024 17:36

msmouse marked this pull request as ready for review May 20, 2024 17:36

msmouse requested review from lightmark and grao1991 as code owners May 20, 2024 17:36

msmouse requested a review from areshand May 20, 2024 17:36

msmouse force-pushed the 0519-alden-debugger branch from 9371751 to 68d4eb4 Compare May 20, 2024 17:39

grao1991 approved these changes May 20, 2024

View reviewed changes

msmouse requested a review from zekun000 May 21, 2024 20:27

zekun000 approved these changes May 21, 2024

View reviewed changes

areshand approved these changes May 21, 2024

View reviewed changes

msmouse enabled auto-merge (rebase) May 21, 2024 21:25

This comment has been minimized.

Sign in to view

msmouse added 2 commits May 21, 2024 21:50

db-debugger: new commands get-leaf, get-value

51f09bf

db-debugger: new command scan-snapshot

daa8442

msmouse force-pushed the 0519-alden-debugger branch from 68d4eb4 to daa8442 Compare May 21, 2024 21:51

This comment has been minimized.

Sign in to view

msmouse merged commit 792881a into main May 21, 2024
49 of 51 checks passed

msmouse deleted the 0519-alden-debugger branch May 21, 2024 22:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

db-debugger: new commands #13340

db-debugger: new commands #13340

msmouse commented May 19, 2024 •

edited

trunk-io bot commented May 19, 2024 •

edited

msmouse commented May 20, 2024 •

edited

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented May 21, 2024

github-actions bot commented May 21, 2024

github-actions bot commented May 21, 2024

db-debugger: new commands #13340

db-debugger: new commands #13340

Conversation

msmouse commented May 19, 2024 • edited

Description

Type of Change

Which Components or Systems Does This Change Impact?

How Has This Been Tested?

Key Areas to Review

Checklist

trunk-io bot commented May 19, 2024 • edited

msmouse commented May 20, 2024 • edited

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented May 21, 2024

✅ Forge suite compat success on 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> daa84423c54f09c7455c65d217568e394fc5fdae

github-actions bot commented May 21, 2024

✅ Forge suite realistic_env_max_load success on daa84423c54f09c7455c65d217568e394fc5fdae

github-actions bot commented May 21, 2024

✅ Forge suite framework_upgrade success on 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> daa84423c54f09c7455c65d217568e394fc5fdae

msmouse commented May 19, 2024 •

edited

trunk-io bot commented May 19, 2024 •

edited

msmouse commented May 20, 2024 •

edited

✅ Forge suite `compat` success on `3ffe0986b5fe4acb76544ae7ae85d73b91a6a411` ==> `daa84423c54f09c7455c65d217568e394fc5fdae`

✅ Forge suite `realistic_env_max_load` success on `daa84423c54f09c7455c65d217568e394fc5fdae`

✅ Forge suite `framework_upgrade` success on `3ffe0986b5fe4acb76544ae7ae85d73b91a6a411` ==> `daa84423c54f09c7455c65d217568e394fc5fdae`