Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Use maybe_dense_stack in non contiguous rollouts #2131

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[Feature] Use maybe_dense_stack in non contiguous rollouts #2131

wants to merge 1 commit into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 29, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 29, 2024 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2131

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 5 Unrelated Failures

As of commit 85ad1f0 with merge base 3c6b9c6 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 29, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 53.5231ms 53.0819ms 18.8388 Ops/s 18.5027 Ops/s $\color{#35bf28}+1.82\%$
test_sync 33.3412ms 29.4319ms 33.9768 Ops/s 33.7157 Ops/s $\color{#35bf28}+0.77\%$
test_async 55.6813ms 28.1144ms 35.5690 Ops/s 34.6769 Ops/s $\color{#35bf28}+2.57\%$
test_simple 0.4175s 0.3583s 2.7909 Ops/s 2.9153 Ops/s $\color{#d91a1a}-4.27\%$
test_transformed 0.5542s 0.4993s 2.0030 Ops/s 1.9841 Ops/s $\color{#35bf28}+0.95\%$
test_serial 1.2730s 1.2251s 0.8163 Ops/s 0.8177 Ops/s $\color{#d91a1a}-0.18\%$
test_parallel 1.0626s 1.0237s 0.9768 Ops/s 0.9835 Ops/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-True-True-True-True] 0.1526ms 21.5049μs 46.5011 KOps/s 46.9043 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[True-True-True-True-False] 37.6000μs 13.0474μs 76.6437 KOps/s 77.6929 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[True-True-True-False-True] 47.8590μs 12.6287μs 79.1847 KOps/s 80.9298 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[True-True-True-False-False] 30.5870μs 7.5987μs 131.6017 KOps/s 133.2605 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[True-True-False-True-True] 56.1550μs 22.6753μs 44.1009 KOps/s 44.4320 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-False-True-False] 37.5000μs 14.4022μs 69.4341 KOps/s 70.9526 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[True-True-False-False-True] 47.0280μs 13.9739μs 71.5622 KOps/s 73.1567 KOps/s $\color{#d91a1a}-2.18\%$
test_step_mdp_speed[True-True-False-False-False] 39.1130μs 8.8997μs 112.3628 KOps/s 115.3200 KOps/s $\color{#d91a1a}-2.56\%$
test_step_mdp_speed[True-False-True-True-True] 61.8560μs 24.1201μs 41.4592 KOps/s 41.9100 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-False-True-True-False] 49.4620μs 15.9477μs 62.7048 KOps/s 64.4288 KOps/s $\color{#d91a1a}-2.68\%$
test_step_mdp_speed[True-False-True-False-True] 36.6290μs 13.9896μs 71.4819 KOps/s 74.0642 KOps/s $\color{#d91a1a}-3.49\%$
test_step_mdp_speed[True-False-True-False-False] 31.0380μs 9.0160μs 110.9139 KOps/s 115.7316 KOps/s $\color{#d91a1a}-4.16\%$
test_step_mdp_speed[True-False-False-True-True] 57.8990μs 25.4060μs 39.3607 KOps/s 40.0486 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[True-False-False-True-False] 61.9060μs 17.1049μs 58.4629 KOps/s 60.1510 KOps/s $\color{#d91a1a}-2.81\%$
test_step_mdp_speed[True-False-False-False-True] 52.0780μs 15.0139μs 66.6049 KOps/s 67.9919 KOps/s $\color{#d91a1a}-2.04\%$
test_step_mdp_speed[True-False-False-False-False] 49.5000μs 9.9442μs 100.5611 KOps/s 100.8981 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-True-True-True] 50.8850μs 24.1083μs 41.4795 KOps/s 41.5809 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-True-True-True-False] 39.1330μs 15.6265μs 63.9939 KOps/s 65.3898 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[False-True-True-False-True] 47.4490μs 16.0912μs 62.1459 KOps/s 63.1788 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-True-True-False-False] 36.8390μs 10.0652μs 99.3518 KOps/s 100.7647 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-True-False-True-True] 52.4880μs 25.2768μs 39.5619 KOps/s 39.9870 KOps/s $\color{#d91a1a}-1.06\%$
test_step_mdp_speed[False-True-False-True-False] 46.2270μs 16.8481μs 59.3538 KOps/s 60.2672 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[False-True-False-False-True] 47.5890μs 17.1967μs 58.1507 KOps/s 59.3359 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[False-True-False-False-False] 36.4690μs 11.3065μs 88.4445 KOps/s 89.2431 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-False-True-True-True] 72.2350μs 26.8249μs 37.2788 KOps/s 37.8713 KOps/s $\color{#d91a1a}-1.56\%$
test_step_mdp_speed[False-False-True-True-False] 52.9790μs 18.2417μs 54.8193 KOps/s 55.3699 KOps/s $\color{#d91a1a}-0.99\%$
test_step_mdp_speed[False-False-True-False-True] 44.2830μs 17.2257μs 58.0529 KOps/s 59.3222 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-False-True-False-False] 31.2780μs 11.4007μs 87.7137 KOps/s 90.2937 KOps/s $\color{#d91a1a}-2.86\%$
test_step_mdp_speed[False-False-False-True-True] 68.2670μs 28.5846μs 34.9838 KOps/s 35.9453 KOps/s $\color{#d91a1a}-2.67\%$
test_step_mdp_speed[False-False-False-True-False] 49.5330μs 19.2421μs 51.9695 KOps/s 53.4336 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[False-False-False-False-True] 47.5590μs 18.2504μs 54.7932 KOps/s 55.8545 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[False-False-False-False-False] 38.4920μs 12.3034μs 81.2783 KOps/s 83.0218 KOps/s $\color{#d91a1a}-2.10\%$
test_values[generalized_advantage_estimate-True-True] 9.5827ms 9.2504ms 108.1034 Ops/s 106.1997 Ops/s $\color{#35bf28}+1.79\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.9017ms 35.6649ms 28.0387 Ops/s 29.7869 Ops/s $\textbf{\color{#d91a1a}-5.87\%}$
test_values[td0_return_estimate-False-False] 0.2210ms 0.1627ms 6.1462 KOps/s 5.6898 KOps/s $\textbf{\color{#35bf28}+8.02\%}$
test_values[td1_return_estimate-False-False] 23.1304ms 22.7860ms 43.8866 Ops/s 42.4719 Ops/s $\color{#35bf28}+3.33\%$
test_values[vec_td1_return_estimate-False-False] 37.5900ms 35.5297ms 28.1454 Ops/s 29.6755 Ops/s $\textbf{\color{#d91a1a}-5.16\%}$
test_values[td_lambda_return_estimate-True-False] 34.4004ms 32.8473ms 30.4439 Ops/s 29.4021 Ops/s $\color{#35bf28}+3.54\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.8592ms 35.5649ms 28.1176 Ops/s 29.7203 Ops/s $\textbf{\color{#d91a1a}-5.39\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.8042ms 8.0739ms 123.8554 Ops/s 123.1913 Ops/s $\color{#35bf28}+0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3912ms 2.0251ms 493.8078 Ops/s 491.5591 Ops/s $\color{#35bf28}+0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4414ms 0.3558ms 2.8107 KOps/s 2.7849 KOps/s $\color{#35bf28}+0.93\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.8137ms 47.5276ms 21.0404 Ops/s 23.7482 Ops/s $\textbf{\color{#d91a1a}-11.40\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6732ms 3.0297ms 330.0638 Ops/s 328.0951 Ops/s $\color{#35bf28}+0.60\%$
test_dqn_speed 6.8933ms 1.3662ms 731.9483 Ops/s 715.6538 Ops/s $\color{#35bf28}+2.28\%$
test_ddpg_speed 4.1290ms 2.8865ms 346.4345 Ops/s 342.0306 Ops/s $\color{#35bf28}+1.29\%$
test_sac_speed 10.0627ms 8.5855ms 116.4757 Ops/s 104.5829 Ops/s $\textbf{\color{#35bf28}+11.37\%}$
test_redq_speed 15.2336ms 13.2345ms 75.5602 Ops/s 74.3631 Ops/s $\color{#35bf28}+1.61\%$
test_redq_deprec_speed 15.2165ms 13.3428ms 74.9467 Ops/s 73.1855 Ops/s $\color{#35bf28}+2.41\%$
test_td3_speed 11.2033ms 8.5092ms 117.5201 Ops/s 115.6057 Ops/s $\color{#35bf28}+1.66\%$
test_cql_speed 37.9787ms 36.4113ms 27.4640 Ops/s 26.2142 Ops/s $\color{#35bf28}+4.77\%$
test_a2c_speed 8.3474ms 7.4586ms 134.0739 Ops/s 130.3525 Ops/s $\color{#35bf28}+2.85\%$
test_ppo_speed 8.3311ms 7.6622ms 130.5100 Ops/s 124.3392 Ops/s $\color{#35bf28}+4.96\%$
test_reinforce_speed 7.7062ms 6.6535ms 150.2974 Ops/s 149.3733 Ops/s $\color{#35bf28}+0.62\%$
test_iql_speed 34.0319ms 32.9481ms 30.3507 Ops/s 30.0343 Ops/s $\color{#35bf28}+1.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1559ms 2.1070ms 474.6159 Ops/s 447.3594 Ops/s $\textbf{\color{#35bf28}+6.09\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8636ms 0.4974ms 2.0104 KOps/s 1.9667 KOps/s $\color{#35bf28}+2.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5725ms 0.4711ms 2.1227 KOps/s 2.0611 KOps/s $\color{#35bf28}+2.99\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.2742ms 2.0717ms 482.6955 Ops/s 449.1253 Ops/s $\textbf{\color{#35bf28}+7.47\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0110ms 0.4897ms 2.0422 KOps/s 1.9844 KOps/s $\color{#35bf28}+2.91\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6743ms 0.4685ms 2.1346 KOps/s 2.0693 KOps/s $\color{#35bf28}+3.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4697ms 1.2303ms 812.8086 Ops/s 796.4021 Ops/s $\color{#35bf28}+2.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4543ms 1.1614ms 861.0222 Ops/s 842.7260 Ops/s $\color{#35bf28}+2.17\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4568ms 2.2429ms 445.8577 Ops/s 415.2494 Ops/s $\textbf{\color{#35bf28}+7.37\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0216ms 0.6123ms 1.6332 KOps/s 1.5650 KOps/s $\color{#35bf28}+4.36\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8296ms 0.5896ms 1.6960 KOps/s 1.6378 KOps/s $\color{#35bf28}+3.55\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.4220ms 2.0949ms 477.3419 Ops/s 444.1420 Ops/s $\textbf{\color{#35bf28}+7.48\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9298ms 0.4963ms 2.0147 KOps/s 1.9501 KOps/s $\color{#35bf28}+3.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6899ms 0.4768ms 2.0974 KOps/s 2.0241 KOps/s $\color{#35bf28}+3.62\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4105ms 2.1089ms 474.1800 Ops/s 450.9352 Ops/s $\textbf{\color{#35bf28}+5.15\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6492ms 0.4921ms 2.0322 KOps/s 1.9878 KOps/s $\color{#35bf28}+2.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.3686ms 0.4730ms 2.1141 KOps/s 2.1126 KOps/s $\color{#35bf28}+0.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.3619ms 2.1980ms 454.9589 Ops/s 437.1436 Ops/s $\color{#35bf28}+4.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1509ms 0.6163ms 1.6226 KOps/s 1.6077 KOps/s $\color{#35bf28}+0.93\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7181ms 0.5885ms 1.6992 KOps/s 1.6444 KOps/s $\color{#35bf28}+3.33\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1093s 7.7104ms 129.6946 Ops/s 127.6030 Ops/s $\color{#35bf28}+1.64\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 13.9863ms 12.1197ms 82.5100 Ops/s 83.1417 Ops/s $\color{#d91a1a}-0.76\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.5970ms 1.0968ms 911.7479 Ops/s 953.6451 Ops/s $\color{#d91a1a}-4.39\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 98.6489ms 5.5306ms 180.8122 Ops/s 173.8867 Ops/s $\color{#35bf28}+3.98\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1034s 13.9310ms 71.7824 Ops/s 71.4261 Ops/s $\color{#35bf28}+0.50\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.7160ms 1.1226ms 890.8228 Ops/s 935.4533 Ops/s $\color{#d91a1a}-4.77\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1047s 6.0182ms 166.1624 Ops/s 164.4261 Ops/s $\color{#35bf28}+1.06\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.5652ms 12.4009ms 80.6393 Ops/s 80.5067 Ops/s $\color{#35bf28}+0.16\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.9638ms 1.4443ms 692.3720 Ops/s 721.2471 Ops/s $\color{#d91a1a}-4.00\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1141s 0.1137s 8.7957 Ops/s 8.7456 Ops/s $\color{#35bf28}+0.57\%$
test_sync 0.1027s 0.1008s 9.9180 Ops/s 9.7700 Ops/s $\color{#35bf28}+1.52\%$
test_async 0.1947s 98.1204ms 10.1916 Ops/s 10.3659 Ops/s $\color{#d91a1a}-1.68\%$
test_single_pixels 0.1967s 0.1345s 7.4356 Ops/s 7.9363 Ops/s $\textbf{\color{#d91a1a}-6.31\%}$
test_sync_pixels 0.1402s 86.2427ms 11.5952 Ops/s 11.5702 Ops/s $\color{#35bf28}+0.22\%$
test_async_pixels 0.1586s 68.0666ms 14.6915 Ops/s 14.3748 Ops/s $\color{#35bf28}+2.20\%$
test_simple 0.7524s 0.7513s 1.3311 Ops/s 1.3059 Ops/s $\color{#35bf28}+1.93\%$
test_transformed 0.9967s 0.9904s 1.0097 Ops/s 0.9912 Ops/s $\color{#35bf28}+1.87\%$
test_serial 2.4499s 2.3979s 0.4170 Ops/s 0.4178 Ops/s $\color{#d91a1a}-0.18\%$
test_parallel 2.3817s 2.3139s 0.4322 Ops/s 0.4313 Ops/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-True-True-True-True] 75.8810μs 33.4231μs 29.9194 KOps/s 30.6559 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[True-True-True-True-False] 43.9810μs 19.7187μs 50.7132 KOps/s 50.1237 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-True-True-False-True] 44.5800μs 18.6199μs 53.7059 KOps/s 52.4388 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[True-True-True-False-False] 28.8300μs 11.1190μs 89.9361 KOps/s 88.4772 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-True-False-True-True] 69.8420μs 34.4020μs 29.0681 KOps/s 28.2986 KOps/s $\color{#35bf28}+2.72\%$
test_step_mdp_speed[True-True-False-True-False] 54.6910μs 21.2087μs 47.1504 KOps/s 46.2272 KOps/s $\color{#35bf28}+2.00\%$
test_step_mdp_speed[True-True-False-False-True] 49.0710μs 20.2753μs 49.3212 KOps/s 48.0271 KOps/s $\color{#35bf28}+2.69\%$
test_step_mdp_speed[True-True-False-False-False] 32.5900μs 12.9850μs 77.0121 KOps/s 75.6666 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-False-True-True-True] 64.4010μs 36.1736μs 27.6445 KOps/s 27.3950 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-False-True-True-False] 0.1116ms 23.0348μs 43.4126 KOps/s 42.2224 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[True-False-True-False-True] 54.5910μs 20.1627μs 49.5966 KOps/s 48.4783 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[True-False-True-False-False] 36.4210μs 12.8654μs 77.7276 KOps/s 75.7804 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-False-False-True-True] 66.8710μs 37.8476μs 26.4217 KOps/s 25.8348 KOps/s $\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-False-False-True-False] 50.1510μs 24.7798μs 40.3555 KOps/s 39.5469 KOps/s $\color{#35bf28}+2.04\%$
test_step_mdp_speed[True-False-False-False-True] 46.5710μs 22.0802μs 45.2895 KOps/s 45.0447 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-False-False-False-False] 37.1100μs 14.6772μs 68.1330 KOps/s 67.1533 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[False-True-True-True-True] 0.1071ms 36.6911μs 27.2545 KOps/s 27.2947 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-True-True-False] 58.4810μs 22.9560μs 43.5617 KOps/s 42.1644 KOps/s $\color{#35bf28}+3.31\%$
test_step_mdp_speed[False-True-True-False-True] 77.8420μs 24.1578μs 41.3946 KOps/s 40.4153 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[False-True-True-False-False] 38.9710μs 14.7178μs 67.9448 KOps/s 66.9186 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-True-False-True-True] 69.1510μs 37.8040μs 26.4523 KOps/s 25.9435 KOps/s $\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-True-False-True-False] 56.4910μs 24.7033μs 40.4804 KOps/s 39.0903 KOps/s $\color{#35bf28}+3.56\%$
test_step_mdp_speed[False-True-False-False-True] 97.4610μs 25.3894μs 39.3866 KOps/s 38.0990 KOps/s $\color{#35bf28}+3.38\%$
test_step_mdp_speed[False-True-False-False-False] 41.4710μs 16.5053μs 60.5865 KOps/s 59.8467 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[False-False-True-True-True] 69.5110μs 39.7360μs 25.1661 KOps/s 24.8700 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-False-True-True-False] 58.5010μs 26.7330μs 37.4069 KOps/s 36.6934 KOps/s $\color{#35bf28}+1.94\%$
test_step_mdp_speed[False-False-True-False-True] 49.6110μs 25.7180μs 38.8832 KOps/s 38.0562 KOps/s $\color{#35bf28}+2.17\%$
test_step_mdp_speed[False-False-True-False-False] 38.9710μs 16.4368μs 60.8392 KOps/s 59.0169 KOps/s $\color{#35bf28}+3.09\%$
test_step_mdp_speed[False-False-False-True-True] 67.8910μs 42.7065μs 23.4156 KOps/s 23.1043 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-False-False-True-False] 54.9710μs 28.9620μs 34.5280 KOps/s 33.7528 KOps/s $\color{#35bf28}+2.30\%$
test_step_mdp_speed[False-False-False-False-True] 54.4810μs 27.1675μs 36.8087 KOps/s 36.0295 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[False-False-False-False-False] 42.3310μs 18.1371μs 55.1357 KOps/s 53.7049 KOps/s $\color{#35bf28}+2.66\%$
test_values[generalized_advantage_estimate-True-True] 24.5221ms 23.9194ms 41.8070 Ops/s 40.6905 Ops/s $\color{#35bf28}+2.74\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.0942ms 3.1854ms 313.9288 Ops/s 312.8166 Ops/s $\color{#35bf28}+0.36\%$
test_values[td0_return_estimate-False-False] 87.3610μs 63.0057μs 15.8716 KOps/s 15.8007 KOps/s $\color{#35bf28}+0.45\%$
test_values[td1_return_estimate-False-False] 54.3327ms 52.8573ms 18.9189 Ops/s 18.9724 Ops/s $\color{#d91a1a}-0.28\%$
test_values[vec_td1_return_estimate-False-False] 2.0921ms 1.7590ms 568.5090 Ops/s 568.0607 Ops/s $\color{#35bf28}+0.08\%$
test_values[td_lambda_return_estimate-True-False] 88.8953ms 85.1201ms 11.7481 Ops/s 12.1742 Ops/s $\color{#d91a1a}-3.50\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0892ms 1.7539ms 570.1658 Ops/s 569.8347 Ops/s $\color{#35bf28}+0.06\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.3283ms 24.5888ms 40.6690 Ops/s 42.5155 Ops/s $\color{#d91a1a}-4.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8659ms 0.6829ms 1.4644 KOps/s 1.4520 KOps/s $\color{#35bf28}+0.85\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7276ms 0.6698ms 1.4930 KOps/s 1.5589 KOps/s $\color{#d91a1a}-4.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5412ms 1.4584ms 685.6789 Ops/s 689.9469 Ops/s $\color{#d91a1a}-0.62\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9319ms 0.6583ms 1.5191 KOps/s 1.5064 KOps/s $\color{#35bf28}+0.84\%$
test_dqn_speed 75.0650ms 1.6031ms 623.7798 Ops/s 686.9225 Ops/s $\textbf{\color{#d91a1a}-9.19\%}$
test_ddpg_speed 3.1826ms 2.9923ms 334.1868 Ops/s 340.8732 Ops/s $\color{#d91a1a}-1.96\%$
test_sac_speed 9.3571ms 8.4594ms 118.2120 Ops/s 117.2634 Ops/s $\color{#35bf28}+0.81\%$
test_redq_speed 10.9115ms 10.1364ms 98.6545 Ops/s 97.9112 Ops/s $\color{#35bf28}+0.76\%$
test_redq_deprec_speed 12.2514ms 11.5124ms 86.8630 Ops/s 88.1863 Ops/s $\color{#d91a1a}-1.50\%$
test_td3_speed 17.2384ms 8.5021ms 117.6174 Ops/s 119.8212 Ops/s $\color{#d91a1a}-1.84\%$
test_cql_speed 26.9978ms 25.7935ms 38.7695 Ops/s 38.8799 Ops/s $\color{#d91a1a}-0.28\%$
test_a2c_speed 5.9265ms 5.7111ms 175.0990 Ops/s 180.3365 Ops/s $\color{#d91a1a}-2.90\%$
test_ppo_speed 6.2163ms 5.9931ms 166.8572 Ops/s 168.6711 Ops/s $\color{#d91a1a}-1.08\%$
test_reinforce_speed 4.8885ms 4.6379ms 215.6139 Ops/s 220.5037 Ops/s $\color{#d91a1a}-2.22\%$
test_iql_speed 20.3767ms 19.5126ms 51.2490 Ops/s 51.1258 Ops/s $\color{#35bf28}+0.24\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9070ms 2.7821ms 359.4433 Ops/s 359.2516 Ops/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1016s 0.6721ms 1.4878 KOps/s 1.7128 KOps/s $\textbf{\color{#d91a1a}-13.14\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7461ms 0.5608ms 1.7831 KOps/s 1.7885 KOps/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0197ms 2.8097ms 355.9116 Ops/s 357.4887 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7677ms 0.5859ms 1.7067 KOps/s 1.7353 KOps/s $\color{#d91a1a}-1.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7417ms 0.5538ms 1.8058 KOps/s 1.7963 KOps/s $\color{#35bf28}+0.53\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8638ms 1.4990ms 667.0945 Ops/s 672.9270 Ops/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5287ms 1.4125ms 707.9408 Ops/s 704.2891 Ops/s $\color{#35bf28}+0.52\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0002ms 2.9033ms 344.4414 Ops/s 345.7773 Ops/s $\color{#d91a1a}-0.39\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8784ms 0.7088ms 1.4109 KOps/s 1.4145 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.5830ms 0.6900ms 1.4492 KOps/s 1.4677 KOps/s $\color{#d91a1a}-1.26\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9927ms 2.7887ms 358.5894 Ops/s 361.0870 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7648ms 0.5827ms 1.7160 KOps/s 1.6919 KOps/s $\color{#35bf28}+1.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.5332ms 0.5647ms 1.7708 KOps/s 1.7620 KOps/s $\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0337ms 2.7994ms 357.2210 Ops/s 357.9841 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5796ms 0.5765ms 1.7347 KOps/s 1.7270 KOps/s $\color{#35bf28}+0.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6998ms 0.5533ms 1.8074 KOps/s 1.8005 KOps/s $\color{#35bf28}+0.38\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0711ms 2.9499ms 338.9950 Ops/s 346.3018 Ops/s $\color{#d91a1a}-2.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8421ms 0.7065ms 1.4154 KOps/s 1.4060 KOps/s $\color{#35bf28}+0.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8653ms 0.6828ms 1.4645 KOps/s 1.4488 KOps/s $\color{#35bf28}+1.08\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1336s 9.6726ms 103.3852 Ops/s 103.3281 Ops/s $\color{#35bf28}+0.06\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.2036ms 14.8305ms 67.4286 Ops/s 67.1354 Ops/s $\color{#35bf28}+0.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.2797ms 1.1451ms 873.3028 Ops/s 852.4956 Ops/s $\color{#35bf28}+2.44\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1156s 7.1178ms 140.4921 Ops/s 139.2266 Ops/s $\color{#35bf28}+0.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.1649ms 14.6691ms 68.1707 Ops/s 67.3526 Ops/s $\color{#35bf28}+1.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.2330ms 1.1493ms 870.0581 Ops/s 843.5522 Ops/s $\color{#35bf28}+3.14\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1192s 9.7514ms 102.5489 Ops/s 132.9313 Ops/s $\textbf{\color{#d91a1a}-22.86\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.6606ms 15.0565ms 66.4164 Ops/s 57.2715 Ops/s $\textbf{\color{#35bf28}+15.97\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.9292ms 1.6254ms 615.2454 Ops/s 613.7739 Ops/s $\color{#35bf28}+0.24\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vmoens @facebook-github-bot