You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Now that .NET 6 was officially released, along with Visual Studio 2022 as the only version of VS that officially supports it, I've been running benchmarks comparing performance with the previous .NET 5 builds.
The only changes I've made have been the target version of .NET. All the code remains the same.
Some algorithms don't seem to change at all (mostly the simplest algorithms). A few seem to gain about 2%-5% in performance. And a few have suspiciously large gains. This last case seems to be an effect which causes the .NET 5 tests to have substantially large variations for no reason that I can immediately identify.
One benchmark (using BenchmarkDotNet) varied between 820 us and 895 us in .NET 5, while the .NET 6 version was 753 us, so the gain could be between 8% and 16%.
Running the loop count saw one algorithm have the highest gain, at about an 8% increase in the number of loops, but it was also the most complex algorithm ([8 of 30] with bitmasking).
Anyway, individual scores seem to vary by up to 4% per run, which is problematic when comparing tests. I feel comfortable saying that a typical algorithm is likely to improve by ~3% (but may not improve at all), but anything beyond that shouldn't be trusted unless you did some major rewriting to take advantage of new .NET 6 tools.
Notable differences from Readme.md (test from several months ago using NET 5 in VS 2019), to NET 5 in VS2019, to NET 5 in VS 2022, to NET 6 in VS 2022:
bool array: 6643 → 5756 → 5652 → 6285
inverted bool array: 8800 → 7274 → 7107 → 8926
direct access inverted bool array: 8964 → 7247 → 7110 → 8928
array pool: 6084 → 6096 → 6034 → 6658
array pool [2 of 6]: 9670 → 9882 → 9528 → 9468
array pool [8 of 30]: 12467 → 12421 → 12265 → 12821
array pool [8 of 30] with bitmasking: 15611 → 15643 → 15817 → 17308
What I see is that several tests are running slower now than they were a few months ago, but compared to .NET 5, .NET 6 is running faster.
There's only two cases where it's clearly faster in .NET 6 than in any .NET 5 run: 1) the basic array pool version; and 2) the [8 of 30] array pool using bitmasking — rather more complicated code than the other algorithms.
The bool array algorithms also run faster than the .NET 5 versions, but only on par with the older tests, which makes me a bit uncertain on whether this is a real performance gain or not.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Now that .NET 6 was officially released, along with Visual Studio 2022 as the only version of VS that officially supports it, I've been running benchmarks comparing performance with the previous .NET 5 builds.
The only changes I've made have been the target version of .NET. All the code remains the same.
Some algorithms don't seem to change at all (mostly the simplest algorithms). A few seem to gain about 2%-5% in performance. And a few have suspiciously large gains. This last case seems to be an effect which causes the .NET 5 tests to have substantially large variations for no reason that I can immediately identify.
One benchmark (using BenchmarkDotNet) varied between 820 us and 895 us in .NET 5, while the .NET 6 version was 753 us, so the gain could be between 8% and 16%.
Running the loop count saw one algorithm have the highest gain, at about an 8% increase in the number of loops, but it was also the most complex algorithm ([8 of 30] with bitmasking).
Anyway, individual scores seem to vary by up to 4% per run, which is problematic when comparing tests. I feel comfortable saying that a typical algorithm is likely to improve by ~3% (but may not improve at all), but anything beyond that shouldn't be trusted unless you did some major rewriting to take advantage of new .NET 6 tools.
BenchmarkDotNet comparison numbers
.NET 5
.NET 6
Latest run data
NET 6 in VS 2022
Notable differences from Readme.md (test from several months ago using NET 5 in VS 2019), to NET 5 in VS2019, to NET 5 in VS 2022, to NET 6 in VS 2022:
bool array: 6643 → 5756 → 5652 → 6285
inverted bool array: 8800 → 7274 → 7107 → 8926
direct access inverted bool array: 8964 → 7247 → 7110 → 8928
array pool: 6084 → 6096 → 6034 → 6658
array pool [2 of 6]: 9670 → 9882 → 9528 → 9468
array pool [8 of 30]: 12467 → 12421 → 12265 → 12821
array pool [8 of 30] with bitmasking: 15611 → 15643 → 15817 → 17308
What I see is that several tests are running slower now than they were a few months ago, but compared to .NET 5, .NET 6 is running faster.
There's only two cases where it's clearly faster in .NET 6 than in any .NET 5 run: 1) the basic array pool version; and 2) the [8 of 30] array pool using bitmasking — rather more complicated code than the other algorithms.
The bool array algorithms also run faster than the .NET 5 versions, but only on par with the older tests, which makes me a bit uncertain on whether this is a real performance gain or not.
@Kinematics: Starting (original)...
Passes: 3545, Time: 5.00124 s, Per Loop: 1.410719 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_original;3545;5.00124;1;algorithm=base,faithful=yes,bits=1
@Kinematics: Starting (standard)...
Passes: 3687, Time: 5.00099 s, Per Loop: 1.356116 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_standard;3687;5.00099;1;algorithm=base,faithful=yes,bits=1
@Kinematics: Starting (bool array)...
Passes: 6285, Time: 5.00022 s, Per Loop: 0.795545 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_bool;6285;5.00022;1;algorithm=base,faithful=yes,bits=8
@Kinematics: Starting (inverted bool array)...
Passes: 8926, Time: 5.00009 s, Per Loop: 0.560161 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_ibool;8926;5.00009;1;algorithm=base,faithful=yes,bits=8
@Kinematics: Starting (direct access inverted bool array)...
Passes: 8928, Time: 5.00052 s, Per Loop: 0.560036 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_dbool;8928;5.00052;1;algorithm=base,faithful=yes,bits=8
@Kinematics: Starting (raw bits)...
Passes: 6449, Time: 5.00052 s, Per Loop: 0.775314 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_raw;6449;5.00052;1;algorithm=base,faithful=yes,bits=1
@Kinematics: Starting (raw bits uint)...
Passes: 6616, Time: 5.00009 s, Per Loop: 0.755744 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_raw32;6616;5.00009;1;algorithm=base,faithful=yes,bits=1
@Kinematics: Starting (raw bits direct)...
Passes: 6454, Time: 5.00025 s, Per Loop: 0.774713 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_rawd;6454;5.00025;1;algorithm=base,faithful=yes,bits=1
@Kinematics: Starting (raw bits [2 of 6])...
Passes: 6125, Time: 5.00021 s, Per Loop: 0.816327 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_raw6;6125;5.00021;1;algorithm=wheel,faithful=yes,bits=1
@Kinematics: Starting (raw bits parallel version)...
Passes: 4366, Time: 5.00071 s, Per Loop: 1.145213 ms, Sieve Size: 1000000, Thread Count: 1, Parallel Thread Count: 16, Primes Found: 78498, Valid: True
kinematics_rawp;4366;5.00071;16;algorithm=base,faithful=yes,bits=1
@Kinematics: Starting (array pool)...
Passes: 6658, Time: 5.00036 s, Per Loop: 0.750976 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool;6658;5.00036;1;algorithm=base,faithful=yes,bits=1
@Kinematics: Starting (array pool [2 of 6])...
Passes: 9468, Time: 5.00034 s, Per Loop: 0.528095 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool2of6;9468;5.00034;1;algorithm=wheel,faithful=yes,bits=1
@Kinematics: Starting (parallel array pool [2 of 6])...
Passes: 5531, Time: 5.0002 s, Per Loop: 0.903996 ms, Sieve Size: 1000000, Thread Count: 1, Parallel Thread Count: 16, Primes Found: 78498, Valid: True
kinematics_pool6p;5531;5.0002;16;algorithm=wheel,faithful=yes,bits=1
@Kinematics: Starting (array pool [8 of 30])...
Passes: 12821, Time: 5.0002 s, Per Loop: 0.389985 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool30;12821;5.0002;1;algorithm=wheel,faithful=yes,bits=1
@Kinematics: Starting (array pool [8 of 30] with bitmasking)...
Passes: 17308, Time: 5.00002 s, Per Loop: 0.288884 ms, Sieve Size: 1000000, Thread Count: 1, Primes Found: 78498, Valid: True
kinematics_pool30m;17308;5.00002;1;algorithm=wheel,faithful=yes,bits=1
Beta Was this translation helpful? Give feedback.
All reactions