Feature request: randomly sample from a parameter list

[tavianator]

I am trying to write some production-grade benchmarks for bfs with early termination, something like

bfs -name pattern -quit

where pattern is the name of some file at a particular depth. However, I don't want to use the same file every time, because performance varies a lot depending on the order that files are visited in. A single file isn't representative.

This seems like a good case for -L, like this

hyperfine -L name foo,bar,baz "bfs -name {name} -quit"

except that generates three different command lines and compares them against each other.

In real life I have a few dozen names, and I don't really care about the performance of each one separately. What I want is a mode that picks a random {name} every time it runs the command, so I could do things like

hyperfine --parameter-sample name foo,bar,baz "bfs -name {name} -quit" "find -name {name} -quit"

and compare bfs to find over the whole set of names. Right now I'm hacking it with shuf:

$ bfs -depth 5 -type f -printf '%f\n' | shuf -n20 >files
$ hyperfine -w1 'bfs -name $(shuf -n1 files) -quit' 'find -name $(shuf -n1 files) -quit'
Benchmark 1: bfs -name $(shuf -n1 files) -quit
  Time (mean ± σ):      22.2 ms ±  12.9 ms    [User: 17.5 ms, System: 98.0 ms]
  Range (min … max):     0.9 ms …  53.2 ms    52 runs
 
Benchmark 2: find -name $(shuf -n1 files) -quit
  Time (mean ± σ):      46.9 ms ±  35.1 ms    [User: 15.2 ms, System: 31.4 ms]
  Range (min … max):     5.2 ms … 107.3 ms    40 runs
 
Summary
  bfs -name $(shuf -n1 files) -quit ran
    2.11 ± 2.00 times faster than find -name $(shuf -n1 files) -quit

[sharkdp]

Thank you. Sounds like a useful feature. I would assume that most use cases for such an option would include numeric parameters, not a list of finite options. Do you think we could come up with a good CLI for this without having to add too many options?