Skip to content

Commit

Permalink
WIP: bench: introduce initial benchmarking infrastructure
Browse files Browse the repository at this point in the history
To better understand the current performance characteristics and to
guide future development, introduce microbenchmarks for various steps in
the `cwe_checker`'s analysis.

This commit adds benchmarks for the following steps:

- CFG construction,
- individual normalization passes,
- function signatures analysis,
- pointer inference,
- string abstractions,
- individual checkers.

All benchmarks are executed on the following input programs:

- ls,
- netfs.ko,

for the following architectures:

- amd64,
- arm64,
- armel,
- armhf,
- mips64el,
- mipsel,
- ppc64el,
- x86.

Inputs are stored in the `benches/_data/` directory. It contains
json-serialized pcode projects and binaries. The pcode projects are
included for the following reasons:

- not requiring a Ghidra installation on the benchmarking system,
- avoid that changes in Ghidra version influence the benchmark results,
- reduce the time it takes to run the benchmarks.

The inputs are not included in the repository and have to be obtained
from an external source. Since the binaries are taken from Debian
packages it shouldn't be a problem to re-distribute them if we include
the copyright information.

This implementation is based on the Criterion.rs crate. It is added to
the dev-dependencies of the `cwe_checker_lib` and the `Cargo.lock` is
updated accordingly.

Signed-off-by: Valentin Obst <[email protected]>
  • Loading branch information
Valentin Obst committed May 7, 2024
1 parent 01aa713 commit 364dc03
Show file tree
Hide file tree
Showing 6 changed files with 920 additions and 0 deletions.
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
test/**/build
target/
src/cwe_checker_lib/benches/_data/
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,8 @@ test/run_real_world_samples.sh
doc/html

test/artificial_samples/dockcross*
src/cwe_checker_lib/benches/_data/*
!src/cwe_checker_lib/benches/_data/.gitkeep

.#*

Expand Down
270 changes: 270 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 364dc03

Please sign in to comment.