Skip to content

A simple benchmark comparing two popular automatic test generation tools

License

Notifications You must be signed in to change notification settings

btuso/automatic-test-generation-benchmark

Repository files navigation

automaticTestGenerationBenchmark

A simple benchmark comparing two popular automatic test generation tools

  • To generate all tests, use "gradle generateTests"
  • To run all generated tests, use "gradle runAllTests"
  • To create coverage reports for the generated tests, use "gradle allCoverages"
  • To create mutation reports for the generated tests, use "gradle allMutations"
  • To create a csv file with coverage information for the tests, use "gradle exportCSV"
  • To clean up generated files and tests, use "gradle cleanUp"

Please note that some of these tasks depend on a previous task. i.e: generateTests must be called before runAllTests

Several convenience methods are available and can be listed by using "gradle tasks"

Available parameters:

  • Time spent creating tests: -PbudgetTime=X
  • Amount of test suites to generate for each class: -PtestSuites=X

Examples:

Generate a csv with information for 10 tests suites, using 2 minutes as a budget

gradle generateTests runAllTests allCoverages allMutations exportCSV -PbudgetTime=120 -PtestSuites=10

Generate and measure only randoop tests

gradle randoop runRandoopTests randoopCoverage randoopMutation exportCSV

Csv output format

class name, tool name, line coverage, branch coverage, mutation score

About

A simple benchmark comparing two popular automatic test generation tools

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages