Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TAP reporter - needs an event::on_tests_begin #591

Open
talisein opened this issue Oct 16, 2023 · 0 comments
Open

TAP reporter - needs an event::on_tests_begin #591

talisein opened this issue Oct 16, 2023 · 0 comments

Comments

@talisein
Copy link
Contributor

talisein commented Oct 16, 2023

The JUnit output is nice, but I'd like TAP as well. But I think I need a new event, on_tests_begin() that would have a size_t num_tests member that indicates how many tests are planned to have run (but they haven't run yet!)

TAP output is like:

1..2     # The plan is to execute 2 tests
ok 1     # The first test passed
not ok 2 # the second test failed
EOF

The idea of reporting how many tests are planned to execute is that if one test causes the whole executable to abort, anything parsing the output can understand that it is incomplete. e.g. the following can be caught as a failed test run:

1..2
ok 1
EOF

I've already implemented a similar on_suites_begin() event, since there is a std::vector suites_ in the runner, but I don't understand the code well enough to make an on_tests_begin(). I suspect each test is simply run as it is encountered and there's no way to count them beforehand. Am I correct about that? Any ideas? In the worst case I can accumulate test results in the TAP reporter state and just output everything at once at the end, but it is less than ideal.

@talisein talisein changed the title TAP reporter - needs an event::on_tests TAP reporter - needs an event::on_tests_begin Oct 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant