Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: passing feature flags #154

Open
elichai opened this issue Jan 21, 2020 · 3 comments
Open

Feature Request: passing feature flags #154

elichai opened this issue Jan 21, 2020 · 3 comments

Comments

@elichai
Copy link

elichai commented Jan 21, 2020

Hi,
First this crate is awesome :) Thank you!.
Second, In my view there are 2 important features missing to add this to a CI:

  1. Feature gate the mutations, in real code you don't want to risk mutagen affecting the code with regular tests and the compilation overhead.
    (a naive solution is master...TheBlueMatt:master a real one involves the ability of passing arbitrary feature flags when running cargo mutagen)

2. There are some mutations that can't be "fixed" with more tests, because their path isn't really reachable or the mutation doesn't really matter to the function, so a way to add "ignore" to mutations, either with a file (like sanitizers supressions) or with attributes (like #[allow(...)]) that can be also feature gated. (opened #156)

Thanks,
Elichai.

@samuelpilz
Copy link
Contributor

samuelpilz commented Jan 22, 2020

Thank you for your feedback.

Did I understand your first request correctly: "make it possible to run cargo mutagen --features x,y,z and cargo mutagen --all-features"? I can do that.

The diff also includes a panic when a single mutation is not killed. Insisting on 100% mutation score is not the intention of mutation testing. However, I understand that regressions in mutation score can be caught in CI. Could you open a seperate issue for that?

The topic of skipping a mutations, should also be discussed in a separate issue. I have experimented with some design, but waited for feedback until publishing that featureset.

PS: what version of mutagen are you using?

@elichai
Copy link
Author

elichai commented Jan 22, 2020

I agree about the panic, Altough it can make CI's easier if it fails on first mutation (assuming I can skip some)

And I'm currently using master.

@samuelpilz
Copy link
Contributor

The first part is implemented in #155. Would it be possible to open separate issues for the remaining concerns?

@elichai elichai changed the title Feature Request: passing feature flags, and ignoring mutations Feature Request: passing feature flags Jan 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants