-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fail Betterer on expired deadline #1180
Comments
Thinking about this further. I think there are at least 2 additional data points you'd need to capture: the first recorded value for the test, and the most recent deadline. Because, in the example of seeing a number get smaller by a deadline, you need to know:
Deadline First Set (current date: Nov 1 2023)Result: 'First Seen' exports[`check linters`] = {
+ initValue: `40`,
value: `40`,
+ deadline: `2024-01-01`'
}; Betterer Run Before DeadlineResult: 'Doing Better' exports[`check linters`] = {
initValue: `40`,
- value: `40`,
+ value: `30`,
deadline: `2024-01-01`'
}; Betterer Run After Deadline (Pass Case)Result: 'Deadline Expired' exports[`check linters`] = {
initValue: `40`,
value: `30`,
deadline: `2024-01-01`'
}; Betterer Run After Deadline (Fail Case)Result: 'Failed: Deadline Reached With No Improvement' exports[`check linters`] = {
initValue: `40`,
value: `40`,
deadline: `2024-01-01`'
}; Deadline UpdatedResult: 'First Seen' exports[`check linters`] = {
- initValue: `40`,
+ initValue: `30`,
value: `30`,
- deadline: `2024-01-01`'
+ deadline: `2024-02-01`'
}; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We currently have a use case where we would like test coverage to increase by a certain deadline. If coverage does not increase, we would like to fail Betterer, prompting an Engineer to increase coverage at this point, before continuing to ship features.
If the goal has been met and the deadline has passed, we would like Betterer to pass with exit code
0
and mark the test as expired.If the goal not been met, and the deadline has passed, we would like Betterer to fail with exit code
1
until either, the deadline has been increased, or the goal has been met. If the goal has been met, it would go back to expired behavior with success code.Without Betterer, we would employ this restriction at an organizational level. (but it would need to be a far more manual process)
Does Betterer provide functionality like this?
The text was updated successfully, but these errors were encountered: