-
Notifications
You must be signed in to change notification settings - Fork 223
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi-line test commands can spuriously pass #1732
Comments
You can use [tool.cibuildwheel]
test-command = [
"pip install -r {project}/requirements-dev.txt",
"pytest {project}/ -k 'not memory_usage'",
"pytest {project}/ -k 'memory_usage'",
] |
Thanks, good to know! The default still feels like a footgun though. |
I agree, it is a shame, and it is a footgun. Unfortunately I don't think there is a way to get Options would be:
|
Seems Powershell also has bad defaults 😢 With latest versions apparently this is how to fix them: PowerShell/PowerShell#3415 (comment) |
Description
Let's say
CIBW_TEST_COMMAND="false; true"
. On Unix this will be considered a success, even though one of the commands failed.Here's a real example that tripped me up:
Now, you could say that this is my responsibility and I should just add a
set -e
. Butset -e
I assume won't work on Windows, where the same commands will be run. So making sure failing commands cause tests to not spuriously pass is probably somethingcibuildwheel
should do.(I can do separate tests for Linux and Windows, but that's a workaround which leads to unnecessary duplication of scripts. This feels like a footgun that has probably hit other projects too. There's a reason most CI systems have
set -e
as the default.)Build log
No response
CI config
No response
The text was updated successfully, but these errors were encountered: