[tests] Workload tests #4168
[tests] Workload tests #4168
Conversation
dotnet-issue-labeler
bot
added
the
area-components
radical
force-pushed
the
workload-tests-prep
branch
2 times, most recently
from
df23466
to
abd154e
Compare
…y handles 'nothing running' case
- also show process run times which is useful in identifying hangs
radical
force-pushed
the
workload-tests-prep
branch
from
c2f7e55
to
8f64d95
Compare
- also, deploy the chromium browser for helix
- for this we use a `PlaywrightWrapper` project to invoke to run
`Main` for `Playwright` and use it to provision the browser.
This can be used to prepare a shell environment for using the sdk+workload from `artifacts/bin/dotnet-latest`. `$ source /path-to-aspire-repo/dogfood.sh`
radical
force-pushed
the
workload-tests-prep
branch
from
8f64d95
to
e7ffa9e
Compare
| @@ -0,0 +1,28 @@ | |||
| if [ -z "$ZSH_VERSION" ]; then | |||
There was a problem hiding this comment.
NIT: should probably have #!/usr/bin/env bash at the top line of this script.
There was a problem hiding this comment.
Also, I don't see in other scripts in the repo us checking if you are using Bash or Zsh, is there a reason why we need that here?
There was a problem hiding this comment.
This is meant to be used like source dogfood.sh, so that wouldn't be appropriate AFAIU.
There was a problem hiding this comment.
${BASH_SOURCE[0]} is not supported in zsh.
There was a problem hiding this comment.
I see, I think that having it be a .sh will be misleading then if the intent is to do source dogfood.sh. Just curious, why not make it an actual bash script that can be executed? Here is precedent of a very similar use cas in the sdk repo: https://github.com/dotnet/sdk/blob/main/eng/dogfood.sh
There was a problem hiding this comment.
If the script is run as /foo/dogfood.sh then the changes to the environment will be limited to that script. The documentation in sdk also suggests using source dogfood.sh:
When using source .. we are using the current shell to run the commands so the shebang should be not be needed, AFAIU.
…Directory.Build.props
…feedback from @ eerhardt
The template generators can handle odd characters in the project name, so this isn't needed anymore.
.. and instead use a pre-generated `runtimeconfig.json` to directly run `Microsoft.Playwright.dll`. Based on feedback from @ eerhardt .
|
|
||
| <RunTestsOnHelix>true</RunTestsOnHelix> | ||
| <TestUsingWorkloads>true</TestUsingWorkloads> | ||
| <InstallWorkloadForTesting>true</InstallWorkloadForTesting> |
There was a problem hiding this comment.
| <InstallWorkloadForTesting>true</InstallWorkloadForTesting> |
I don't think this is needed, since it defaults to TestUsingWorkloads
There was a problem hiding this comment.
It doesn't default to TestUsingWorkloads.
The workload testing targets install two sdks - dotnet-latest, and dotnet-none. The former has the manifest+workload installed, but that latter has only the manifest installed. It is useful for running tests that want to check the case the user does not have the workload installed.
The TestUsingWorkloads property essentially controls that. I'll open a PR in dotnet/runtime to rename this property so its purpose is clearer. And possibly clean this up a bit.
| </Exec> | ||
|
|
||
| <PropertyGroup> | ||
| <_Regex>^\s*(Aspire.Workload.Tests[^\($]+)</_Regex> |
There was a problem hiding this comment.
What happens if a new test gets added that doesn't follow this naming? Does the test just get skipped and never executed?
There was a problem hiding this comment.
Yeah :/ I'll open an issue to make this more robust.
| { | ||
| public IEnumerable<KeyValuePair<string, string>> GetTraits(IAttributeInfo traitAttribute) | ||
| { | ||
| if (!RequiresDockerTheoryAttribute.IsSupported) |
There was a problem hiding this comment.
(not necessarily for this PR) - it would be nice if we had a consolidated "RequiresDocker" story across all our tests. This feels pretty spread out - some in Aspire.Components.Common.Tests, some here, some in Aspire.Hosting.Tests.
Co-authored-by: Eric Erhardt <[email protected]>
Co-authored-by: Eric Erhardt <[email protected]>
This was used for inserting a source for the locally built nugets. But this was changed to instead have `<add key="built-local" value="%BUILT_NUGETS_PATH%" />` so the path comes from the environment variable `BUILT_NUGETS_PATH` now, and we can avoid needing to patch the nuget.config.
This adds
Aspire.Workload.Testswhich are meant to exercise theaspireworkload. The individual tests create projects from templates just like a user would, and then run, and validate them.It uses the existing infrastructure from
Aspire.EndToEnd.Teststo install sdk+workload for testing.Tests
Two kinds of tests are being added here:
aspire-startertemplate project, and then some tests interact with that running app to validate itaspire-starterwith [xunit,nunit,mstest] optionsPlaywright.aspire-startertemplate to validate some parts of it.Fixes #47 .
Follow up work is tracked in #4222
Microsoft Reviewers: Open in CodeFlow