Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the fluent assertions version to improve test failure output. #17982

Open
wants to merge 7 commits into
base: release/8.0.1xx
Choose a base branch
from

Conversation

marcpopMSFT
Copy link
Member

Include a failing tests to verify if the output is better.

@marcpopMSFT
Copy link
Member Author

@vitek-karas I was looking at dotnet/runtime@faa272f which improved the output for the tests in AzDo. That was later removed when fluent assertions was updated but making the same change here was not sufficient. Do you know why your workaround was no longer needed in runtime?

@marcpopMSFT
Copy link
Member Author

Also, AssertionScope doesn't appear to have a definition for AddFailure. I'm trying to figure out why it did in the runtime when your change went in but doesn't here.

@marcpopMSFT
Copy link
Member Author

Updating the version of Fluent didn't work and I still can't get the AddFailure workaround that Vitek had put into place to work in this repo. @vitek-karas how did you end up being able to remove your workaround and why doesn't it work for
C:\repos\installer\test\Microsoft.DotNet.Tools.Tests.Utilities\Assertions\AssertionScopeExtensions.cs(14,32): error CS1061: 'AssertionScope' does not contain a definition for 'AddFailure' and no access ible extension method 'AddFailure' accepting a first argument of type 'AssertionScope' could be found (are you missing a using directive or an assembly reference?) [C:\repos\installer\test\Microsoft.Do tNet.Tools.Tests.Utilities\Microsoft.DotNet.Tools.Tests.Utilities.csproj]

@vitek-karas
Copy link
Member

The FluentAssertions library changed. The version I was working against back then was 4.19.4, you're working with 6.12.0.
Looking in disassembler, it seems the method just got renamed though. Now it's AssertionScope.AddPreFormattedFailure - its implementation is identical to the old AssertionScope.AddFailure.

@marcpopMSFT
Copy link
Member Author

@vitek-karas thanks for the pointer. I missed that originally as there were two spots we have the fluent version listed and I hadn't updated the one the tests actually use.

The issue I'm having now is that the assertionScope.Succeeded was moved to be internal so I can't check when to call AddPreFormattedFailure. The FailWith method has access to that property and it uses it to determine whether to record the failure. There's a HasFailures method but it checks if there is already a failure (which there isn't at the time of the call).

I was not able to find any good examples of how to detect the failure and call AddPreFormattedFailure at that time and the new version of Fluent still has the new line issues. Any other ideas?

@akoeplinger
Copy link
Member

I looked into this a bit and the original FluentAssertion issue is indeed fixed but now you're running into the xunit console runner issue where it still escapes newlines in the xml result file in the <message>: dotnet/runtime#1664
image

This was worked around for runtime by unescaping the xml as part of the helix reporter: dotnet/arcade#7323

The installer repo doesn't use Helix so you're still hitting this, there is a workaround though: switch to the VSTest runner which produces .trx output files.

I've pushed a change to do that, here's what the the trx and html files look now:

image image

@akoeplinger
Copy link
Member

It works in AzDO:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants