build-server.md

Build Server

Getting .received files from CI

AppVeyor

Use a on_failure build step to call Push-AppveyorArtifact.

on_failure:
  - ps: Get-ChildItem *.received.* -recurse | % { Push-AppveyorArtifact $_.FullName -FileName $_.Name }

See also Pushing artifacts from scripts.

GitHub Actions

Use a if: failure() condition to upload any *.received.* files if the build fails.

- name: Upload Test Results
  if: failure()
  uses: actions/upload-artifact@v2
  with:
    name: verify-test-results
    path: |
      **/*.received.*

Azure DevOps YAML Pipeline

Directly after the test runner step add a build step to set a flag if the testrunner failed. This is done by using a failed condition. This flag will be evaluated in the CopyFiles and PublishBuildArtifacts steps below.

- task: CmdLine@2
  displayName: 'Set flag to publish Verify *.received.* files when test step fails'
  condition: failed()
  inputs:
    script: 'echo ##vso[task.setvariable variable=publishverify]Yes'

Since the PublishBuildArtifacts step in DevOps does not allow a wildcard it is necessary to need stage the 'received' files before publishing:

- task: CopyFiles@2
  condition: eq(variables['publishverify'], 'Yes')
  displayName: 'Copy Verify *.received.* files to Artifact Staging'
  inputs:
    contents: '**\*.received.*' 
    targetFolder: '$(Build.ArtifactStagingDirectory)\Verify'
    cleanTargetFolder: true
    overWrite: true

Publish the staged files as a build artifact:

- task: PublishBuildArtifacts@1
  displayName: 'Publish Verify *.received.* files as Artifacts'
  name: 'verifypublish'
  condition: eq(variables['publishverify'], 'Yes')
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)\Verify'
    ArtifactName: 'Verify'
    publishLocation: 'Container'

Custom directory and file name

In some scenarios, as part of a build, the test assemblies are copied to a different directory or machine to be run. In this case custom code will be required to derive the path to the .verified. files. This can be done using DerivePathInfo.

For example a possible implementation for AppVeyor could be:

if (BuildServerDetector.Detected)
{
    var buildDirectory = Environment.GetEnvironmentVariable("APPVEYOR_BUILD_FOLDER")!;
    DerivePathInfo(
        (sourceFile, projectDirectory, typeName, methodName) =>
        {
            var testDirectory = Path.GetDirectoryName(sourceFile)!;
            var testDirectorySuffix = testDirectory.Replace(projectDirectory, string.Empty);
            return new(directory: Path.Combine(buildDirectory, testDirectorySuffix));
        });
}

^{snippet source | anchor}

Attachments

For MSTest and NUnit, Verify will add new received files to the test context's attachments. So for build servers that respect MSTest or NUnit attachment APIs, no changes are required to build configurations.

• NUnit Attachments
• MSTest TestContext.AddResultFile