Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[test] fix integration tests #19692

Merged
merged 14 commits into from May 7, 2024
Merged

[test] fix integration tests #19692

merged 14 commits into from May 7, 2024

Conversation

kylos101
Copy link
Collaborator

@kylos101 kylos101 commented May 1, 2024

Description

A few tests have been consistently failing.

  1. TestNetworkLimiting was failing because the user we use for testing in previews does not use UBP, and the default is not to network limit for those users. I've changed server, so that we always limit network connections regardless of billing mode, and to also limit if there was an error inspecting the billing mode.
  2. TestProcessPriority was failing because it was running the assertion too early. We need to give ws-daemon time to apply process priority before asserting priority has been adjusted.
  3. TestOpenWorkspaceFromPrebuild had a couple problems. First, grep needs to be set to explicitly recurse through folders (it was failing on the logs folder). Second, the test wasn't waiting for a workspace to stop, prior to attempting to restart the workspace.

Related Issue(s)

Relates to ENT-13

How to test

The tests are green now. After this lands on main, we'll need to trigger a manual run with the updated main-gha asset. Why? We only test the last successful build of main-gha when testing, and the tests passing are considered part of the success.

Documentation

Preview status

Gitpod was successfully deployed to your preview environment.

Build Options

Build
  • /werft with-werft
    Run the build with werft instead of GHA
  • leeway-no-cache
  • /werft no-test
    Run Leeway with --dont-test
Publish
  • /werft publish-to-npm
  • /werft publish-to-jb-marketplace
Installer
  • analytics=segment
  • with-dedicated-emulation
  • workspace-feature-flags
    Add desired feature flags to the end of the line above, space separated
Preview Environment / Integration Tests
  • /werft with-local-preview
    If enabled this will build install/preview
  • /werft with-preview
  • /werft with-large-vm
  • /werft with-gce-vm
    If enabled this will create the environment on GCE infra
  • /werft preemptible
    Saves cost. Untick this only if you're really sure you need a non-preemtible machine.
  • with-integration-tests=workspace
    Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
  • with-monitoring

/hold

Need to give ws-daemon time before it applies process priority
We are not limiting network connections for preview environments
* Regardless of billing mode
* If there's an error
It was failing with an exit code of 2 on grep, because it was trying to grep a folder, rather than recurse into it.

When the grep command was run manually from a workspace that started from a prebuild, we saw `grep: logs: Is a directory`. So, it's important for integration tests to output the exit code, but also error text.
We need to wait for the started workspace to stop, before we try to restart it
@roboquat roboquat added size/M and removed size/S labels May 3, 2024
@roboquat roboquat added size/S and removed size/M labels May 3, 2024
@kylos101 kylos101 marked this pull request as ready for review May 4, 2024 20:36
@kylos101 kylos101 requested review from a team as code owners May 4, 2024 20:36
Copy link
Member

@geropl geropl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kylos101 kylos101 marked this pull request as draft May 6, 2024 17:51
@kylos101 kylos101 marked this pull request as ready for review May 7, 2024 02:23
@kylos101 kylos101 requested a review from geropl May 7, 2024 02:23
Copy link
Member

@geropl geropl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️

@roboquat roboquat merged commit 606963a into main May 7, 2024
16 checks passed
@roboquat roboquat deleted the kylos101/gen111-tests branch May 7, 2024 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants