Fix the RecommendedExtensions test #22966

musienko-maxim · 2024-05-15T10:56:50Z

What does this PR do?

Fix problem for performing when recommended extension more than one. And correct select and search extensions by author with the find menu. Also was improved and simplified mapping of recommended extension objects. Add blocks which close WorkspaceTrustBox menus in typical places

Screenshot/screencast of this PR

> @eclipse-che/[email protected] test
> ./configs/sh-scripts/initDefaultValues.sh npm run lint && npm run tsc && export USERSTORY=$USERSTORY && mocha --config dist/configs/mocharc.js

Initialized default values

TS_SELENIUM_VALUE_TLS_SUPPORT =       
TS_SELENIUM_VALUE_OPENSHIFT_OAUTH =   true
TS_OCP_LOGIN_PAGE_PROVIDER_TITLE =    htpasswd
E2E_OCP_CLUSTER_VERSION =             4.x
TS_SELENIUM_LOG_LEVEL =               DEBUG
TS_SELENIUM_OCP_USERNAME =            admin
TS_SELENIUM_W3C_CHROME_OPTION =       true
NODE_TLS_REJECT_UNAUTHORIZED =        0
TS_SELENIUM_EDITOR =                  che-code

> @eclipse-che/[email protected] tsc
> rm -rf ./dist && ./configs/sh-scripts/generateIndex.sh && tsc -p .

Generating index.ts file...

################## Launch Information ##################

      TS_SELENIUM_BASE_URL: https://devspaces.apps.ocp414-mmusiie.crw-qe.com
      TS_SELENIUM_HEADLESS: false
      TS_SELENIUM_OCP_USERNAME: admin
      TS_SELENIUM_EDITOR:   che-code

      TS_SELENIUM_HAPPY_PATH_WORKSPACE_NAME: EmptyWorkspace
      TS_SELENIUM_DELAY_BETWEEN_SCREENSHOTS: 1000
      TS_SELENIUM_REPORT_FOLDER: ./report
      TS_SELENIUM_EXECUTION_SCREENCAST: false
      DELETE_SCREENCAST_IF_TEST_PASS: true
      TS_SELENIUM_REMOTE_DRIVER_URL: 
      DELETE_WORKSPACE_ON_FAILED_TEST: false
      TS_SELENIUM_LOG_LEVEL: TRACE
      TS_SELENIUM_LAUNCH_FULLSCREEN: true

      TS_COMMON_DASHBOARD_WAIT_TIMEOUT: 5000
      TS_SELENIUM_START_WORKSPACE_TIMEOUT: 360000
      TS_WAIT_LOADER_PRESENCE_TIMEOUT: 60000

      TS_SAMPLE_LIST: Node.js MongoDB,Node.js Express,Java Lombok,Quarkus REST API,Python,.NET,C/C++,Go,PHP,Ansible

      MOCHA_DIRECTORY: dashboard-samples
      USERSTORY: RecommendedExtensions

      to output timeout variables, set TS_SELENIUM_PRINT_TIMEOUT_VARIABLES to true
 ######################################################## 


            ‣ DriverHelper.getDriver
  Check if recommended extensions installed for Node.js MongoDB 
          ▼ LoginTests.loginIntoChe
.......................................
    ✔ Check if extensions are installed and enabled
          ▼ Dashboard.openDashboard
            ‣ DriverHelper.navigateToUrl
            ‣ DriverHelper.getDriver
          ▼ Dashboard.waitPage
            ‣ DriverHelper.waitVisibility - By(xpath, //div[@id="page-sidebar"]//a[contains(text(), "Workspaces (")])
            ‣ DriverHelper.waitVisibility - polling timed out attempt #1, retrying with 1000ms timeout
.......................................
          ▼     at /home/mmusiien/che-projects/che/tests/e2e/specs/MochaHooks.ts:39:12 - delete workspace name

  Check if recommended extensions installed for Ansible 
          ▼ LoginTests.loginIntoChe
            ‣ BrowserTabsUtil.getCurrentUrl
            ‣ DriverHelper.getDriver
          ▼ Dashboard.waitPage
            ‣ DriverHelper.waitVisibility - By(xpath, //div[@id="page-sidebar"]//a[contains(text(), "Workspaces (")])
            ‣ DriverHelper.waitVisibility - element is located and is visible.
.......................................
            ‣ DriverHelper.getDriver
    ✔ Create and open new workspace, stack:Ansible
            ‣ DriverHelper.getDriver
            ‣ WorkspaceHandlingTests.obtainWorkspaceNameFromStartingPage - failed to obtain name from workspace start page, element not visible yet. Retrying.
  .......................................
    ✔ Obtain workspace name from workspace loader page
          ▼ registerRunningWorkspace - with workspaceName:ansible-demo
    ✔ Registering the running workspace
          ▼ ProjectAndFileTests.waitWorkspaceReadinessForCheCodeEditor - waiting for editor.
 .......................................
            ‣ DriverHelper.waitVisibility - element is located and is visible.
          ▼ ProjectAndFileTests.waitWorkspaceReadinessForCheCodeEditor - editor was opened in 49655 seconds.
    ✔ Wait workspace readiness (49655ms)
            ‣ DriverHelper.wait - (20000 milliseconds)
          ▼ ProjectAndFileTests.getProjectViewSession
            ‣ DriverHelper.waitVisibility - By(css selector, .monaco-list-row)
            ‣ DriverHelper.waitVisibility - element is located and is visible.
          ▼ ProjectAndFileTests.getProjectTreeItem - .vscode
            ‣ DriverHelper.waitVisibility - By(xpath, .//div[@role='treeitem' and @aria-level='2'])
            ‣ DriverHelper.waitVisibility - element is located and is visible.
    ✔ Check the project files were imported
          ▼ ProjectAndFileTests.performTrustAuthorDialog
            ‣ DriverHelper.waitAndClick - By(xpath, //div[@class="monaco-dialog-box"]//a[@class="monaco-button monaco-text-button"])
            ‣ DriverHelper.waitVisibility - By(xpath, //div[@class="monaco-dialog-box"]//a[@class="monaco-button monaco-text-button"])
            ‣ DriverHelper.waitVisibility - element is located and is visible.
    ✔ Accept the project as a trusted one
          ▼ ProjectAndFileTests.getProjectTreeItem - .vscode
            ‣ DriverHelper.waitVisibility - By(xpath, .//div[@role='treeitem' and @aria-level='2'])
            ‣ DriverHelper.waitVisibility - element is located and is visible.
          ▼ ProjectAndFileTests.getProjectTreeItem - extensions.json
            ‣ DriverHelper.waitVisibility - By(xpath, .//div[@role='treeitem' and @aria-level='3'])
            ‣ DriverHelper.waitVisibility - element is located and is visible.
          ▼ EditorView().openEditor(extensions.json)
            ‣ DriverHelper.waitVisibility - By(css selector, .inputarea)
            ‣ DriverHelper.waitVisibility - element is located and is visible.
          ▼ editor.getText(): get recommended extensions as text from editor, delete comments and parse to object.
          ▼ recommendedExtensions.recommendations: Get recommendations clear names using map().
          ▼ Recommended extension for this workspace:
          [{"publisher":"ms-python","name":"python"},{"publisher":"redhat","name":"ansible"},{"publisher":"redhat","name":"vscode-openshift-connector"},{"publisher":"ms-python","name":"black-formatter"}].
    ✔ Get recommended extensions list from extensions.json
          ▼ ActivityBar().getViewControl("Extensions"))?.openView(): open Extensions view.
    ✔ Open "Extensions" view section
          ▼ Time for extensions installation TimeoutConstants.TS_COMMON_PLUGIN_TEST_TIMEOUT=30000
            ‣ DriverHelper.wait - (30000 milliseconds)
    ✔ Let extensions complete installation (30033ms)
          ▼ ActivityBar().getViewControl("Extensions"))?.openView(): open Extensions view.
.......................................
            ‣ extensionMenuItems -> item.getLabel(): get menu items names.
          ▼ extensionMenuItemLabels: Disable Disable (Workspace) Install Another Version... Uninstall Copy Copy Extension ID Extension Settings Apply Extension to all Profiles Ignore Recommendation Remove from Workspace Recommendations .
    ✔ Check if extensions are installed and enabled
          ▼ Dashboard.openDashboard
 .......................................

            ‣ DriverHelper.wait - (5000 milliseconds)
            ‣ DriverHelper.quit
            ‣ DriverHelper.getDriver

  100 passing (19m)


Process finished with exit code 0

What issues does this PR fix or reference?

https://issues.redhat.com/projects/CRW/issues/CRW-6339

How to test this PR?

PR Checklist

As the author of this Pull Request I made sure that:

Reviewers

Reviewers, please comment how you tested the PR when approving it.

dmytro-ndp · 2024-05-15T11:33:06Z

tests/e2e/specs/dashboard-samples/RecommendedExtensions.spec.ts

@@ -81,6 +81,9 @@ for (const sample of samples) {
 });

 test('Check the project files were imported', async function (): Promise<void> {
+ // add 20 sec timeout for waiting for finishing animation of all IDE parts (Welcome parts. bottom widgets. etc.)


I would not mention particulate number 20 sec of timeout in here, because TIMEOUT_CONSTANTS.TS_IDE_LOAD_TIMEOUT could be redefined using environment variable when run the test.

dmytro-ndp · 2024-05-15T11:37:14Z

tests/e2e/specs/dashboard-samples/RecommendedExtensions.spec.ts

@@ -91,27 +94,39 @@ for (const sample of samples) {
 });

 test(`Get recommended extensions list from ${extensionsListFileName}`, async function (): Promise<void> {
- await (await projectAndFileTests.getProjectTreeItem(projectSection, pathToExtensionsListFileName))?.select();
+ // sometimes the Trust Dialog does not appear as expected - as result we can get opened Trust Box dialog. In this case.
+ // we need to perform this case


did you mean we need to execute "projectAndFileTests.performManageWorkspaceTrustBox()" method ?

dmytro-ndp · 2024-05-15T11:37:25Z

tests/e2e/specs/dashboard-samples/RecommendedExtensions.spec.ts

 });

 test('Open "Extensions" view section', async function (): Promise<void> {
 Logger.debug('ActivityBar().getViewControl("Extensions"))?.openView(): open Extensions view.');
- extensionsView = await (await new ActivityBar().getViewControl('Extensions'))?.openView();
+ // sometimes the Trust Dialog does not appear as expected - as result we can get opened Trust Box dialog. In this case.
+ // we need to perform this case


did you mean we need to execute "projectAndFileTests.performManageWorkspaceTrustBox()" method ?

dmytro-ndp · 2024-05-15T14:08:33Z

@musienko-maxim : here is the run of RecommendedExtensions test against DS 3.14.0.EC: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/complex/job/next/job/next-ds-functional-tests/678/console

Tests are still failing tests, now with a different error message and screenshot, e.g.:
https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/typescript-tests/22244/console

15:46:07    1) Check if recommended extensions installed for Java Lombok 
15:46:07         Check if extensions are installed and enabled:
15:46:07  
15:46:07        Extension vscode-java-debug not installed
15:46:07        + expected - actual
15:46:07  
15:46:07        -false
15:46:07        +true
15:46:07        
15:46:07        at Context.<anonymous> (/tmp/e2e/specs/dashboard-samples/RecommendedExtensions.spec.ts:211:75)
15:46:07        at processTicksAndRejections (node:internal/process/task_queues:95:5)

Previous error message before the fix:
https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/typescript-tests/22229/console

07:53:04    1) Check if recommended extensions installed for Java Lombok 
07:53:04         Check if extensions are installed and enabled:
07:53:04       AssertionError: Extension java not found: expected undefined not to be undefined
07:53:04        at Context.<anonymous> (/tmp/e2e/specs/dashboard-samples/RecommendedExtensions.spec.ts:186:88)
07:53:04        at processTicksAndRejections (node:internal/process/task_queues:95:5)

Could you please take a look?

dmytro-ndp · 2024-05-21T21:49:34Z

@musienko-maxim : here is " 'Java Lombok' RecommendedExtensions" test run with new changes from PR: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/typescript-tests/22500/console:

I found Extension was found: Debugger for Java four times in the test output, while should be four different extension names if I understood the test scenario correctly.
Could you please check it?

dmytro-ndp · 2024-05-30T11:32:48Z

@musienko-maxim : thank you for fixing the test!

I checked test run and noticed, that the fixed test is testing on extension author name, and it seems to be treating several extensions of one author as the same extension, e.g. https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/typescript-tests/22997/console

14:11:02            ▼ getVisibleFilteredItemsAndCompareWithRecommended - marketplaceSection.getVisibleItems()
14:11:02            ▼ All recommended extensions were found by  @recommended filter: ---- redhat,redhat,vscjava,vscjava ----
14:11:02            ▼ extensionSection.findItem by @installed filter
14:11:06            ▼ getVisibleFilteredItemsAndCompareWithInstalled - marketplaceSection.getVisibleItems()
14:11:06            ▼ All recommended extensions were found by  @installed filter: ---- redhat,redhat,vscjava,vscjava ----

Source code:

che/tests/e2e/specs/dashboard-samples/RecommendedExtensions.spec.ts

Line 81 in 4f79e3a

return allFoundAuthorsAsSortedString.includes(allPublisherNamesAsSortString);

At the same time test has expected to test ALL recommended extensions to be installed one by one doesn't matter if there is the same author, or not.

musienko-maxim · 2024-05-30T13:55:50Z

Not really, the test compares two lists:

The first one from the recommended extension JSON file
The second one from the recommended extension widget
But as you mentioned we can have several names with the same publisher. To simplify comparison (because it can be in a different order in the widget and in the JSON recommended extension file), I sorted this by alphabetical order and converted it to the string.

dmytro-ndp · 2024-05-30T15:54:43Z

Okay, let's test on extension publisher name assuming that there is a very little chance to have installed wrong extension of the same publisher.

Quarkus test passed: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/typescript-tests/22992/
Lombok test passed: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/Testing/job/e2e/job/basic/job/typescript-tests/22997/

musienko-maxim added 2 commits May 15, 2024 13:47

Fix the RecommendedExtensions test

17f12bf

revert webdriver version

cc7ccd7

musienko-maxim requested a review from dmytro-ndp as a code owner May 15, 2024 10:56

musienko-maxim requested review from artaleks9 and SkorikSergey May 15, 2024 11:12

dmytro-ndp reviewed May 15, 2024

View reviewed changes

Add recommended suggestions and stabilizations steps

e5f92f9

rework logic for getiing anf compare extensions

4f79e3a

dmytro-ndp approved these changes May 30, 2024

View reviewed changes

musienko-maxim merged commit ccb99a6 into main May 30, 2024
5 checks passed

musienko-maxim deleted the CRW-5357 branch May 30, 2024 16:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix the RecommendedExtensions test #22966

Fix the RecommendedExtensions test #22966

musienko-maxim commented May 15, 2024 •

edited

dmytro-ndp May 15, 2024 •

edited

dmytro-ndp May 15, 2024

dmytro-ndp May 15, 2024

dmytro-ndp commented May 15, 2024

dmytro-ndp commented May 21, 2024

dmytro-ndp commented May 30, 2024 •

edited

musienko-maxim commented May 30, 2024 •

edited

dmytro-ndp commented May 30, 2024 •

edited

Fix the RecommendedExtensions test #22966

Fix the RecommendedExtensions test #22966

Conversation

musienko-maxim commented May 15, 2024 • edited

What does this PR do?

Screenshot/screencast of this PR

What issues does this PR fix or reference?

How to test this PR?

PR Checklist

Reviewers

dmytro-ndp May 15, 2024 • edited

Choose a reason for hiding this comment

dmytro-ndp May 15, 2024

Choose a reason for hiding this comment

dmytro-ndp May 15, 2024

Choose a reason for hiding this comment

dmytro-ndp commented May 15, 2024

dmytro-ndp commented May 21, 2024

dmytro-ndp commented May 30, 2024 • edited

musienko-maxim commented May 30, 2024 • edited

dmytro-ndp commented May 30, 2024 • edited

musienko-maxim commented May 15, 2024 •

edited

dmytro-ndp May 15, 2024 •

edited

dmytro-ndp commented May 30, 2024 •

edited

musienko-maxim commented May 30, 2024 •

edited

dmytro-ndp commented May 30, 2024 •

edited