End to end testing for aria-at-app

[ccanash]

Proof of concept phase

• Team discussion on potential approach
• Investigate Selenium vs. Puppeteer (and Web Driver vs. DevTools Protocol.) (We decided on Puppeteer.)
• Create proof of concept for transactions approach (Proof of concept for using database session transactions to reset the database between tests #895)
• Approve proof of concept as a team

Smoke Test

• Implement smoke test which verifies that three pages have loaded as expected (Implement end-to-end smoke test #904)
• Implement concurrent sign in as admin / tester / vendor functionality (Allow End-to-End Testing as an Admin or Tester #908)
• Replace existing fake role system with a more capable fake user system
• Document new fake user system (https://github.com/w3c/aria-at-app/blob/main/docs/local-development.md#signing-in-as-a-tester-admin-or-vendor)
• Make sure e2e tests work in CI

Transactions Phase

In addition to preventing corruption to the database in the case of an error partway through a set of database operations, transactions allow tests to quickly reset the database to a known state.

• Create new services folder and move the existing services to a services.deprecated folder.
• Update 108/108 imports from services to services.deprecated
• Point 12/12 test suites to new services folder.
• Add 21/21 missing tests.
• Overhaul nonstandard model services - TestPlanService and CollectionJobService.
• Add transactions to model APIs.
• Implement long-delayed redesign to model APIs from positional arguments to object-based arguments.
• Part 1 PR. (Support transactions in models #922)
• Implement Express middleware through req.transaction.
• Make transactions available in GraphQL through context
• Add transaction support to 3/3 mock api servers
• Make context a required parameter across 79/79 GraphQL resolvers and 129/129 places they are used.
• Remove all references to services.deprecated pointing at new services folder.
• Part 2 PR. (Model transactions part 2 #940)
• Update 3/3 open PRs still using services.deprecated.

Finalize End-To-End Testing

• Select feature for initial full end-to-end test (We chose AT Version creation, update and delete).
• Remove services.deprecated folder.
• Create convenient toolkit for end-to-end testing (we created a getPage function to encompass repeated end-to-end tasks like creating a page).
• Implement solid debugger support to make writing end-to-end tests easier. (https://github.com/w3c/aria-at-app/blob/main/docs/local-development.md#debugging)
• Define a workflow for implementing future end-to-end tests (using the debugger to view the page in intermediate steps ended up being the approach.)
• Implement first end-to-end test which includes database mutations.
• Discuss new testing requirements as a team.
• Document new testing requirements. (https://github.com/w3c/aria-at-app/blob/main/CONTRIBUTING.md#guidance-on-testing)
• PR. (Adding and Removing AT Versions End-to-End Test #951)

Additional Tasks

• Expand number of simultaneous connections to accommodate increased connections with transactions. (Fix intermittent connection timeout #990)