You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our codebase has different groups (folders) of tests, viz.
./tests/CRUD
./tests/UI
./tests/Happy-Path
./tests/E2E
Now these groups (folders) contain a lot of suites (files) that contain one or more tests.
Note that each test generate a lot of logs since those tests perform a lot of assertions, 2000+ in almost all test cases.
We used to run these tests using different npm scripts for different groups (folders). But while upgrading our codebase to use Nightwatch v2.22, we have run into a problem where the test execution is running out of memory.
I have created a mini-project where you can get an idea of what lies within one of the folders mentioned above. Imagine the tests folder in the repo below is same as one of the folders mentioned above. https://github.com/kumarchandresh/nightwatch-issue-4022
How to run
Start express server on port 3000: npm start
Run nightwatch tests: npm test
Explanation
The web page and the test script is created with the sole purpose of generating high amounts of logging which happens during assertions, and is similar to our codebase.
This is what happens usually,
Don't focus on 4 GB size (I know about the --max-old-space-size and it also fails for 12 GB in our project). The point is that, a lot of assertions were ran, which in turn produce lots of logging statements, and the end result is Node.js ran out of heap memory.
I also captured snapshots of memory usage at every 5 seconds interval, and here is the graph of memory usage during test run
const{rss, heapUsed}=process.memoryUsage()
Step to produce your own graph (will require python)
python -m venv venv
venv/Scripts/activate
pip install -r requirements.txt
Run the tests
Find the latest pid_xxxx.csv file under memory-usage dir
jupyter notebook
Open analysis.ipynb and replace the csv file name and run all cells.
What I think
I think it is the new reporters in Nightwatch v2 like JSON reporter and HTML reporter due to which a lot of memory is being used. Nightwatch is holding out on all the information of the test run to successfully write any report it wants, but at the end of execution of all tests.
Suggested solution
We only use JUnit reporter during our test run. JUnit reports doesn't comsume too much memory. I would suggest,
Either Nightwatch should be aware of the reporter it is going to use at the end of test execution and keep only that much information in memory that is required to successfully write such reports.
Or, Nightwatch should write the report for each suite (file) as soon as it completes execution and release the occupied memory for GC.
Alternatives / Workarounds
No response
Additional Information
No response
The text was updated successfully, but these errors were encountered:
Description
Our codebase has different groups (folders) of tests, viz.
./tests/CRUD
./tests/UI
./tests/Happy-Path
./tests/E2E
Now these groups (folders) contain a lot of suites (files) that contain one or more tests.
Note that each test generate a lot of logs since those tests perform a lot of assertions, 2000+ in almost all test cases.
We used to run these tests using different npm scripts for different groups (folders). But while upgrading our codebase to use Nightwatch v2.22, we have run into a problem where the test execution is running out of memory.
I have created a mini-project where you can get an idea of what lies within one of the folders mentioned above. Imagine the
tests
folder in the repo below is same as one of the folders mentioned above.https://github.com/kumarchandresh/nightwatch-issue-4022
How to run
npm start
npm test
Explanation
The web page and the test script is created with the sole purpose of generating high amounts of logging which happens during assertions, and is similar to our codebase.
This is what happens usually,
Don't focus on 4 GB size (I know about the
--max-old-space-size
and it also fails for 12 GB in our project). The point is that, a lot of assertions were ran, which in turn produce lots of logging statements, and the end result is Node.js ran out of heap memory.I also captured snapshots of memory usage at every 5 seconds interval, and here is the graph of memory usage during test run
Step to produce your own graph (will require python)
python -m venv venv
venv/Scripts/activate
pip install -r requirements.txt
pid_xxxx.csv
file undermemory-usage
dirjupyter notebook
analysis.ipynb
and replace the csv file name and run all cells.What I think
I think it is the new reporters in Nightwatch v2 like JSON reporter and HTML reporter due to which a lot of memory is being used. Nightwatch is holding out on all the information of the test run to successfully write any report it wants, but at the end of execution of all tests.
Suggested solution
We only use JUnit reporter during our test run. JUnit reports doesn't comsume too much memory. I would suggest,
Alternatives / Workarounds
No response
Additional Information
No response
The text was updated successfully, but these errors were encountered: