uva-se-evo-16

Repo for Matt Chapman & Dennis van der Werf, for Software Evolution 2016 @ UvA-SE

Series 1

Series 1 Usage

To execute the series1 analysis, first import the smallsql and hsqldb projects into your Eclipse workspace.

Run runTests(x) to execute the metric analysis, where x is either project2 for hsqldb or project3 for smallsql. A .csv file containing Unit Test results generated by EclEmma should also be present, in order to calculate unit test coverage.

The tests will execute and output metrics, followed by SIG scores for each aspect of maintainability.

Caveats

The code has the following considerations that need to be taken into account while reading the results.

• Counts of lines for volume and code complexity calculations may not be exactly correct, due to the use of regular expressions to handle catching comment lines. There are likely edge cases that the regular expressions do not catch, that may result in divergence between our output and that of the reference implementation.
• Unit size metrics are calculated, but a risk profile is not provided. This was due to us being unable to locate documentation to back up the selection of risk profile boundaries, and the various materials we did check did not agree with each other.
• Automated testing is not present, but we did run tests for correctness by creating a HelloWorld project that had known properties in terms of line count (with and without comments), unit size and cyclomatic complexity. This means we could have implemented unit tests to ensure the correctness of our code, if we did not have time constraints.
• Unit test coverage is calculated by running the unit tests in the project with EclEmma - and exporting the resulting test session. This export is then parsed using Rascal-MPL, and the resulting figures used to provide a rank for test quality.
• Running metrics against hsqldb will result in a crash unless the following is carried out in the hsqldb project:
  • In ValidatingResourceBundle.java line 51 change Class<? extends Enum<?>> to Class.
  • In ValidatingResourceBundle.java line 167 change the type of e to Object.