This repository provides supplementary material to the ICSE 2023 paper "Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software Systems", including experimental data (configuration samples, subject system code) and measurements (performance, option- and workload-specific code coverage) as well as an interactive dashboard. The latter complements the presentation of the study results in the paper and allows for reproduction of our analyses and findings.
The material includes information about the following nine software systems:
| Name | Domain | Language | Repository | Code Used / Version | License |
|---|---|---|---|---|---|
jump3r |
Audio Encoder | Java | Link |
v1.0.4 |
LGPL 2.1 |
kanzi |
File Compressor | Java | Link |
v1.9 |
Apache License 2.0 |
dconvert |
Image Scaling | Java | Link |
v1.0.0.-alpha7 |
Apache License 2.0 |
h2 |
Database | Java | Link |
v1.4.200 |
Mozilla Public License version 2.0 |
batik |
SVG Rasterizer | Java | Link |
v.1.14 |
Apache License 2.0 |
xz |
File Compessor | C/C++ | Link |
v5.2.0 |
GPL 2.0 and GPL 3.0 |
lrzip |
File Compressor | C/C++ | Link |
v0.651 |
GPL 2.0 |
x264 |
Video Encoder | C/C++ | Link |
baee400.. |
GPL 2.0 |
z3 |
SMT Solver | C/C++ | Link |
v4.8.14 |
MIT License |
This archive includes the configurations sampled and used for conducting the experiments as CSV files:
data/coverage_raw.tar.gz
The files used as workloads/inputs in the experiments were collected from various sources. We provide a list of the provenance of the files at:
data/workload_sources.csv
In compliance with the license (LICENSE.txt). This archive does not include the complete set of files used because not all files used allow for redistribution under the CC BY-SA 4.0 license.
This archive includes the performance measurements (throughput and execution time) per configuration and workload as CSV files.
The performance measurements for each <software system> can be found in the following subfolder:
dashboard/resources/<software_system>/measurements.csv
This archive includes (as a separate file!) raw coverge measurements per configuration and workload as CSV files. Based on these files, the option- and workload-specific code is inferred. The raw code coverage measuremnts can be found as a gz-ipped tar archive:
./coverage_raw.tar.gz
This archive includes the calculated code sections that are a) option-specific and b) workload- and option-specific.
The code sections for each <software system> can be found in the following subfolder:
dashboard/resources/<software_system>/code/
We provide an interactive dashboard using the frameword streamlit that allows exploring in
detail our analysis for each configuration option and workload. The original paper presents a representative subset of
the results, all analyses and visualizations can be reproduced using this dashboard. We provide a Docker-ized
version to run the dashboard locally.
To build and run the Docker container execute the following commands:
#!/bin/sh
cd ./dashboard
docker build -t streamlitapp:latest .
docker run -p 8501:8501 streamlitapp:latest
You can now explore the dashboard running locally at https://127.0.0.1:8501 or https://localhost:8501.
To use the dockerized application, the container environment Docker has to be set up and running. To install Docker, these tutorials provide orientation: