A runtime environment to execute statistical services. IS2 is a workbench that offers a set of tools for data analysis and processing.
Among the tools for data processing and integration, the workbench allows to perform the probabilistic record linkage applying the Fellegi-Sunter method (RELAIS statistical service)
In order to build the IS2 application, your environment should fulfill the following requirements:
- A favorite text editor or IDE;
- JDK 11 or later;
- Maven 3.0+;
- Mysql Server 8.0 or later;
- PostgreSQL 9.6 or later
Istat has realized a generalized environment (Istat Statistical Service - IS2) that allows to select statistical services from a catalogue and execute them through a web application (IS2 workbench). The IS2 Workbench has been designed to offer a set of functionalities that allow to:
- Select a business function: the landing page contains the list of available business functions classified according to GSBPM phases (e.g. ReLais performs GSBPM 5.1 “Data integration”). A business function is a high level goal (What) that can be realized by one or more statistical processes, implemented by one or more services available in the catalogue.
- Select a business process: the system provides the list of available processes for the selected function (e.g. Probabilistic Record Linkage or Deterministic Record Linkage). A business process is implemented by a set of process steps. Each process step is linked to a statistical service available in the catalogue. Statistical services perform specific statistical method, implemented in an open source language.
- Upload process input data: in order to launch a process, the system requires the specification of input data to process. The initial set of data may include a list of rules, and/or other parameters used by the statistical method embedded in the process steps.
- Set process metadata: a statistical service may require further information, depending on the statistical function to perform. This set of metadata is provided by the user and is usually tied to input data structure, or concerns model parameters (e.g. specification of matching variables in the datasets to be linked, setting of matching/unmatching thresholds).
- Execute a business process according to a predetermined workflow: this function allows to execute the process previously configured.
Download and unzip the source code in your workspace IS2_PATH.
Before building the application you must create a MySQL database. From the command line go to MySQL installation directory MYSQL_PATH:
cd MYSQL_PATH\bin;
mysql -u db_username -p
Then create the tables needed to run the application, using the script is2.sql stored in the IS2_PATH/db folder:
mysql> source is2-create.sql
mysql> source is2-insert.sql
The script will populate the USER/ROLES tables with the user:
Username: [email protected]
Password: istat
After DB installation, you need to increase the max_allowed_packet parameter in the my.ini configuration file and restart the MySQL Sever:
max_allowed_packet=256M
From your IDE select and open the unzipped maven project.
As a first step check the content of the application.properties file, located in the path Other Sources > src/main/resources:
spring.datasource.url = jdbc:mysql://localhost:3306/IS2?useSSL=false&useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
spring.datasource.username = db_username
spring.datasource.password = db_password
Now you can perform your first build of the application. If the build process ends successfully, you are ready to run the application. The application is built using the open source framework Spring Boot, which generates an executable jar (that can be run from the command line). Spring Boot creates a stand-alone Spring based Applications, with an embedded Tomcat, that you can "just run".
java –jar is2.jar
Dockerize the PostgreSQL database
docker build -t mecdcme/is2-postgres . -f db.Dockerfile
docker run -p 5432:5432 mecdcme/is2-postgres
Dockerize the web application
docker build -t mecdcme/is2 .
docker run -p 8080:8080 mecdcme/is2
Docker compose
docker-compose up
The application will be at http://localhost:8080/is2 If you want to inspect the database you can use the Adminer application at http://localhost:8081/
IS2 is EUPL-licensed