Demo Project for Online Stream #16,#17,#20, #21 - Microservice for Excel File Processing

Demo project for online stream #16 where a small microservice is created that processes Excel file uploaded through REST API, saves file to AWS S3, parses it to JSON and send that to another service.

On stream #17 the dockerization of the project is done together with additional properties and settings to prepare for running in production.

On stream #20 additional features are added: Sending of file to SFTP folder and then sending email. The integration with all external services is tested with running services in containers.

On stream #21 monitoring was added to service with Prometheus and Grafana setup to see application metrics

Access to Online Stream on YouTube

To get a link to online stream on YouTube please do the following:

• 💰 Make any donation to support my volunteering initiative to help Ukrainian Armed Forces by means described on my website
• 📧 Write me an email indicating donation amount and time
• 📺 I will reply with the link to the stream on YouTube.

Thank you in advance for your support! Слава Україні! 🇺🇦

Technologies

• Spring Boot
• Apache POI
• AWS Java SDK
• SSHJ for SFTP
• JavaMail
• Prometheus for monitoring
• TestContainers for testing
• GreenMail for testing mail server

SFTP notes

To connect to SFTP server the SSH private key is used. The generated key is located in src/test/resources.

To generate new key you can use ssh-keygen tool on your system. Note, that key would be generated in OPENSSH format.

To convert to PEM format use the following command:

ssh-keygen -p -m pem -f /path/to/file

Building and Running the Application

To build the app, use gradle:

./gradlew build

To run application, use gradle:

./gradlew bootRun

After that you can access application on http://localhost:8080

Docker

To build docker image, use gradle:

./gradlew bootBuildImage

Then you run the docker container with exposing app port 8899:

docker run -p 8899:8080 -t stream-16-demo-app:0.0.1-SNAPSHOT

After that you can access application on http://localhost:8899

Running docker-compose with external dependencies

Project includes the docker-compose.yml file where external services are defined together with application.

Monitoring with Prometheus and Grafana

Prometheus and Grafana are started with docker-compose. Prometheus is set up to scape metrics from application that is also running in docker. The configuration for prometheus is defined in docker/prometheus.yml file.

Grafana is set up to read metrics from Prometheus. Grafana is started with default configuration defined in docker/grafana.ini. Default username and password to login to Grafana is admin:admin. You can import dashboards that were created during online stream from json files:

• docker/grafana-dashboard-online-stream.json
• docker/grafana-dashboard-spring-boot-stats.json

Documentation

Reference Documentation

For further reference, please consider the following sections:

• Official Gradle documentation
• Spring Boot Gradle Plugin Reference Guide
• Create an OCI image
• Spring Web
• Spring Boot Actuator
• Testcontainers

Guides

The following guides illustrate how to use some features concretely:

• Building a RESTful Web Service
• Serving Web Content with Spring MVC
• Building REST services with Spring
• Building a RESTful Web Service with Spring Boot Actuator

Additional Links

These additional references should also help you:

• Gradle Build Scans – insights for your project's build