image-processing-service

A service that resizes and uploads images to S3.

Project Overview

This an image processing service that resizes and uploads images to AWS S3.

Working

This service allows a user to register for an account using their email address, user name ad a profile picture. The picture is then optionally resized and uploaded to amazon s3. To use it:

1. Register for an account by submitting your details using the register route.
2. You can update your details using the update route.

Features

This application has several features including:

1. Deployed to an AWS EBS.
2. Versioned using git and Hosted on GitHub.
3. Auto-deployed to AWS EBS using Github actions.
4. Uses gunicorn as the application servers.
5. Uses an Application Load Balancer to redirect traffic to the frontend
6. Uses AWS S3 to store the image.

Local Setup

Here is how to set up the application locally:

1. Clone the application repo:
   

   git clone https://github.com/twyle/image-processing-service.git

2. Navigate into the cloned repo:

   cd image-processing-service

3. Create a Virtual environment:

   python3 -m venv venv

4. Activate the virtual environmnet:

   source venv/bin/activate

5. Install the project dependancies:

   pip install --upgrade pip # update the package manager
   pip install -r requirements.txt  

6. Create the environment variables for each service:

   touch .env

   Then paste the following into the file:

   FLASK_APP=manage.py
   FLASK_DEBUG=True
   FLASK_ENV=development
   
   S3_BUCKET=flask-image-service
   AWS_ACCESS_KEY=<AWS_ACCESS_KEY>
   AWS_ACCESS_SECRET=<AWS_ACCESS_SECRET>
   
   POSTGRES_HOST=db
   POSTGRES_USER=lyle
   POSTGRES_PASSWORD=lyle
   POSTGRES_DB=image-service
   POSTGRES_PORT=5432
   
   CELERY_RESULT_BACKEND=redis://redis:6379/0
   CELERY_BROKER_URL=redis://redis:6379/0
   

7. Start the services:

   docker-compose -f docker-compose-dev.yml up --build

8. Create the database:

   docker-compose -f docker-compose-dev.yml exec image-service python manage.py create_db

9. View the running application

   Head over to http://0.0.0.0:5000/apidocs

Development

1. Application Design

1. Services

   The application consists of one service that send account activation emails to users:

   1. Email Service

   

   This service is resposible for sending account activation emails. The routes include:

   Route	Method	Description
   'api/v1/auth/register'	POST	Register a new user
   'api/v1/user/	POST	Update a new user

   1. Just enter user details and select an image and it will be uploaded.

   This service uses AWS S3 to store the images.

2. Project Management

1. Coding standards
   

   The application had to adhere to the following coding standards:

   1. Variable names
   2. Function names
   3. Test driven development
   4. Individual modules need 60% coverage and an overall coverage of 60%.
   5. CI/CD pipeline has to pass before deployments.
   6. Commit messages format has to be adhered to.
   7. Only push code to github using development branches.
   8. Releases have to be tagged.
   9. Use pre-commit to run code quality checks
   10. Use comitizen to format commit messages

2. Application development process management
   

   The project uses GitHub Projects for management.

3. Development Workflow

The application uses atleast 5 branches:

1. Features branch used to develop new features.
2. Development branch used to hold the most upto date features that are yet to be deployed.
3. Staging branch holds the code that is currently being tested for production.
4. The release branch holds all the assets used when creating a release.
5. The production branch holds the code for the currently deployed application.

The development workflow follows the following steps:

1. A feature branch is created for the development of a new feature.
2. The code is then pushed to GitHub, triggering the feature-development-workflow.yml workflow. If all the tests pass, the feature is reviewde and merged into the development branch.
3. The code in the development branch is then deployed to the development environment. If the deployment is succesful, the development branch is merged into the staging branch.
4. This triggers the staging workflow. If all the tests are succesful, this branch is reviewed and deployed to a staging environment.
5. For creatinga release, the staging branch is merged into the release branch. This happens when a tag is pushed to GitHub.
6. Once a release is created, the release branch is merged into the production branch, which is deployed into production.

The workflows require a couple of secrets to work:

  ```sh
  FLASK_APP=manage.py
  FLASK_DEBUG=True
  FLASK_ENV=development

  S3_BUCKET=flask-image-service
  AWS_ACCESS_KEY=<AWS_ACCESS_KEY>
  AWS_ACCESS_SECRET=<AWS_ACCESS_SECRET>

  POSTGRES_HOST=db
  POSTGRES_USER=lyle
  POSTGRES_PASSWORD=lyle
  POSTGRES_DB=image-service
  POSTGRES_PORT=5432

  CELERY_RESULT_BACKEND=redis://redis:6379/0
  CELERY_BROKER_URL=redis://redis:6379/0

  ```

The workflows also require the followingenvironments to work:

1. Test
2. Staging
3. Development
4. Production

And within each environment, create a secret that indicates the environment type i.e

1. Test -> FLASK_ENV=test
2. Staging -> FLASK_ENV=stage
3. Development -> FLASK_ENV=development
4. Production -> FLASK_ENV=production

Contribution

1. Fork it https://github.com/twyle/email-service/fork
2. Create your feature branch (git checkout -b feature/fooBar)
3. Commit your changes (git commit -am 'Add some fooBar')
4. Push to the branch (git push origin feature/fooBar)
5. Create a new Pull Request

Developer

Lyle Okoth – @lylethedesigner on twitter

lyle okoth on medium

My email is [email protected]

Here is my GitHub Profile

You can also find me on LinkedIN

License

Distributed under the MIT license. See LICENSE for more information.