End2End Dog Breed Classification Chatbot

End-to-end dog breed image classification with machine learning
Report Bug · Request Feature

Note:

• From data collection/scraping to deployment
• Bot was previously deployed on an EC2 instance (Ubuntu) as a service
  • When I still had AWS Educate credits

Table of Contents:

• Data Scraping
• Data Collection
• Custom Image Processing
• Initial Modeling
• Transfer Learning
• Bot Development
• Deployment
• Usage
• Built With
• Contact
• License
• Acknowledgements

Data Scraping

The class labels (dog breeds) are obtained from a website listing 209 dog breeds and are extracted using a simple function utilising BeautifulSoup4 library.

The function is named scrape_for_scraps and its Python corresponding script can be found here.

Data Collection

The image dataset (dog breed images) is gathered with the use of Google Image Search API. Image links are retrieved from Google Image Search API (limit of 100 images per search) and the images are downloaded using Python's Requests library.

The Google Image Search API function can be found here.

The image downloader function can be found here.

Custom Image Processing

A image processor was developed to handle basic image manipulation:

• Loading images
• Image pixel normalisation
• Image resizing
• Image padding

Initial Modeling

A simple convolution neural network architecture is developed and a model was trained with the image dataset.

• However, too computationally heavy to train the model till a respectable performance is achieved
• Attained ~5% classification accuracy on test set after training with a laptop's CPU (X1 Carbon 4th Gen)

Transfer Learning

A pre-trained image classification model trained on the ImageNet dataset was used and the chosen architecture was Xception as it has the same number of parameters as Inception V3 but has better classification performance. Only the last layer of weights are retrained to refit the model to be capable of performing dog breed classification.

The code for transfer learning can be found here.

Bot Development

The Telegram bot functionality is created by utilising Python's requests library to interact with Telegram's Bot API.

Some of the bot's functionality includes:

• Image message recognition (uploaded via Gallery or via @pic)
• Responding to directed messages
• Downloading the image in the message
• Managing the dog breed classification model and using it for image prediction

The Telegram bot implementation can be found here.

Deployment

Dog breed image classification bot was deployed on a AWS EC2 t3.small instance:

• Initially set to run indefinitely with a cron job restarting the bot every hour
• Switched to running the bot as a service with systemd, enabling the bot to be restarted automatically only if it terminates

Usage

The gif below shows a sample usage of the bot when it was active.

Built With

• Google Search API - Data collection
• BeautifulSoup4 - Web scraping for class labels
• Requests - Image downloads and bot requests
• OpenCV - Image processing and manipulation
• Keras - Modeling, transfer learning, etc.

Contact

Lim Jia Xiang - [email protected]

Project Link: https://github.com/limjiaxiang/doggo-cv-chatbot

License

This project is licensed under the Apache License Version 2.0 - see the LICENSE.txt file for details

Acknowledgements

• othneildrew/Best-README-Template