FI QA Chat-bot

This source code of Fi Chat-bot that created as a course work and of course for the freshmen students and for the entrants on my faculty.

Also search feature is available as RESTful API web service.

Stack:

• aiogram python telegram bot async framework with uvloop, ujson, aiohttp[speedups] boosters;
• Flask python web micro-framework;
• Flask-Migrate - Flask extensions for database migrations managing and Flask-SQLAlchemy - Flask ORM;
• pdfminer.six - python .pdf document parser for parsing parts of knowledge base;
• gunicorn - production WSGI HTTP server;
• torch - python ML framework. I'm using it for building BERT -based ML model for question-answering system;
• elasticsearch - NoSQL database for indexing knowledge for question-answering system;
• ingest - Elasticsearch plugin for indexing file attachments. Was used for indexing knowledge base .pdf docs paragraphs;
• PostgreSQL - RDBMS for managing all bot data.
• Redis - Key-Value database for managing users' special answers in bot.

Environment variables (put them in a file in the root of project called .env):

# postgres env
POSTGRES_DB
POSTGRES_HOST
POSTGRES_PORT
POSTGRES_EXTERNAL_PORT
POSTGRES_USER
POSTGRES_PASS

# web app env
WEB_APP_PORT
WEB_APP_HOST
PDF_URL
QA_TXT_URL

# elastic env
ELASTIC_PORT
ELASTIC_EXTERNAL_PORT
ELASTIC_HOST

# bot env
BOT_API_TOKEN
API_HOST
SUPPORT_CHAT_ID

# common
PG_HOST
ES_HOST
ES_USER
ES_PASS

Requirements (docker-based start):

• Engine ^18.0
• Compose ^1.18

Requirements (local start):

• Python ^3.7.0
• gunicorn ^20.0
• PostgreSQL ^9.6
• Redis ^5.0
• Elasticsearch ^7.0.0 (with installed ingest plugin via bin/elasticsearch-plugin install --batch ingest-attachment command)

Start (docker-based start):

• To build and start all services:

docker-compose up --build (Web service will be available at $YOUR_HOST:$YOUR_PORT from .env file)

• To stop all services:

docker-compose stop

• To stop specific service:

docker-compose stop <SERVICE_NAME>

• To kill specific service:

docker-compose kill <SERVICE_NAME>

• To start specific service:

docker-compose start <SERVICE_NAME>

• To build specific service:

docker-compose build <SERVICE_NAME>

• To attach to a specific service:

docker-compose attach <SERVICE_NAME>

Start (local start):

1. Make sure PostgreSQL, Redis and Elasticsearch services are up.
2. Make sure you create a PostgreSQL Database (CREATE DATABASE <db_name>;).
3. Create a venv for python python -m venv venv and activate it source venv/bin/activate.
4. Install requirements from web_service and bot folders with pip install -r requirements.txt.
5. Migrate a database with flask db upgrade --directory web_service/migrations command.
6. (Recommended) Reseed database from knowledge base: flask force_reseed_db.
7. (Recommended) Check app for all needed data: flask check_app.
8. To start server: gunicorn -w <WORKERS_COUNT (Python processes)> -b <YOUR_HOST>:<YOUR_PORT> "main:app". You will see a message in console that your server is running on $YOUR_HOST:$YOUR_PORT.