ETL Pipeline Pharmacy sales Tracker

Overview

This is a Pharmacy Sales tracker which connects Streamlit and Google Sheets. The Streamlit application provides a user interphace for the employees to input the product quantity sales of a pharmaceutical company across several distributed towns.

The User Interface looks like:

After employee input about sales, the same sheet is extracted and loaded into an OLAP database precisely DuckDB where DBT is used to perform transformation of the sales data which can be found here.

DUCKDB is efficient for handling large volumes of data and scales well with any type of data.

With sufficient data, an increemental model using DBT will be implemented to only ensure new incoming data is transformed so as not to slow the database operations.

DBT is a good tool for transformartion as it allows building efficient scalable ETL data pipelines. With DBT, different transformation SQL models can be chained together through model referencing.

Using the starter project

Try running the following commands:

• dbt run
• dbt test

Resources:

• Learn more about dbt in the docs
• Check out Discourse for commonly asked questions and answers
• Join the chat on Slack for live discussions and support
• Find dbt events near you
• Check out the blog for the latest news on dbt's development and best practices