ETL Pipeline for processing scraped web data and preparation for loading
-
Updated
Sep 6, 2018 - Jupyter Notebook
#
etl-pipeline
Here are 1,378 public repositories matching this topic...
Data modeling with Postgres and ETL pipeline using Python.
-
Updated
Oct 21, 2019 - Python
Complete data platform that performs sentiment analysis on tweets. Built using Cassandra, Kafka, Spark, Node, and React.
-
Updated
Jan 5, 2023 - Java
A self-paced learning sandbox for rocking a data pipeline with Python.
-
Updated
Jul 30, 2019 - Python
generating an ETL pipeline using pyspark, from a supermarket_sales CSV file, finally storing in a PostgreSQL DB
-
Updated
Jan 8, 2020 - Python
Create a Apache Cassandra DB
-
Updated
Jun 3, 2020 - Jupyter Notebook
Data Tweak is a simplified, lightweight ETL framework based on Apache Spark.
-
Updated
Jan 26, 2021 - Scala
From data gathering to productionizing LLMs using LLMOps good practices.
-
Updated
Mar 18, 2024 - TypeScript
data warehouse & data modelling in AWS using s3 and Redshift
-
Updated
Dec 26, 2020 - Jupyter Notebook
Extract, Transform, and Load Data: Extract from wikipedia JSON, transform using python pandas dataframe, then load into PostgreSQL database
-
Updated
Jan 23, 2021 - Jupyter Notebook
This project is to classify the messages that will popup on time of natural disaster. It will be useful for countries to tackle that disaster. In this project, NLP and ML pipelines are used.
classifier
flask
machine-learning
pipelines
disaster
dataset
nlp-machine-learning
disaster-response
etl-pipeline
trains-classifier
-
Updated
May 5, 2020 - Python
DATA SCIENCE PORTFOLIO & BLOG
automation
big-data
sklearn
machine-learning-algorithms
portfolio-website
blog-engine
ci-cd
python3
etl-pipeline
-
Updated
Jul 23, 2023 - JavaScript
An ETL project: to load google stock data to Cassandra and do some regression.
docker
cassandra
docker-compose
pandas-dataframe
sklearn
stock-price-prediction
etl-pipeline
cassandra-driver
-
Updated
Sep 4, 2020 - Python
Udacity Data Engeneering Nanodegree Program - My submission of Project: Data Modeling with Postgres
-
Updated
Dec 29, 2020 - Python
Data Analysis for Amazon Product Reviews
-
Updated
Apr 6, 2021 - Jupyter Notebook
In this project, We will use Spark and data lakes to build an ETL pipeline for a data lake hosted on S3. To complete the project, you will need to load data from S3, process the data into analytics tables using Spark, and load them back into S3. We will deploy this Spark process on a cluster using AWS.
-
Updated
Jun 14, 2021 - Jupyter Notebook
ETL pipeline that extracts their data from S3, stages them in Redshift, and transforms data into a set of dimensional tables for their analytics team to continue finding insights in what songs their users are listening to. Then we will test the database and ETL pipeline by running queries given to us by the analytics team from Sparkify and compa…
-
Updated
Jul 18, 2020 - Python
-
Updated
May 21, 2021
Dockerized Data Pipeline that analyzes the sentiment of tweets and a Slack Bot that publishes selected tweets
docker
mongodb
sentiment-analysis
pipeline
etl
docker-compose
docker-container
docker-image
postgresql
data-engineering
slack-bot
etl-pipeline
-
Updated
Aug 30, 2021 - Jupyter Notebook
Improve this page
Add a description, image, and links to the etl-pipeline topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the etl-pipeline topic, visit your repo's landing page and select "manage topics."