Python | ETL | Google APIs

Amazing Prime loves the dataset and wants to keep it updated on a daily basis. We create one function that takes in the three files Wikipedia data, Kaggle metadata, the MovieLens rating data and creates an automated pipeline that takes in new data, performs the appropriate transformations, and loads the data into existing tables.

Amazon Reviews Metrics

Antenna Distribution is a project that shows how to run business analysis tools on a set of a data.

Opiniated Framework to write ETL Pipelines controlled by a central config store.

Repository for playing with spark

This is a PHP project which combines ETL with different strategies to extract data from multiple databases, files, and services, transform it and load it into multiple destinations.

dataflow pipeline mlflow spider

A simple data processing framework for a quick, no-frills setup of a local data pipeline.

Introduction to the data pipeline management with Airflow. Airflow schedule and maintain numerous ETL processes running on a large scale Enterprise Data Warehouse

etl日志收集分析工具

Data Tweak is a simplified, lightweight ETL framework based on Apache Spark.

From Cyber security perspective anomalous data points indicates suspicious Activity. So, to withstand the Attack using a self made ML model to detect and take action

Python package that enables customized loading of data from a CSV file into a MySQL database

Bamboo Connect is a lightweight ETL (Extract, Transform, Load) library with examples and templates. It enables developers to quickly extract, transform, reconcile and then load resulting data securely. This avoids time consuming manual error prone tasks.

The repository contains Structured Query Language (SQL) Scripts. The Multiple SQL scripts for various projects which includes data cleaning, data pre-processing, data processing, data transformation and insights gaining through Query Language.

Apache Spark based 'Dist' utility to supplement Data Cooker ETL tool

Collection of pkgs to build pipelines in JS/TS

utility to enable flexible ETL scenarios, supports golang plug-in for built-in consumer|transformer|producer options