Utilizing my background and love for Apache Airflow and Data to build a real-time data streaming pipeline
-
Updated
Jun 21, 2024 - Python
Utilizing my background and love for Apache Airflow and Data to build a real-time data streaming pipeline
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
M.E.R.I.T : The Microservices NextGen 🕊️
Fully Managed Apache Zookeeper Cluster with Ansible & Terraform.
A look at the Apache ZooKeeper coordination service for distributed services including distributed messaging brokers with kafka and a distributed search engine using TF-IDF Algorithm
Apache Drill and Apache Zookeeper helm charts for kubernetes
Helm Charts to Deploy Apache Drill on Kubernetes
The project demonstrates an end-to-end data pipeline using Apache Kafka to fetch and stream data from a website. The project is containerized with Docker for streamlined deployment and dependency management.
This project is the core of Exchange developed during Full Cycle immersion for learning purposes. This Exchange uses Golang for multithreading processing of trades and Apache Kafka as Broker.
End-to-end data engineering pipeline with various technologies to ingest real time data.
End-to-end data engineering pipeline with various technologies to ingest real time data.
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
Real time streaming of a time series with corresponding forecasts.
A distributed and highly available URL Shortener built on MERN stack, using Apache Zookeeper, cached using Redis, containerized by Docker.
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
This project aims to consume a twitter stream via Apache Kafka, apply a sentiment analysis on the tweets with Apache Spark jobs and save the result into Apache HBase.
Distributed consensus algorithms implemented using Apache Kafka as a message broker.
Implementation of ZNode election and re-election using Apache ZooKeeper
Microservices with Apache Kafka, Kafka Streams
Distributed Java Applications at Scale, Parallel Programming, Distributed Computing
Add a description, image, and links to the apache-zookeeper topic page so that developers can more easily learn about it.
To associate your repository with the apache-zookeeper topic, visit your repo's landing page and select "manage topics."