Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
O projeto Pokémon TCG Data Pipeline visa criar uma solução de pipeline de dados para coletar, transformar e analisar informações sobre as cartas de Pokémon TCG (Trading Card Game).
This project involved managing real-time streaming data from the New York Times Developer API to ensure immediate access to the latest insights and articles. Utilizing Apache Airflow within GCP Cloud Composer, seamless workflow pipelines were orchestrated to automate data retrieval, preprocessing, and incremental loading into Snowflake DW
Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
Implement the word embedding for exploring the correlation among words - Design a sequence model for generating text
A Helm chart to install Apache Airflow on Kubernetes
In this Project, I'll be building a real-time data streaming pipeline, covering each phase from data ingestion to processing and finally storage. We'll utilize a powerful stack of tools and technologies, including Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra—all neatly containerised using Docker.
Helm Charts for the Astronomer Platform, Apache Airflow as a Service on Kubernetes
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
Airflow Providers containing Deferrable Operators & Sensors from Astronomer
🍃 [Apache Airflow] DAGs 🍃
CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer
Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee
Elyra extends JupyterLab with an AI centric approach.
A plugin for Apache Airflow that allows you to edit DAGs in browser
Streaming data pepiline using apache airflow, kafka , amazon S3 bucket
Merge data from OpenStreetMap and Wikidata to map information about entities and points of interest (mirror of https://gitlab.com/openetymologymap/osm-wikidata-map-framework )
Distributed run of dbt models using Airflow
Add a description, image, and links to the apache-airflow topic page so that developers can more easily learn about it.
To associate your repository with the apache-airflow topic, visit your repo's landing page and select "manage topics."