AWS CloudTrail Hands-on Lab

It decribes how to use aws athena for data analytics

Este projeto tem como objetivo realizar a coleta, catalogo, governança, processamento e visualização de dados.

This projects uses ETL (Extract, Transform and Load) pipeline to extract data from Spotify using its API and loads the data to a data source(AWS Athena). The entire pipeline will be built using Amazon Web Services (AWS).

User, Event, and Predictive Metric Dashboard on 2GB/month of log files from Brackets IDE

Unveiling job market trends with Scrapy and AWS

This project aims to securely manage, streamline, and perform analysis on the structured and semi-structured YouTube videos data based on the video categories and the trending metrics.

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK and MSK Connect (Debezium)

The project is to simulate Real-time streaming for movie details using Kafka. We used different technologies such as Python, Amazon EC2, Apache Kafka, Glue, Athena, and SQL.

Data streaming project with Apache Druid & Grafana: Real-time data processing, alerts, integration with AWS. It uses a combination of technologies and services, including Confluent-Kafka, Apache Druid, AWS SNS, EC2, Athena, S3, Glue and EventBridge, StepFunctions. Contribute to this powerful solution!

An automated SQL script generator to migrate AWS Redshift schemas (or tables) to AWS Redshift Spectrum

Athena JDBC Authentication provider for Azure AD

The application is the documentation of my solution for the iFood data architect test.

AWS Athena query counter

This project is based for legacy applications that works with positional files to process data. The objetive is read these positional files when they arrives in AWS S3, and then send to a dataware-house like AWS Redshift, and finally read the results with a Business Intelligence tool as AWS QuickSight.

Get the dataset intro a S3 bucket, use AWS glue to transform the dataset, write a Lambda script to clean the dataset, query the dataset via AWS Athena then build a dashboard using AWS Quicksight.

Analyzed a multicategory e-commerce store using big data techniques on a Kaggle dataset with the help of AWS EC2, AWS S3, PySpark, AWS Glue ETL, AWS Athena, AWS CloudFormation, AWS Lambda and Power BI!

A small walkthrough how to create an AWS Glue Job Pipeline with AWS CDK

Demonstration of how to run interactive Athena queries using the ODBC driver in a Jupyter Notebook running in Docker

This project repo 📺 offers a robust solution meticulously crafted to efficiently manage, process, and analyze YouTube video data leveraging the power of AWS services. Whether you're diving into structured statistics or exploring the nuances of trending key metrics, this pipeline is engineered to handle it all with finesse.