This is a event driven meta data ingestion tool that I am building with Azure leveraging several of Azure PaaS services.

Content for What's new in Databricks talk

Notes about platforms such as Azure-Databricks, Apache Spark, Azure-DevOps etc.

Azure DevOps extension for interacting with Azure Databricks via REST API

Data Engineering Project using Tokyo Olympic Data

[👩‍🏫] In this repository I'll show you how to use Azure Databricks for development and training machine learning models, and build a MLOps pipeline to serving them with CI/CD process.

A summative coursework for CSC8101 Engineering for AI

Data Engineering Project - Python, PySpark & SQL - Azure Data Factory (ADF), DataBricks, Synapse Analytics, Azure Data Lake Storage (ADLS) Gen2, Power BI, Tableau and Looker Studio

Import data from a CSV into a Hive partitioned and non-partitioned table

Microsoft Azure Databricks Playground

A simple pipeline to transform data within Azure Data Factory using Azure Databricks. Although it is written in Scala the same can be replicated in Python.

Pyspark tutorials

Mount file from Azure to Databricks and ETL transformation on the datasets

Azure Databricks notebook sample to connect Blob Storage of Azure Log Analytics

Repository contains TF files pertaining to Azure pipelines and Azure databricks

Conducted comprehensive data analysis on Retail Sales data leveraging Azure Core Services, Azure Databricks and Power BI

This repository contains code for an end-to-end IoT data pipeline using Azure services. It ingests, processes, and stores IoT device data from AWS S3 to Azure Data Lake Storage and Azure SQL Database, leveraging Azure Data Factory and Azure Functions for seamless integration and automation.