jamilvilela / databricks-stocks-analysis Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

End-to-end data engineering process using Databricks, AWS S3, PySpark, Delta Lake, Python, MongoDB, Power BI

0 stars 0 forks Branches Tags Activity

Name		Name	Last commit message	Last commit date
Latest commit History 23 Commits
data_analysis		data_analysis
images		images
scripts-ELT		scripts-ELT
utilities		utilities
.gitignore		.gitignore
README.md		README.md

Repository files navigation

databricks-stocks-analysis

End-to-end data engineering process using Databricks, AWS S3, PySpark, Delta Lake, Python, MongoDB, Power BI

This project get data from source systems:

Yahoo Finance API - Stocks data
USA Federal Reserve Interest tax rate
Brazil Central Bank Interest tax rate
MongoDB database - App Stocks Portfolio

Data model

Databricks Visualization dashboard

Databricks Repos integrated to Github

Databricks Workflow

AWS S3 bucket

AWS S3 bucket growth