Big Data MLOps Platform

Data Science and Machine Learning Jupyter Notebooks

Setup References

• Ready-to-run Docker images containing Jupyter applications - jupyter/docker-stacks

• Kaggle/docker-python - Kaggle Python Docker image

Dataset References

• Apple Stock Price

• Spotify Dataset 2023

• Online Retail Transactions

• Stock Market

• American Airlines Group Stock

• Amazon (AMZN) Historical Stock Price

Docker commands

 docker system prune --all --volumes --force

 docker build --no-cache -f Dockerfile --progress=auto --compress --rm -t shortthirdman-org/bigdata-mlops-platform:latest .

docker buildx build --progress=auto --compress --rm -t shortthirdman-org/bigdata-mlops-platform:latest .

 docker run -d -n mlops -p 8888:8888 --restart unless-stopped shortthirdman-org/bigdata-mlops-platform:latest

Local Setup

• Create a Python virtual environment and activate

  python -m venv dev

  .\dev\Scripts\activate

• Install the packages and dependencies as listed in requirements file

  pip install -r requirements.txt --no-cache-dir --disable-pip-version-check

• Start your development Jupyter Notebook or Jupyter Lab server

  jupyter lab --notebook-dir=.\notebooks --no-browser

  jupyter notebook

References

• TimeGPT: The First Foundation Model for Time Series Forecasting

• Staggering Returns with PyCryptoBot

• Trading Data Analysis

• Phenomenal Returns with PyCryptoBot

• Leveraging PyCryptoBot for Optimal Cryptocurrency Trading

• Forecasting Stock Using Deep Learning Along With Indicators | Medium

• Forecasting Stock Using Deep Learning Along With Indicators | OnePageCode@SubStack

• Advanced Stock Pattern Prediction using LSTM with the Attention Mechanism in TensorFlow: A step by step Guide with Apple Inc. (AAPL) Data

• Spark and Docker: Your Spark development cycle just got 10x faster!

• datamechanics/spark on Docker Hub

• Setting up a Spark standalone cluster on Docker in layman terms

• Apache Spark Standalone Cluster on Docker

• Visualizing Trading Signals in Python - Plot buy and sell trading signals in Python's graph

• Apache Hadoop and Apache Spark for Big Data Analysis

• Agent-Based Stock Trading: Design and Implementation