diff --git a/index222.qmd b/index222.qmd deleted file mode 100644 index cfa3a9f..0000000 --- a/index222.qmd +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: "Introduction" ---- - -Welcome to **Building Machine Learning Systems That Don't Suck!** - -## Program Structure - -#### Session 1 - How To Start (Almost) Any Project - -* What makes production machine learning different from what you've learned. -* The strategy to solve the right problem using the right solution. -* Critical questions to ask before starting any project. -* Problem framing, inversion, and the haystack principle for building successful applications. -* The first rule of machine learning engineering and how to start building. -* Data collection strategies. A technique to determine how much data you need. -* The problem of selection bias and how to deal with it. -* Labeling data. Human annotations, natural labels and weak supervision. -* Active learning using the uncertainty and diversity sampling strategies. - -#### Session 2 - Building Models - -* The role of data cleaning and feature engineering to build better models. -* Turning data into numbers using vectorization techniques. -* Producing homogeneous features using normalization and standardization. -* Handling and interpreting missing values using imputation techniques. -* The approach to choosing the best model to solve any problem. -* Random baselines and the zero-rule algorithm. -* How to use overfitting to build models that don’t suck. -* Hyperparameter tuning and experiment tracking. -* Measuring the quality of your holdout set. -* An introduction to distributed training using data parallelism and model parallelism. - -#### Session 3 - Offline Evaluation - -* The role of a baseline to contextualize the evaluation process. -* Framing evaluation metrics in the context of business performance. -* Evaluating models using holdout sets and cross-validation. -* Introduction to data leakages and leaky validation strategies. -* Invariance tests and model fairness. -* The problems with metric summarization. -* Testing models on specific slices of data. -* Error analysis and measuring the impact of potential improvements. -* Introduction to backtesting and time-based evaluation strategies. -* Dealing with disproportionally important examples and rare cases. - -#### Session 4 - Serving Predictions - -* Introduction to model versioning and the model registry. -* The trade-offs when serving predictions. An introduction to latency and throughput. -* Dynamic serving and static serving predictions. -* Two-phase predictions and deploying models on the edge. -* Enforced modularization and inference pipelines. -* Evaluating individual pipeline models. -* Introduction to human-in-the-loop workflows. -* Cost-sensitive deployment architectures. -* Using test-time augmentations to improve predictions. -* Model compression using pruning, quantization, and knowledge distillation. - -#### Session 5 - Drift and Monitoring - -* Dealing with catastrophic predictions and the problem with edge cases. -* The problem with feedback loops and how to fix them. -* Data distribution shifts. Understanding concept drift and data drift. -* Using adversarial validation to detect distribution shifts. -* An introduction to monitoring model behavior in production systems. -* Monitoring model inputs, operational metrics, predictions, and user feedback. -* How to keep your models working in the face of distribution shifts. - -#### Session 6 - Continual Learning - -* Introduction to continual learning. -* Techniques to determine how frequently to retrain your models. -* Techniques to determine what data to use to retrain a model. -* Common triggers to initiate a retraining process. -* Understanding stateless training and stateful training. -* Catastrophic forgetting and how to prevent it. -* The importance of testing in production. -* Testing models using A/B testing, shadow deployments, canary releases, and interleaving experiments.