Training pipelines, model serving, monitoring, drift
Practise these as flashcards. One or two lines each; the ones you miss come back first.
Start freeMLOps is the practice of applying DevOps principles to machine learning workflows. It automates and streamlines the ML lifecycle, making model deployment, monitoring, and maintenance more reliable and scalable.
Experiment tracking records parameters, metrics, and artifacts for each model run. This helps compare results, reproduce experiments, and select the best models for deployment.
A feature store is a centralized repository for storing and serving features used in ML models. It ensures consistency between training and inference, prevents feature leakage, and streamlines feature engineering.
Batch serving processes predictions for large datasets at scheduled intervals, while online serving provides real-time predictions via APIs. Online serving requires low latency and is used for time-sensitive applications.
MLflow provides a model registry to store, version, and manage models. It allows promoting models through lifecycle stages like staging and production, and integrates with deployment tools for automated rollout.
A typical CI/CD pipeline includes data validation, model training, unit and integration tests, evaluation against benchmarks, model registry update, and automated deployment. This ensures reproducibility and reduces manual errors.
I monitor data drift by comparing statistical distributions of incoming data to the training data using tests like KS or Chi-squared. Alerts are triggered if significant drift is detected, prompting investigation or retraining.
Concept drift occurs when the relationship between features and target changes over time. This can cause model predictions to become less accurate, requiring retraining or model updates to maintain performance.
Feature stores manage feature versioning and access, ensuring that only valid features are used during training and inference. They enforce data boundaries and prevent using future information in training, which avoids leakage.
I use version control for code and data, fixed random seeds, environment management tools like Docker, and experiment tracking platforms. This allows anyone to rerun the pipeline and obtain the same results.
A/B testing allows comparing different model versions on real users, measuring performance and impact before full rollout. It helps identify the best model and reduces risk of deploying underperforming models.
I use tools like MLflow for model versioning and DVC for data versioning. Each model version is linked to a specific dataset snapshot, ensuring traceability and reproducibility across experiments and deployments.