MLOps Patterns

Operationalize machine learning models from experimentation to production deployment and monitoring.

Purpose

Provide strategic guidance for ML engineers and platform teams to build production-grade ML infrastructure. Cover the complete lifecycle: experiment tracking, model registry, feature stores, deployment patterns, pipeline orchestration, and monitoring.

When to Use This Skill

Use this skill when:

• Designing MLOps infrastructure for production ML systems
• Selecting experiment tracking platforms (MLflow, Weights & Biases, Neptune)
• Implementing feature stores for online/offline feature serving
• Choosing model serving solutions (Seldon Core, KServe, BentoML, TorchServe)
• Building ML pipelines for training, evaluation, and deployment
• Setting up model monitoring and drift detection
• Establishing model governance and compliance frameworks
• Optimizing ML inference costs and performance
• Migrating from notebooks to production ML systems
• Implementing continuous training and automated retraining

Core Concepts

1. Experiment Tracking

Track experiments systematically to ensure reproducibility and collaboration.

Key Components:

• Parameters: Hyperparameters logged for each training run
• Metrics: Performance measures tracked over time (accuracy, loss, F1)
• Artifacts: Model weights, plots, datasets, configuration files
• Metadata: Tags, descriptions, Git commit SHA, environment details

Platform Comparison:

MLflow (Open-source standard):

• Framework-agnostic (PyTorch, TensorFlow, scikit-learn, XGBoost)
• Self-hosted or cloud-agnostic deployment
• Integrated model registry
• Basic UI, adequate for most use cases
• Free, requires infrastructure management

Weights & Biases (SaaS, collaboration-focused):

• Advanced visualization and dashboards
• Integrated hyperparameter optimization (Sweeps)
• Excellent team collaboration features
• SaaS pricing scales with usage
• Best-in-class UI

Neptune.ai (Enterprise-grade):

• Enterprise features (RBAC, audit logs, compliance)
• Integrated production monitoring
• Higher cost than W&B
• Good for regulated industries

Selection Criteria:

• Open-source requirement → MLflow
• Team collaboration critical → Weights & Biases
• Enterprise compliance (RBAC, audits) → Neptune.ai
• Hyperparameter optimization primary → Weights & Biases (Sweeps)

For detailed comparison and decision framework, see references/experiment-tracking.md.

2. Model Registry and Versioning

Centralize model artifacts with version control and stage management.

Model Registry Components:

• Model artifacts (weights, serialized models)
• Training metrics (accuracy, F1, AUC)
• Hyperparameters used during training
• Training dataset version
• Feature schema (input/output signatures)
• Model cards (documentation, use cases, limitations)

Stage Management:

• None: Newly registered model
• Staging: Testing in pre-production environment
• Production: Serving live traffic
• Archived: Deprecated, retained for compliance

Versioning Strategies:

Semantic Versioning for Models:

• Major version (v2.0.0): Breaking change in input/output schema
• Minor version (v1.1.0): New feature, backward-compatible
• Patch version (v1.0.1): Bug fix, model retrained on new data

Git-Based Versioning:

• Model code in Git (training scripts, configuration)
• Model weights in DVC (Data Version Control) or Git-LFS
• Reproducibility via commit SHA + data version hash

For model lineage tracking and registry patterns, see references/model-registry.md.

3. Feature Stores

Centralize feature engineering to ensure consistency between training and inference.

Problem Addressed: Training/serving skew

• Training: Features computed with future knowledge (data leakage)
• Inference: Features computed with only past data
• Result: Model performs well in training but fails in production

Feature Store Solution:

Online Feature Store:

• Purpose: Low-latency feature retrieval for real-time inference
• Storage: Redis, DynamoDB, Cassandra (key-value stores)
• Latency: Sub-10ms for feature lookup
• Use Case: Real-time predictions (fraud detection, recommendations)

Offline Feature Store:

• Purpose: Historical feature data for training and batch inference
• Storage: Parquet files (S3/GCS), data warehouses (Snowflake, BigQuery)
• Latency: Seconds to minutes (batch retrieval)
• Use Case: Model training, backtesting, batch predictions

Point-in-Time Correctness:

• Ensures no future data leakage during training
• Feature values at time T only use data available before time T
• Critical for avoiding overly optimistic training metrics

Platform Comparison:

Feast (Open-source, cloud-agnostic):

• Most popular open-source feature store
• Supports Redis, DynamoDB, Datastore (online) and Parquet, BigQuery, Snowflake (offline)
• Cloud-agnostic, no vendor lock-in
• Active community, growing adoption

Tecton (Managed, production-grade):

• Feast-compatible API
• Fully managed service
• Integrated monitoring and governance
• Higher cost, enterprise-focused

SageMaker Feature Store (AWS):

• Integrated with AWS ecosystem
• Managed online/offline stores
• AWS lock-in

Databricks Feature Store (Databricks):

• Unity Catalog integration
• Delta Lake for offline storage
• Databricks ecosystem lock-in

Selection Criteria:

• Open-source, cloud-agnostic → Feast
• Managed solution, production-grade → Tecton
• AWS ecosystem → SageMaker Feature Store
• Databricks users → Databricks Feature Store

For feature engineering patterns and implementation, see references/feature-stores.md.

4. Model Serving Patterns

Deploy models for synchronous, asynchronous, batch, or streaming inference.

Serving Patterns:

REST API Deployment:

• Pattern: HTTP endpoint for synchronous predictions
• Latency: <100ms acceptable
• Use Case: Request-response applications
• Tools: Flask, FastAPI, BentoML, Seldon Core

gRPC Deployment:

• Pattern: High-performance RPC for low-latency inference
• Latency: <10ms target
• Use Case: Microservices, latency-critical applications
• Tools: TensorFlow Serving, TorchServe, Seldon Core

Batch Inference:

• Pattern: Process large datasets offline
• Latency: Minutes to hours acceptable
• Use Case: Daily/hourly predictions for millions of records
• Tools: Spark, Dask, Ray

Streaming Inference:

• Pattern: Real-time predictions on streaming data
• Latency: Milliseconds
• Use Case: Fraud detection, anomaly detection, real-time recommendations
• Tools: Kafka + Flink/Spark Streaming

Platform Comparison:

Seldon Core (Kubernetes-native, advanced):

• Advanced deployment strategies (canary, A/B testing, multi-armed bandits)
• Multi-framework support
• Integrated explainability (Alibi)
• High complexity, steep learning curve

KServe (CNCF standard):

• Standardized InferenceService API
• Serverless scaling (scale-to-zero with Knative)
• Kubernetes-native
• Growing adoption, CNCF backing

BentoML (Python-first, simplicity):

• Easiest to get started
• Excellent developer experience
• Local testing → cloud deployment
• Lower complexity than Seldon/KServe

TorchServe (PyTorch official):

• PyTorch-specific serving
• Production-grade, optimized for PyTorch models
• Less flexible for multi-framework use

TensorFlow Serving (TensorFlow official):

• TensorFlow-specific serving
• Production-grade, optimized for TensorFlow models
• Less flexible for multi-framework use

Selection Criteria:

• Kubernetes, advanced deployments → Seldon Core or KServe
• Python-first, simplicity → BentoML
• PyTorch-specific → TorchServe
• TensorFlow-specific → TensorFlow Serving
• Managed solution → SageMaker/Vertex AI/Azure ML

For model optimization and serving infrastructure, see references/model-serving.md.

5. Deployment Strategies

Deploy models safely with rollback capabilities.

Blue-Green Deployment:

• Two identical environments (Blue: current, Green: new)
• Deploy to Green, test, switch 100% traffic instantly
• Instant rollback (switch back to Blue)
• Trade-off: Requires 2x infrastructure, all-or-nothing switch

Canary Deployment:

• Gradual rollout to subset of traffic
• Route 5% → 10% → 25% → 50% → 100% over time
• Monitor metrics at each stage, rollback if degradation
• Trade-off: Complex routing logic, longer deployment time

Shadow Deployment:

• New model receives traffic but predictions not used
• Compare new model vs old model offline
• Zero risk to production
• Trade-off: Requires 2x compute, delayed feedback

A/B Testing:

• Split traffic between model versions
• Measure business metrics (conversion rate, revenue)
• Statistical significance testing
• Use Case: Optimize for business outcomes, not just ML metrics

Multi-Armed Bandit (MAB):

• Epsilon-greedy: Explore (try new models) vs Exploit (use best model)
• Thompson Sampling: Bayesian approach to exploration
• Use Case: Continuous optimization, faster convergence than A/B

Selection Criteria:

• Low-risk model → Blue-green (instant cutover)
• Medium-risk model → Canary (gradual rollout)
• High-risk model → Shadow (test in production, no impact)
• Business optimization → A/B testing or MAB

For deployment architecture and examples, see references/deployment-strategies.md.

6. ML Pipeline Orchestration

Automate training, evaluation, and deployment workflows.

Training Pipeline Stages:

1. Data Validation (Great Expectations, schema checks)
2. Feature Engineering (transform raw data)
3. Data Splitting (train/validation/test)
4. Model Training (hyperparameter tuning)
5. Model Evaluation (accuracy, fairness, explainability)
6. Model Registration (push to registry if metrics pass thresholds)
7. Deployment (promote to staging/production)

Continuous Training Pattern:

• Monitor production data for drift
• Detect data distribution changes (KS test, PSI)
• Trigger automated retraining when drift detected
• Validate new model before deployment
• Deploy via canary or shadow strategy

Platform Comparison:

Kubeflow Pipelines (ML-native, Kubernetes):

• ML-specific pipeline orchestration
• Kubernetes-native (scales with K8s)
• Component-based (reusable pipeline steps)
• Integrated with Katib (hyperparameter tuning)

Apache Airflow (Mature, general-purpose):

• Most mature orchestration platform
• Large ecosystem, extensive integrations
• Python-based DAGs
• Not ML-specific but widely used for ML workflows

Metaflow (Netflix, data science-friendly):

• Human-centric design, easy for data scientists
• Excellent local development experience
• Versioning built-in
• Simpler than Kubeflow/Airflow

Prefect (Modern, Python-native):

• Dynamic workflows, not static DAGs
• Better error handling than Airflow
• Modern UI and developer experience
• Growing community

Dagster (Asset-based, testing-focused):

• Asset-based thinking (not just task dependencies)
• Strong testing and data quality features
• Modern approach, good for data teams
• Smaller community than Airflow

Selection Criteria:

• ML-specific, Kubernetes → Kubeflow Pipelines
• Mature, battle-tested → Apache Airflow
• Data scientists, ease of use → Metaflow
• Software engineers, testing → Dagster
• Modern, simpler than Airflow → Prefect

For pipeline architecture and examples, see references/ml-pipelines.md.

7. Model Monitoring and Observability

Monitor production models for drift, performance, and quality.

Data Drift Detection:

• Definition: Input feature distributions change over time
• Impact: Model trained on old distribution, predictions degrade
• Detection Methods:
  • Kolmogorov-Smirnov (KS) Test: Compare distributions
  • Population Stability Index (PSI): Measure distribution shift
  • Chi-Square Test: For categorical features
• Action: Trigger automated retraining when drift detected

Model Drift Detection:

• Definition: Model prediction quality degrades over time
• Impact: Accuracy, precision, recall decrease
• Detection Methods:
  • Ground truth accuracy (delayed labels)
  • Prediction distribution changes
  • Calibration drift (predicted probabilities vs actual outcomes)
• Action: Alert team, trigger retraining

Performance Monitoring:

• Metrics:
  • Latency: P50, P95, P99 inference time
  • Throughput: Predictions per second
  • Error Rate: Failed predictions / total predictions
  • Resource Utilization: CPU, memory, GPU usage
• Alerting Thresholds:
  • P95 latency > 100ms → Alert
  • Error rate > 1% → Alert
  • Accuracy drop > 5% → Trigger retraining

Business Metrics Monitoring:

• Downstream impact: Conversion rate, revenue, user satisfaction
• Model predictions → business outcomes correlation
• Use Case: Optimize models for business value, not just ML metrics

Tools:

• Evidently AI: Data drift, model drift, data quality reports
• Prometheus + Grafana: Performance metrics, custom dashboards
• Arize AI: ML observability platform
• Fiddler: Model monitoring and explainability

For monitoring architecture and implementation, see references/model-monitoring.md.

8. Model Optimization Techniques

Reduce model size and inference latency.

Quantization:

• Convert model weights from float32 to int8
• Model size reduction: 4x smaller
• Inference speed: 2-3x faster
• Accuracy impact: Minimal (<1% degradation typically)
• Tools: PyTorch quantization, TensorFlow Lite, ONNX Runtime

Model Distillation:

• Train small student model to mimic large teacher model
• Transfer knowledge from teacher (BERT-large) to student (DistilBERT)
• Size reduction: 2-10x smaller
• Speed improvement: 2-10x faster
• Use Case: Deploy small model on edge devices, reduce inference cost

ONNX Conversion:

• Convert models to Open Neural Network Exchange (ONNX) format
• Cross-framework compatibility (PyTorch → ONNX → TensorFlow)
• Optimized inference with ONNX Runtime
• Speed improvement: 1.5-3x faster than native framework

Model Pruning:

• Remove less important weights from neural networks
• Sparsity: 30-90% of weights set to zero
• Size reduction: 2-10x smaller
• Accuracy impact: Minimal with structured pruning

For optimization techniques and examples, see references/model-serving.md.

9. LLMOps Patterns

Operationalize Large Language Models with specialized patterns.

LLM Fine-Tuning Pipelines:

• LoRA (Low-Rank Adaptation): Parameter-efficient fine-tuning
• QLoRA: Quantized LoRA (4-bit quantization)
• Pipeline: Base model → Fine-tuning dataset → LoRA adapters → Merged model
• Tools: Hugging Face PEFT, Axolotl

Prompt Versioning:

• Version control for prompts (Git, prompt management platforms)
• A/B testing prompts for quality and cost optimization
• Monitoring prompt effectiveness over time

RAG System Monitoring:

• Retrieval quality: Relevance of retrieved documents
• Generation quality: Answer accuracy, hallucination detection
• End-to-end latency: Retrieval + generation time
• Tools: LangSmith, Arize Phoenix

LLM Inference Optimization:

• vLLM: High-throughput LLM serving
• TensorRT-LLM: NVIDIA-optimized LLM inference
• Text Generation Inference (TGI): Hugging Face serving
• Batching: Dynamic batching for throughput

Embedding Model Management:

• Version embeddings alongside models
• Monitor embedding drift (distribution changes)
• Update embeddings when underlying model changes

For LLMOps patterns and implementation, see references/llmops-patterns.md.

10. Model Governance and Compliance

Establish governance for model risk management and regulatory compliance.

Model Cards:

• Documentation: Model purpose, training data, performance metrics
• Limitations: Known biases, failure modes, out-of-scope use cases
• Ethical considerations: Fairness, privacy, societal impact
• Template: Model Card Toolkit (Google)

Bias and Fairness Detection:

• Measure disparate impact across demographic groups
• Tools: Fairlearn, AI Fairness 360 (IBM)
• Metrics: Demographic parity, equalized odds, calibration
• Mitigation: Reweighting, adversarial debiasing, threshold optimization

Regulatory Compliance:

• EU AI Act: High-risk AI systems require documentation, monitoring
• Model Risk Management (SR 11-7): Banking industry requirements
• GDPR: Right to explanation for automated decisions
• HIPAA: Healthcare data privacy

Audit Trails:

• Log all model versions, training runs, deployments
• Track who approved model transitions (staging → production)
• Retain historical predictions for compliance audits
• Tools: MLflow, Neptune.ai (audit logs)

For governance frameworks and compliance, see references/governance.md.

Decision Frameworks

Framework 1: Experiment Tracking Platform Selection

Decision Tree:

Start with primary requirement:

• Open-source, self-hosted requirement → MLflow
• Team collaboration, advanced visualization (budget available) → Weights & Biases
• Team collaboration, advanced visualization (no budget) → MLflow
• Enterprise compliance (audit logs, RBAC) → Neptune.ai
• Hyperparameter optimization primary use case → Weights & Biases (Sweeps)

Detailed Criteria:

Criteria	MLflow	Weights & Biases	Neptune.ai
Cost	Free	$200/user/month	$300/user/month
Collaboration	Basic	Excellent	Good
Visualization	Basic	Excellent	Good
Hyperparameter Tuning	External (Optuna)	Integrated (Sweeps)	Basic
Model Registry	Included	Add-on	Included
Self-Hosted	Yes	No (paid only)	Limited
Enterprise Features	No	Limited	Excellent

Recommendation by Organization:

• Startup (<50 people): MLflow (free, adequate) or W&B (if budget)
• Growth (50-500 people): Weights & Biases (team collaboration)
• Enterprise (>500 people): Neptune.ai (compliance) or MLflow (cost)

For detailed decision framework, see references/decision-frameworks.md.

Framework 2: Feature Store Selection

Decision Matrix:

Primary requirement:

• Open-source, cloud-agnostic → Feast
• Managed solution, production-grade, multi-cloud → Tecton
• AWS ecosystem → SageMaker Feature Store
• GCP ecosystem → Vertex AI Feature Store
• Azure ecosystem → Azure ML Feature Store
• Databricks users → Databricks Feature Store
• Self-hosted with UI → Hopsworks

Criteria Comparison:

Factor	Feast	Tecton	Hopsworks	SageMaker FS
Cost	Free	$$$$	Free (self-host)	$$$
Online Serving	Redis, DynamoDB	Managed	RonDB	Managed
Offline Store	Parquet, BigQuery, Snowflake	Managed	Hive, S3	S3
Point-in-Time	Yes	Yes	Yes	Yes
Monitoring	External	Integrated	Basic	External
Cloud Lock-in	No	No	No	AWS

Recommendation:

• Open-source, self-managed → Feast
• Managed, production-grade → Tecton
• AWS ecosystem → SageMaker Feature Store
• Databricks users → Databricks Feature Store

For detailed decision framework, see references/decision-frameworks.md.

Framework 3: Model Serving Platform Selection

Decision Tree:

Infrastructure:

• Kubernetes-based → Advanced deployment patterns needed?
  • Yes → Seldon Core (most features) or KServe (CNCF standard)
  • No → BentoML (simpler, Python-first)
• Cloud-native (managed) → Cloud provider?
  • AWS → SageMaker Endpoints
  • GCP → Vertex AI Endpoints
  • Azure → Azure ML Endpoints
• Framework-specific → Framework?
  • PyTorch → TorchServe
  • TensorFlow → TensorFlow Serving
• Serverless / minimal infrastructure → BentoML or Cloud Functions

Detailed Criteria:

Feature	Seldon Core	KServe	BentoML	TorchServe
Kubernetes-Native	Yes	Yes	Optional	No
Multi-Framework	Yes	Yes	Yes	PyTorch-only
Deployment Strategies	Excellent	Good	Basic	Basic
Explainability	Integrated	Integrated	External	No
Complexity	High	Medium	Low	Low
Learning Curve	Steep	Medium	Gentle	Gentle

Recommendation:

• Kubernetes, advanced deployments → Seldon Core or KServe
• Python-first, simplicity → BentoML
• PyTorch-specific → TorchServe
• TensorFlow-specific → TensorFlow Serving
• Managed solution → SageMaker/Vertex AI/Azure ML

For detailed decision framework, see references/decision-frameworks.md.

Framework 4: ML Pipeline Orchestration Selection

Decision Matrix:

Primary use case:

• ML-specific pipelines, Kubernetes-native → Kubeflow Pipelines
• General-purpose orchestration, mature ecosystem → Apache Airflow
• Data science workflows, ease of use → Metaflow
• Modern approach, asset-based thinking → Dagster
• Dynamic workflows, Python-native → Prefect

Criteria Comparison:

Factor	Kubeflow	Airflow	Metaflow	Dagster	Prefect
ML-Specific	Excellent	Good	Excellent	Good	Good
Kubernetes	Native	Compatible	Optional	Compatible	Compatible
Learning Curve	Steep	Steep	Gentle	Medium	Medium
Maturity	High	Very High	Medium	Medium	Medium
Community	Large	Very Large	Growing	Growing	Growing

Recommendation:

• ML-specific, Kubernetes → Kubeflow Pipelines
• Mature, battle-tested → Apache Airflow
• Data scientists → Metaflow
• Software engineers → Dagster
• Modern, simpler than Airflow → Prefect

For detailed decision framework, see references/decision-frameworks.md.

Implementation Patterns

Pattern 1: End-to-End ML Pipeline

Automate the complete ML workflow from data to deployment.

Pipeline Stages:

1. Data Validation (Great Expectations)
2. Feature Engineering (transform raw data)
3. Data Splitting (train/validation/test)
4. Model Training (with hyperparameter tuning)
5. Model Evaluation (accuracy, fairness, explainability)
6. Model Registration (push to MLflow registry)
7. Deployment (promote to staging/production)

Architecture:

Data Lake → Data Validation → Feature Engineering → Training → Evaluation
    ↓
Model Registry (staging) → Testing → Production Deployment

For implementation details and code examples, see references/ml-pipelines.md.

Pattern 2: Continuous Training

Automate model retraining based on drift detection.

Workflow:

1. Monitor production data for distribution changes
2. Detect data drift (KS test, PSI)
3. Trigger automated retraining pipeline
4. Validate new model (accuracy, fairness)
5. Deploy via canary strategy (5% → 100%)
6. Monitor new model performance
7. Rollback if metrics degrade

Trigger Conditions:

• Scheduled: Daily/weekly retraining
• Data drift: KS test p-value < 0.05
• Model drift: Accuracy drop > 5%
• Data volume: New training data exceeds threshold (10K samples)

For implementation details, see references/ml-pipelines.md.

Pattern 3: Feature Store Integration

Ensure consistent features between training and inference.

Architecture:

Offline Store (Training):
  Parquet/BigQuery → Point-in-Time Join → Training Dataset

Online Store (Inference):
  Redis/DynamoDB → Low-Latency Lookup → Real-Time Prediction