Production ML

From prototype to production:

Data pipeline:

• Source data: SQL, CSV, files, streaming.
• Cleaning: handle nulls, outliers, encoding.
• Feature engineering: derive useful features.
• Train/val/test split: ensure no leakage.
• Versioning: track which data trained which model.

Model pipeline:

• Train: monitor loss, save checkpoints.
• Evaluate: validation set + held-out test.
• Hyperparameter tuning: grid/random/Bayesian.
• Model selection: best validation performance.

Deployment:

• Serving: REST API, gRPC, batch.
• Latency: GPU vs CPU inference.
• Quantization: FP32 → INT8 for 4x speedup.
• ONNX: portable model format.
• Triton, TorchServe, TF Serving: production servers.

Monitoring:

• Input distribution drift.
• Output distribution drift.
• Model performance on production data.
• Latency, throughput, errors.

Retraining:

• Periodic: weekly/monthly with fresh data.
• Triggered: on drift detection.
• Offline (batch) vs online (incremental).

Tooling ecosystem:

• MLflow: experiment tracking.
• Weights & Biases: cloud experiment tracking.
• DVC: data + model versioning.
• Kubeflow: ML on K8s.
• Vertex AI / SageMaker / Azure ML: managed.

Pretrained models:

• Hugging Face: BERT, GPT-2, Llama, ViT, etc.
• Save weeks of training.
• Fine-tune on your task.

Privacy + safety:

• Federated learning: train on devices, aggregate.
• Differential privacy: bound information leakage.
• Adversarial examples: small input perturbations fool models.
• Bias auditing: model behaves equally across demographics.

LLM-specific:

• RAG (retrieval-augmented generation): inject context.
• Fine-tuning: LoRA + adapters.
• Prompt engineering: shape model behavior via prompts.
• Constitutional AI: train models to refuse harmful requests.

Common mistakes:

• Train/test leakage.
• Imbalanced classes.
• Forgetting data preprocessing in production.
• Trusting model on out-of-distribution input.
• Ignoring fairness/bias.

For most apps: don't roll your own. Use Hugging Face + fine-tune.