15. Trade-Offs

Every ML system design decision involves trade-offs. The ability to recognize, articulate, and navigate trade-offs is what separates senior from junior engineers.

Accuracy vs Latency

Approach	Accuracy	Latency	Cost
Large deep ensemble (3 models)	Best (AUC 0.96)	~150ms	3 GPUs
Single XGBoost	Good (AUC 0.93)	~20ms	1 CPU
Linear model	Moderate (AUC 0.88)	~2ms	Minimal
Rule-based heuristic	Low (AUC 0.80)	~1ms	None

How to decide: Let business requirements drive the choice. If a 3% accuracy gain is worth $1M/year in revenue, use the ensemble. If latency SLA is p99 < 50ms, use XGBoost.

Freshness vs Stability

Real-time features: Fresh but noisy. A single user action can change features drastically.
Batch features: Stable but stale. Daily snapshots miss intra-day patterns.
Solution: Micro-batch features (5-minute windows) for near-real-time freshness with smoothed transitions.

Cost vs Redundancy

Single-region deployment: Lower cost, risk of full outage if region goes down.
Multi-region active-active: 2x—3x cost, near-zero downtime.
Multi-region active-passive: ~1.5x cost, RTO of minutes.

Consistency vs Availability (CAP for ML)

Scenario	Choice	Rationale
Fraud detection (financial loss on stale data)	Strong consistency	Stale features = wrong fraud decision
Content recommendations	Eventual consistency	Stale recommendations still reasonable
Real-time bidding (must respond fast)	Available over consistent	Better to serve slightly stale data than miss the bid

Batch vs Real-Time Training

Batch retraining (nightly): Simple, stable, but model is always 1 day behind.
Continuous training: Model updates with every new batch of data. Complex infrastructure, faster adaptation.
Online learning: Model updates per-sample. Fastest adaptation, hardest to debug and validate.

Model Complexity vs Interpretability

# Trade-off visualization
Simple (Logistic Regression):
  ✓ Interpretable (coefficients = feature importance)
  ✓ Fast training and inference
  ✗ Lower accuracy for complex patterns

Complex (Deep Neural Network):
  ✓ Highest accuracy
  ✗ Black box (needs SHAP/LIME for explanation)
  ✗ Expensive training and inference

Middle ground: Gradient Boosted Trees (XGBoost, LightGBM)
  ✓ High accuracy
  ✓ Feature importance available
  ✓ Reasonably fast

Exercise: You are designing a real-time ML system for stock trading signals. Identify 5 key trade-offs. For each: describe the two options, the pros/cons of each, and make a recommendation with justification. Consider: latency vs accuracy, model complexity vs speed, batch vs online training, consistency vs availability, cost vs redundancy.