6. Batch vs Real-Time Systems

Batch ML Systems

Batch systems process data in scheduled intervals. All predictions are pre-computed and stored.

Aspect	Batch
Frequency	Hourly, daily, weekly
Latency	Minutes to hours
Compute	Spark, Airflow, scheduled jobs
Storage	Pre-computed predictions in DB or cache
Use cases	Daily recommendations, churn predictions, credit scoring

# Batch ML pipeline (daily)
At 2 AM:
  1. Load yesterday's features (offline store)
  2. Load ground truth for evaluation
  3. Retrain model if drift detected or scheduled
  4. Batch predict for all active users
  5. Write predictions to predictions table
  6. Applications read pre-computed predictions

Real-Time ML Systems

Real-time systems make predictions on-demand with sub-second latency.

Aspect	Real-Time
Trigger	Request-driven (user action, API call)
Latency	Milliseconds to seconds
Compute	Inference servers (Triton, TorchServe, vLLM)
Storage	Online feature store (Redis), prediction cache
Use cases	Fraud detection, real-time recommendations, ad serving

Lambda Architecture

Processes data through both batch (comprehensive, high latency) and speed (approximate, low latency) layers, merging results in the serving layer.

Batch layer: Complete data, accurate results, higher latency.
Speed layer: Recent data, approximate results, low latency.
Serving layer: Merges batch and speed results for final output.

Kappa Architecture

Everything is treated as a stream. Batch is replayed stream processing. Simpler infrastructure but requires robust stream processing (Kafka + Flink).

Choosing Your Approach

Scenario	Recommendation
You need instant predictions (fraud, ads)	Real-time
Predictions can be pre-computed (daily feed)	Batch
Mix of historical and real-time data	Lambda
Simple pipeline, stream-native team	Kappa

Exercise: A music streaming app needs two ML features: a) a daily "Discover Weekly" playlist for each user, b) a "Next Song" recommendation that updates instantly based on the current song and user behavior. Design the system — which features use batch, which use real-time, and how do they share infrastructure (feature store, model registry)?