6. Key Components

Production AI systems are built from a set of reusable components. Understanding each component's role, technology options, and design considerations is essential.

Feature Store

A centralized repository for features shared across training and serving. Eliminates the training-serving skew caused by inconsistent feature computation.

Offline store: Parquet/AVRO files in S3 for batch training.
Online store: Redis, DynamoDB, Cassandra for low-latency feature lookup during inference.
Feature serving API: gRPC or REST endpoint returning feature vectors for given entity keys.
Point-in-time correctness: Ensures training data uses features as they existed at the prediction time, preventing data leakage.

# Feature store usage
# Training: query historical features
features = feature_store.get_historical_features(
    entity_ids=user_ids, feature_names=["avg_purchase_7d", "session_count"],
    timestamp_column="event_time"
)

# Inference: query latest features
features = feature_store.get_online_features(
    entity_ids={"user_id": 42}, feature_names=["avg_purchase_7d"]
)

Training Pipeline

Orchestrates data loading, preprocessing, training, evaluation, and model registration.

Orchestration: Airflow, Kubeflow, Prefect, or cloud-native (Vertex AI Pipelines, SageMaker Pipelines).
Compute: GPU clusters (A100, H100), spot instances for cost savings.
Artifact storage: MLflow, W&B, DVC track experiments, metrics, and model binaries.

Model Registry

Versioned catalog of trained models with metadata: metrics, hyperparameters, training dataset ID, deployment status (staging, production, archived).

Serving Infrastructure

Exposes models for inference. Key considerations:

Tool	Best For	Key Feature
NVIDIA Triton	GPU-accelerated serving	Multi-framework, dynamic batching
vLLM	LLM inference	PagedAttention, continuous batching
TorchServe	PyTorch models	Model versioning, A/B testing
BentoML	Python model serving	Unified API, containerization
ONNX Runtime	Cross-framework serving	Optimized execution, quantization

Monitoring Infrastructure

Metrics pipeline: Prometheus collects inference latency, throughput, error rates.
Drift detection service: Runs statistical tests (KS, PSI, chi-squared) comparing training vs serving data.
Alerting: PagerDuty, Slack alerts when metrics breach thresholds.
Dashboard: Grafana dashboards with per-model and per-version views.

Feedback Collection Service

Captures explicit (ratings, corrections) and implicit (clicks, conversions) feedback. Stores it in a feedback data store, timestamped, for future retraining cycles and drift analysis.

Exercise: List the components you would need for an AI-powered email classification system (spam vs not-spam). For each component, identify: a) what technology you would use, b) what data flows through it, c) what could go wrong if the component fails.