3. Core Objectives

Every AI system design balances five core objectives. Understanding their interactions and trade-offs is essential to making good architectural decisions.

1. Accuracy

The model's ability to produce correct predictions. Measured by task-specific metrics: accuracy, precision, recall, F1, RMSE, perplexity, etc.

Trade-off: Higher accuracy often requires larger models, more data, and longer training — increasing cost and latency.
Strategies: Ensemble methods increase accuracy at the cost of serving complexity. Distillation reduces model size while retaining most accuracy.

2. Scalability

The ability to handle growing load (data volume, request rate, model count) without degrading performance.

Horizontal scaling: Add more inference servers behind a load balancer.
Vertical scaling: More powerful GPUs/TPUs for training.
Data scalability: Feature stores, data versioning, and streaming pipelines that handle TB/day.

3. Latency

End-to-end response time from request to prediction. Critical for real-time applications.

Application	Target Latency
Real-time fraud detection	<100ms
Chatbot response	<500ms
Document summarization	<5s (async acceptable)
Batch processing (training)	Hours to days

Latency budget breakdown: Network (5ms), feature fetch (20ms), model inference (50ms), post-processing (10ms), cache lookups (5ms). Budgeting these explicitly helps identify bottlenecks.

4. Adaptability

The system's ability to adapt to changing data distributions, user behavior, and business requirements.

Model retraining: Scheduled (daily/weekly) or triggered (drift detected).
Online learning: Models update incrementally with new data (e.g., bandit algorithms).
Feature drift: Detect when input distributions shift and alert before model quality degrades.
Configurability: Feature flags, model version routing, prompt versioning.

5. Observability

The ability to understand the system's internal state from its external outputs. For AI systems, this means:

Model metrics: Latency histograms, throughput, error rates, prediction distribution.
Data drift: Statistical divergence between training and serving data.
Prediction explanations: SHAP/LIME values for model interpretability.
Resource utilization: GPU/CPU/memory usage, queue depths, cache hit ratios.
Alerting: Automated alerts when metrics breach thresholds.

# Observable metrics for an inference service
latency_p50: 45ms
latency_p99: 185ms
throughput: 3,200 req/s
error_rate: 0.02%
cache_hit_ratio: 0.78
model_drift_score: 0.12  (threshold: 0.2 → retrain)

Exercise: For a real-time translation service, rank the five objectives (accuracy, scalability, latency, adaptability, observability) by priority and explain your ordering. Describe a scenario where improving one objective would hurt another.