19. Case Study: Recommendation Engine

Design a production recommendation engine for an e-commerce platform with 50M users and 5M products. The system must serve personalized product recommendations on the homepage, product page, and checkout page.

Requirements

Functional: Show personalized product recommendations to each user. Support "similar products" on product pages. Support "frequently bought together" on checkout.
Non-functional: p99 latency < 200ms for homepage, 500M DAU, 10K QPS peak, 99.95% uptime. Recommendations must update within 1 hour of user action.

Scale Estimation

MAU: 50M → DAU: ~20M (40%)
Homepage loads per day: 40M (2 per user per day)
Peak QPS: ~2,000 with 5x holiday surge → 10,000 QPS
Products per recommendation: 20
Total recommendations per day: 40M × 20 = 800M

Features per user: ~200 (demographics, purchase history, browsing, etc.)
Features per product: ~100 (category, price, rating, inventory, etc.)
Feature store read per request: ~300 features × 8 bytes ≈ 2.4KB

Architecture

The recommendation system uses a two-stage retrieval + ranking architecture:

# Two-stage pipeline
Stage 1: Retrieval (candidate generation)
  Retrieve 500 candidate products per user
  Methods:
    - Collaborative filtering (matrix factorization)
    - Content-based filtering (product category similarity)
    - Trending / popular items (cold-start fallback)

Stage 2: Ranking
  Score and rank the 500 candidates using a deep neural network
  Features: user embeddings, product embeddings, cross features, context
  Output: top-20 ranked products

Stage 3: Post-processing
  Apply business rules (exclude out-of-stock, respect price filters)
  Apply diversity rules (don't show 20 products from the same brand)
  Apply personalization rules (boost categories user browsed recently)

Caching Strategy

Cache Type	Key	TTL	Rationale
Homepage recommendations	user_id + page	1 hour	Slow-changing for most users
Similar products	product_id	24 hours	Product similarity changes rarely
Product embeddings	product_id	24 hours	Stored in Redis, recomputed daily
User embeddings	user_id	1 hour	Updated after purchase/click
Trending products	category + region	5 minutes	Changes with real-time trends

Data Pipeline

# Offline training (daily at midnight)
1. Load user interaction logs (clicks, purchases, cart adds) from past 30 days
2. Compute user and product embeddings (Matrix Factorization, 128-dim)
3. Train ranking model (DNN with cross features)
4. Evaluate on held-out test set
5. Register if metrics improve

# Online feature computation (streaming)
Real-time events (click, add-to-cart, purchase):
  → Kafka → feature update → invalidate relevant caches
  → Update user embedding incrementally

# Batch feature computation (hourly)
Compute aggregated features:
  - Popularity scores per category
  - "Frequently bought together" pairs
  - Price elasticity per segment

Monitoring

Business metrics: Click-through rate, conversion rate, revenue per recommendation.
System metrics: Latency p50/p95/p99, retrieval count, cache hit ratio, feature fetch time.
Model metrics: Prediction distribution, feature drift, user embedding drift.
A/B testing: Continuous comparison of champion vs challenger models.

Key Trade-offs

Personalization vs cold-start: New users have no history → rely on trending/popular items. Gradually blend in personalized recommendations as data accumulates.
Freshness vs stability: Real-time embedding updates improve freshness but can cause thrashing. Micro-batch updates every 5 minutes balance both.
Diversity vs relevance: Strictly relevant recommendations may all be from one category. Apply diversity penalty during ranking.

Exercise: Extend this recommendation engine to include a "real-time session-based" component that uses the user's current browsing session (last 5 clicks) to adjust recommendations instantly, without waiting for daily retraining. Describe: a) how you compute session vectors, b) where you store them, c) how they blend with the user's long-term profile, d) the latency budget for the session component.