8. Performance

Caching

Caching stores frequently accessed data in a fast, temporary storage layer to reduce latency and offload backend systems.

Cache Strategies

Strategy	How It Works	When to Use
Cache Aside	Application checks cache first; on miss, reads DB and updates cache	Simplest, most common; good for general-purpose
Read Through	Cache automatically loads from DB on miss	Consistent caching behavior
Write Through	Write to cache and DB simultaneously	Strong consistency between cache and DB
Write Behind	Write to cache, asynchronously write to DB	High write throughput, eventual consistency OK
Cache Invalidation	Evict or update cache when data changes	Stale data tolerance varies

# Cache Aside (Pseudocode)
function getUser(userId):
    user = cache.get("user:" + userId)
    if user is None:
        user = db.query("SELECT * FROM users WHERE id = ?", userId)
        cache.set("user:" + userId, user, ttl=3600)
    return user

Cache Eviction Policies

LRU (Least Recently Used): Evicts the oldest accessed item.
LFU (Least Frequently Used): Evicts the least accessed item.
TTL (Time To Live): Items expire after a set duration.
FIFO (First In, First Out): Evicts the oldest item regardless of access.

Caching Tiers

Browser cache: Static assets stored locally (Cache-Control headers).
CDN: Edge nodes cache content geographically close to users.
In-memory cache: Redis, Memcached — sub-millisecond access, sits between app and DB.
Application cache: In-process memory, fastest but limited by RAM.

Message Queues

Queues decouple producers and consumers, allowing asynchronous processing and buffering during traffic spikes.

Competing consumers: Multiple consumers process from the same queue; each message goes to one consumer.
Pub/Sub: Each message is delivered to all subscribers (fan-out).
Back-pressure handling: Queues act as buffers; consumers process at their own pace.

# Producer
queue.send("order.created", {"order_id": 42, "user_id": 7})

# Consumer (scale horizontally)
def handle(message):
    if message.type == "order.created":
        process_payment(message.data)
        send_confirmation(message.data)

Popular queue systems: RabbitMQ, Amazon SQS, Google Pub/Sub, Apache Kafka (for high-throughput event streaming).

Concurrency & Parallelism

Concurrency: Multiple tasks making progress, interleaved on a single CPU.
Parallelism: Multiple tasks running simultaneously on multiple CPUs/cores.
Thread pool: Pre-created threads ready to execute tasks; avoids thread creation overhead.
Async I/O: Non-blocking operations (Node.js, Python asyncio) allow handling many connections with few threads.

Database Optimization

Indexes: B-tree indexes for range queries; hash indexes for equality lookups; composite indexes for multi-column filters.
Query optimization: Use EXPLAIN/ANALYZE to identify slow queries; avoid SELECT *; use covering indexes.
Connection pooling: Reuse database connections instead of opening new ones per request.
Read replicas: Offload read queries to replica databases.
Denormalization: Trade storage for query speed by duplicating data across tables.
Materialized views: Pre-compute and store expensive query results.

Exercise: Your e-commerce site's product page loads in 4 seconds. The bottleneck is a complex SQL query joining 7 tables for each product. Propose three optimization strategies (one cache-based, one DB-index-based, one architectural) and estimate the improvement for each.