Cache

Cache

Part 10: Caching Strategies - Complete Performance Optimization Guide

Part 10: Caching Strategies

Optimizing Performance with Caches - Complete Guide

📌 Quick Summary (30 seconds):
Caching stores frequently accessed data in fast memory (Redis, Memcached) to reduce database load and improve response times. Cache Aside (lazy loading) is the most common pattern. Key challenges: cache invalidation, consistency, and preventing the cache stampede problem. Use TTL with random jitter to avoid stampedes.

❓ The Problem: Database is Slow

Why We Need Caching:
  • Database queries take 10-100ms (too slow for high traffic)
  • Network round trips add latency
  • Database scales poorly for repeated reads
  • Cost of DB queries adds up ($$$ in cloud)
  • Same data requested thousands of times
Without Cache vs With Cache

WITHOUT CACHE (Every request hits DB):
User Request → Database (50ms) → Response
1000 requests = 1000 database queries = 50,000ms = Database overload!

WITH CACHE (First request hits DB, rest from cache):
Request 1: User → Cache (miss) → Database (50ms) → Cache → Response
Requests 2-1000: User → Cache (HIT! ~1ms) → Response
1000 requests = 1 DB query + 999 cache hits = FAST!

📊 Cache Read Patterns

1. Cache Aside (Lazy Loading) - MOST COMMON

Cache Aside Flow

┌─────────┐ 1. Read ┌─────────┐
│ Client │ ─────────────► │ Cache │
└─────────┘ └────┬────┘
▲ │
│ 2. Cache Miss
│ ▼
│ ┌─────────┐
│ 3. Read from │
│ 4. Return │ Database │
│ Data └──────┬──────┘
│ │
│ 5. Update Cache
│ │
└────────────────────────┘
// Cache Aside Pattern Implementation
public class CacheAsideService {
    
    public User getUser(String userId) {
        // Step 1: Check cache
        User user = redis.get("user:" + userId);
        if (user != null) {
            return user;  // Cache HIT - fast path
        }
        
        // Step 2: Cache MISS - load from database
        user = database.findUser(userId);
        
        // Step 3: Store in cache for future requests
        redis.set("user:" + userId, user, Duration.ofMinutes(30));
        
        return user;
    }
}
✅ PROS
  • Simple to implement
  • Only caches requested data
  • Works for any data source
  • Redis/DB failures are isolated
❌ CONS
  • Cache miss penalty (extra latency)
  • Stale data possible
  • Cache stampede risk
  • Not good for write-heavy workloads

2. Read-Through

Read-Through Flow

Client → Cache (miss) → Cache loads from DB automatically → Cache returns to Client
(Cache library handles DB loading, client doesn't know)
✅ PROS: Client simplified, no DB code in client
❌ CONS: Less control, library specific

✍️ Cache Write Patterns

1. Write-Through

Write-Through Flow

Client → Write to Cache → Cache writes to DB (sync) → Response
(Cache and DB always consistent, but slower writes)

2. Write-Behind (Write-Back)

Write-Behind Flow

Client → Write to Cache → Immediate Response
↓ (async, batching)
Write to DB (eventual)

3. Write-Around

Write-Around Flow

Client → Write directly to DB → Cache is NOT updated
(Cache updated only on read miss - avoids cache pollution for write-heavy data)

📋 Cache Pattern Comparison

PatternRead BehaviorWrite BehaviorConsistencyBest For
Cache Aside Load on miss Write to DB, invalidate cache Eventual Read-heavy, most common
Read-Through Cache loads from DB on miss Same as Cache Aside Eventual When client wants simplicity
Write-Through Same as Cache Aside Cache + DB sync Strong Write-heavy, strong consistency needed
Write-Behind Same as Cache Aside Cache first, DB async Eventual High write throughput, latency sensitive
Write-Around Load on miss Write only to DB Eventual Write-heavy, avoid cache pollution

🗑️ Cache Invalidation Strategies

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton
StrategyHow It WorksProsConsBest For
TTL (Time To Live) Data expires after fixed time Simple, no coordination Stale data until expiration Product catalogs, configuration
Event-based Update/delete cache on data change Fresh data quickly Requires event infrastructure User profiles, real-time data
Manual Explicit invalidation API Full control Human error, operational overhead Admin-triggered updates
Version-based Cache key includes version No explicit invalidation needed Key management complexity APIs, versioned data

🐘 Cache Stampede Problem & Solutions

Problem: When a popular cache key expires, thousands of concurrent requests hit the database simultaneously, potentially bringing it down.
Cache Stampede

Time: 10:00:00 - Cache expires for "popular_product_123"
Time: 10:00:00.001 - 10,000 requests arrive simultaneously
ALL 10,000 see cache miss → ALL hit database → DATABASE CRASH!

Solutions:

1. Locking/Mutex - Only 1 request fetches, others wait
2. Request Coalescing - Deduplicate in-flight requests
3. Random TTL Jitter - Stagger expiration times
4. Probabilistic Early Expiration - Refresh early before expiry
// Solution 1: Locking (Only one request fetches)
public User getUserWithLock(String userId) {
    String cacheKey = "user:" + userId;
    User user = redis.get(cacheKey);
    if (user != null) return user;
    
    // Acquire distributed lock
    if (redis.setnx("lock:" + cacheKey, "1", Duration.ofSeconds(5))) {
        try {
            // Only one thread gets here
            user = database.findUser(userId);
            redis.set(cacheKey, user, Duration.ofMinutes(30));
        } finally {
            redis.del("lock:" + cacheKey);
        }
    } else {
        // Wait for first thread to populate cache
        Thread.sleep(100);
        user = redis.get(cacheKey);
    }
    return user;
}

// Solution 2: Random TTL Jitter
Duration ttl = Duration.ofMinutes(30).plus(Duration.ofMillis(random.nextInt(5000)));
redis.set(cacheKey, data, ttl);  // 30 minutes ± 5 seconds

🗑️ Cache Eviction Policies (When Cache is Full)

PolicyHow It WorksExample LRU (Least Recently Used) Remove least recently accessed item Redis maxmemory-policy allkeys-lru LFU (Least Frequently Used) Remove least frequently accessed item Redis 4.0+ (allkeys-lfu) FIFO (First In First Out) Remove oldest item first Simple queue-based caches TTL (Time To Live) Remove expired items Redis with EXPIRE Random Remove random item Simple, fast

📊 Cache Levels (Where to Cache)

Multi-Level Caching

Level 1: Browser Cache (HTTP Cache-Control)
↓ (miss)
Level 2: CDN (CloudFront, Cloudflare) - Static assets, API responses
↓ (miss)
Level 3: API Gateway Cache - Responses for identical requests
↓ (miss)
Level 4: Application Cache (Caffeine, Guava) - In-memory, local
↓ (miss)
Level 5: Distributed Cache (Redis, Memcached) - Shared across instances
↓ (miss)
Level 6: Database (PostgreSQL, MySQL) - Source of truth

🛠️ Popular Cache Technologies

TechnologyTypeUse CaseKey Feature
Redis In-memory data store Distributed cache, session store Persistence, data structures, cluster mode
Memcached In-memory cache Simple key-value cache Fast, simple, multi-threaded
Caffeine Local cache (JVM) Single-node, high-performance Near-zero latency, async loading
Ehcache Local/distributed Hibernate L2 cache JCache compliant, disk overflow
Hazelcast In-memory data grid Distributed computing + cache Queries, events, security

⚡ Pros & Cons of Caching

✅ BENEFITS
  • 10-100x faster responses (1ms vs 50ms)
  • Reduces database load (80%+ reduction)
  • Lower cloud costs (fewer DB reads)
  • Handles traffic spikes gracefully
  • Enables global distribution (CDN)
  • Protects database during failures
❌ CHALLENGES
  • Cache invalidation is hard
  • Stale data (inconsistency)
  • Additional infrastructure to manage
  • Memory costs (RAM is expensive)
  • Cache stampede risk
  • Cold start problem (empty cache)

✅ Caching Best Practices

📦 What to Cache
  • Read-heavy data (80%+ reads)
  • Expensive to compute queries
  • Reference data (country lists, config)
  • Session data
  • API responses (idempotent GETs)
🚫 What NOT to Cache
  • Write-heavy data
  • Highly sensitive data (PII, passwords)
  • Huge objects (>1MB)
  • Data that changes every second
  • Unique data per user (unless necessary)
⏰ TTL Guidelines
  • Static config: 24 hours+
  • Product catalog: 1-12 hours
  • User profile: 5-30 minutes
  • Stock/price: 1-60 seconds
  • High volatility: 1-5 seconds
  • Add random jitter (10-20%)
🔑 Key Design
  • Use consistent namespace (e.g., "user:123")
  • Include version in key for breaking changes
  • Use short keys (but readable)
  • Consider key hashing for large values

✈️ Real-World Example: E-Commerce Product Cache

Complete Caching Strategy for Product Page

1. Request product page for "product_123"
2. Check CDN cache (static assets, product HTML)
3. Check Redis for product data (price, stock, description)
4. If Redis miss → Load from PostgreSQL → Store in Redis (TTL: 5 minutes)
5. Cache product HTML in CDN (TTL: 1 hour)
6. When price changes → Publish event → Invalidate Redis + CDN

Result: 10,000 req/sec → Only 100 DB queries/sec (99% cache hit rate)

💡 Interview Power Phrases

"Cache Aside is our default pattern. We use TTL for expiration with random jitter to prevent cache stampede."
"We use multi-level caching: browser cache, CDN, Redis, and local cache (Caffeine). Each layer handles different latency requirements."
"Cache invalidation is the hardest problem. We use event-based invalidation with TTL as fallback, and versioned keys for breaking changes."
"For write-heavy data, we use write-through or write-behind patterns to keep cache consistent without blocking writes."

📚 Quick Memory Reference

🔑 Key Terms
  • Cache Aside - Load on miss (most common)
  • Write-Through - Cache + DB sync
  • Write-Behind - Cache first, DB async
  • TTL - Time To Live (auto-expiry)
  • Cache Stampede - Many requests on cache miss
  • LRU/LFU - Eviction policies
🧠 Mental Models
  • Cache = Fast memory desk (notes you keep nearby)
  • Database = Slow library (full source of truth)
  • TTL = Post-it note that fades over time
  • Cache Stampede = 1000 people rushing to library at once
  • Invalidation = Throwing away old notes when info changes
📌 One-Line Summary

"Cache stores hot data closer to the user — dramatically faster at the cost of eventual consistency and invalidation complexity."

✅ Key Takeaways (Remember These 5 Points)

  • Cache Aside is most common - Load on miss, simple, works for most cases
  • Invalidation is hard - Use TTL + event-based + version keys
  • Cache Stampede - Use locking or random TTL jitter to prevent
  • Multi-level caching - Browser → CDN → Redis → Local → Database
  • Choose pattern based on read/write ratio - Cache Aside for read-heavy, Write-Through for consistency
🎯 Remember This:
Without Cache → 50ms response, 1000 DB queries/sec → Database bottleneck
With Cache → 1ms response, 10 DB queries/sec → 100x faster, 99% less DB load