Part 10: Caching Strategies
Optimizing Performance with Caches - Complete Guide
📌 Quick Summary (30 seconds):
Caching stores frequently accessed data in fast memory (Redis, Memcached) to reduce database load and improve response times. Cache Aside (lazy loading) is the most common pattern. Key challenges: cache invalidation, consistency, and preventing the cache stampede problem. Use TTL with random jitter to avoid stampedes.
Caching stores frequently accessed data in fast memory (Redis, Memcached) to reduce database load and improve response times. Cache Aside (lazy loading) is the most common pattern. Key challenges: cache invalidation, consistency, and preventing the cache stampede problem. Use TTL with random jitter to avoid stampedes.
❓ The Problem: Database is Slow
Why We Need Caching:
- Database queries take 10-100ms (too slow for high traffic)
- Network round trips add latency
- Database scales poorly for repeated reads
- Cost of DB queries adds up ($$$ in cloud)
- Same data requested thousands of times
Without Cache vs With Cache
WITHOUT CACHE (Every request hits DB):
User Request → Database (50ms) → Response
1000 requests = 1000 database queries = 50,000ms = Database overload!
WITH CACHE (First request hits DB, rest from cache):
Request 1: User → Cache (miss) → Database (50ms) → Cache → Response
Requests 2-1000: User → Cache (HIT! ~1ms) → Response
1000 requests = 1 DB query + 999 cache hits = FAST!
WITHOUT CACHE (Every request hits DB):
User Request → Database (50ms) → Response
1000 requests = 1000 database queries = 50,000ms = Database overload!
WITH CACHE (First request hits DB, rest from cache):
Request 1: User → Cache (miss) → Database (50ms) → Cache → Response
Requests 2-1000: User → Cache (HIT! ~1ms) → Response
1000 requests = 1 DB query + 999 cache hits = FAST!
📊 Cache Read Patterns
1. Cache Aside (Lazy Loading) - MOST COMMON
Cache Aside Flow
┌─────────┐ 1. Read ┌─────────┐
│ Client │ ─────────────► │ Cache │
└─────────┘ └────┬────┘
▲ │
│ 2. Cache Miss
│ ▼
│ ┌─────────┐
│ 3. Read from │
│ 4. Return │ Database │
│ Data └──────┬──────┘
│ │
│ 5. Update Cache
│ │
└────────────────────────┘
┌─────────┐ 1. Read ┌─────────┐
│ Client │ ─────────────► │ Cache │
└─────────┘ └────┬────┘
▲ │
│ 2. Cache Miss
│ ▼
│ ┌─────────┐
│ 3. Read from │
│ 4. Return │ Database │
│ Data └──────┬──────┘
│ │
│ 5. Update Cache
│ │
└────────────────────────┘
// Cache Aside Pattern Implementation
public class CacheAsideService {
public User getUser(String userId) {
// Step 1: Check cache
User user = redis.get("user:" + userId);
if (user != null) {
return user; // Cache HIT - fast path
}
// Step 2: Cache MISS - load from database
user = database.findUser(userId);
// Step 3: Store in cache for future requests
redis.set("user:" + userId, user, Duration.ofMinutes(30));
return user;
}
}
✅ PROS
- Simple to implement
- Only caches requested data
- Works for any data source
- Redis/DB failures are isolated
❌ CONS
- Cache miss penalty (extra latency)
- Stale data possible
- Cache stampede risk
- Not good for write-heavy workloads
2. Read-Through
Read-Through Flow
Client → Cache (miss) → Cache loads from DB automatically → Cache returns to Client
(Cache library handles DB loading, client doesn't know)
Client → Cache (miss) → Cache loads from DB automatically → Cache returns to Client
(Cache library handles DB loading, client doesn't know)
✅ PROS: Client simplified, no DB code in client
❌ CONS: Less control, library specific
✍️ Cache Write Patterns
1. Write-Through
Write-Through Flow
Client → Write to Cache → Cache writes to DB (sync) → Response
(Cache and DB always consistent, but slower writes)
Client → Write to Cache → Cache writes to DB (sync) → Response
(Cache and DB always consistent, but slower writes)
2. Write-Behind (Write-Back)
Write-Behind Flow
Client → Write to Cache → Immediate Response
↓ (async, batching)
Write to DB (eventual)
Client → Write to Cache → Immediate Response
↓ (async, batching)
Write to DB (eventual)
3. Write-Around
Write-Around Flow
Client → Write directly to DB → Cache is NOT updated
(Cache updated only on read miss - avoids cache pollution for write-heavy data)
Client → Write directly to DB → Cache is NOT updated
(Cache updated only on read miss - avoids cache pollution for write-heavy data)
📋 Cache Pattern Comparison
| Pattern | Read Behavior | Write Behavior | Consistency | Best For |
|---|---|---|---|---|
| Cache Aside | Load on miss | Write to DB, invalidate cache | Eventual | Read-heavy, most common |
| Read-Through | Cache loads from DB on miss | Same as Cache Aside | Eventual | When client wants simplicity |
| Write-Through | Same as Cache Aside | Cache + DB sync | Strong | Write-heavy, strong consistency needed |
| Write-Behind | Same as Cache Aside | Cache first, DB async | Eventual | High write throughput, latency sensitive |
| Write-Around | Load on miss | Write only to DB | Eventual | Write-heavy, avoid cache pollution |
🗑️ Cache Invalidation Strategies
"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton
| Strategy | How It Works | Pros | Cons | Best For |
|---|---|---|---|---|
| TTL (Time To Live) | Data expires after fixed time | Simple, no coordination | Stale data until expiration | Product catalogs, configuration |
| Event-based | Update/delete cache on data change | Fresh data quickly | Requires event infrastructure | User profiles, real-time data |
| Manual | Explicit invalidation API | Full control | Human error, operational overhead | Admin-triggered updates |
| Version-based | Cache key includes version | No explicit invalidation needed | Key management complexity | APIs, versioned data |
🐘 Cache Stampede Problem & Solutions
Problem: When a popular cache key expires, thousands of concurrent requests hit the database simultaneously, potentially bringing it down.
Cache Stampede
Time: 10:00:00 - Cache expires for "popular_product_123"
Time: 10:00:00.001 - 10,000 requests arrive simultaneously
ALL 10,000 see cache miss → ALL hit database → DATABASE CRASH!
Solutions:
1. Locking/Mutex - Only 1 request fetches, others wait
2. Request Coalescing - Deduplicate in-flight requests
3. Random TTL Jitter - Stagger expiration times
4. Probabilistic Early Expiration - Refresh early before expiry
Time: 10:00:00 - Cache expires for "popular_product_123"
Time: 10:00:00.001 - 10,000 requests arrive simultaneously
ALL 10,000 see cache miss → ALL hit database → DATABASE CRASH!
Solutions:
1. Locking/Mutex - Only 1 request fetches, others wait
2. Request Coalescing - Deduplicate in-flight requests
3. Random TTL Jitter - Stagger expiration times
4. Probabilistic Early Expiration - Refresh early before expiry
// Solution 1: Locking (Only one request fetches)
public User getUserWithLock(String userId) {
String cacheKey = "user:" + userId;
User user = redis.get(cacheKey);
if (user != null) return user;
// Acquire distributed lock
if (redis.setnx("lock:" + cacheKey, "1", Duration.ofSeconds(5))) {
try {
// Only one thread gets here
user = database.findUser(userId);
redis.set(cacheKey, user, Duration.ofMinutes(30));
} finally {
redis.del("lock:" + cacheKey);
}
} else {
// Wait for first thread to populate cache
Thread.sleep(100);
user = redis.get(cacheKey);
}
return user;
}
// Solution 2: Random TTL Jitter
Duration ttl = Duration.ofMinutes(30).plus(Duration.ofMillis(random.nextInt(5000)));
redis.set(cacheKey, data, ttl); // 30 minutes ± 5 seconds
🗑️ Cache Eviction Policies (When Cache is Full)
📊 Cache Levels (Where to Cache)
Multi-Level Caching
Level 1: Browser Cache (HTTP Cache-Control)
↓ (miss)
Level 2: CDN (CloudFront, Cloudflare) - Static assets, API responses
↓ (miss)
Level 3: API Gateway Cache - Responses for identical requests
↓ (miss)
Level 4: Application Cache (Caffeine, Guava) - In-memory, local
↓ (miss)
Level 5: Distributed Cache (Redis, Memcached) - Shared across instances
↓ (miss)
Level 6: Database (PostgreSQL, MySQL) - Source of truth
Level 1: Browser Cache (HTTP Cache-Control)
↓ (miss)
Level 2: CDN (CloudFront, Cloudflare) - Static assets, API responses
↓ (miss)
Level 3: API Gateway Cache - Responses for identical requests
↓ (miss)
Level 4: Application Cache (Caffeine, Guava) - In-memory, local
↓ (miss)
Level 5: Distributed Cache (Redis, Memcached) - Shared across instances
↓ (miss)
Level 6: Database (PostgreSQL, MySQL) - Source of truth
🛠️ Popular Cache Technologies
| Technology | Type | Use Case | Key Feature |
|---|---|---|---|
| Redis | In-memory data store | Distributed cache, session store | Persistence, data structures, cluster mode |
| Memcached | In-memory cache | Simple key-value cache | Fast, simple, multi-threaded |
| Caffeine | Local cache (JVM) | Single-node, high-performance | Near-zero latency, async loading |
| Ehcache | Local/distributed | Hibernate L2 cache | JCache compliant, disk overflow |
| Hazelcast | In-memory data grid | Distributed computing + cache | Queries, events, security |
⚡ Pros & Cons of Caching
✅ BENEFITS
- 10-100x faster responses (1ms vs 50ms)
- Reduces database load (80%+ reduction)
- Lower cloud costs (fewer DB reads)
- Handles traffic spikes gracefully
- Enables global distribution (CDN)
- Protects database during failures
❌ CHALLENGES
- Cache invalidation is hard
- Stale data (inconsistency)
- Additional infrastructure to manage
- Memory costs (RAM is expensive)
- Cache stampede risk
- Cold start problem (empty cache)
✅ Caching Best Practices
📦 What to Cache
- Read-heavy data (80%+ reads)
- Expensive to compute queries
- Reference data (country lists, config)
- Session data
- API responses (idempotent GETs)
🚫 What NOT to Cache
- Write-heavy data
- Highly sensitive data (PII, passwords)
- Huge objects (>1MB)
- Data that changes every second
- Unique data per user (unless necessary)
⏰ TTL Guidelines
- Static config: 24 hours+
- Product catalog: 1-12 hours
- User profile: 5-30 minutes
- Stock/price: 1-60 seconds
- High volatility: 1-5 seconds
- Add random jitter (10-20%)
🔑 Key Design
- Use consistent namespace (e.g., "user:123")
- Include version in key for breaking changes
- Use short keys (but readable)
- Consider key hashing for large values
✈️ Real-World Example: E-Commerce Product Cache
Complete Caching Strategy for Product Page
1. Request product page for "product_123"
2. Check CDN cache (static assets, product HTML)
3. Check Redis for product data (price, stock, description)
4. If Redis miss → Load from PostgreSQL → Store in Redis (TTL: 5 minutes)
5. Cache product HTML in CDN (TTL: 1 hour)
6. When price changes → Publish event → Invalidate Redis + CDN
Result: 10,000 req/sec → Only 100 DB queries/sec (99% cache hit rate)
1. Request product page for "product_123"
2. Check CDN cache (static assets, product HTML)
3. Check Redis for product data (price, stock, description)
4. If Redis miss → Load from PostgreSQL → Store in Redis (TTL: 5 minutes)
5. Cache product HTML in CDN (TTL: 1 hour)
6. When price changes → Publish event → Invalidate Redis + CDN
Result: 10,000 req/sec → Only 100 DB queries/sec (99% cache hit rate)
💡 Interview Power Phrases
"Cache Aside is our default pattern. We use TTL for expiration with random jitter to prevent cache stampede."
"We use multi-level caching: browser cache, CDN, Redis, and local cache (Caffeine). Each layer handles different latency requirements."
"Cache invalidation is the hardest problem. We use event-based invalidation with TTL as fallback, and versioned keys for breaking changes."
"For write-heavy data, we use write-through or write-behind patterns to keep cache consistent without blocking writes."
📚 Quick Memory Reference
🔑 Key Terms
- Cache Aside - Load on miss (most common)
- Write-Through - Cache + DB sync
- Write-Behind - Cache first, DB async
- TTL - Time To Live (auto-expiry)
- Cache Stampede - Many requests on cache miss
- LRU/LFU - Eviction policies
🧠 Mental Models
- Cache = Fast memory desk (notes you keep nearby)
- Database = Slow library (full source of truth)
- TTL = Post-it note that fades over time
- Cache Stampede = 1000 people rushing to library at once
- Invalidation = Throwing away old notes when info changes
📌 One-Line Summary
"Cache stores hot data closer to the user — dramatically faster at the cost of eventual consistency and invalidation complexity."
"Cache stores hot data closer to the user — dramatically faster at the cost of eventual consistency and invalidation complexity."
✅ Key Takeaways (Remember These 5 Points)
- Cache Aside is most common - Load on miss, simple, works for most cases
- Invalidation is hard - Use TTL + event-based + version keys
- Cache Stampede - Use locking or random TTL jitter to prevent
- Multi-level caching - Browser → CDN → Redis → Local → Database
- Choose pattern based on read/write ratio - Cache Aside for read-heavy, Write-Through for consistency
🎯 Remember This:
Without Cache → 50ms response, 1000 DB queries/sec → Database bottleneck
With Cache → 1ms response, 10 DB queries/sec → 100x faster, 99% less DB load
Without Cache → 50ms response, 1000 DB queries/sec → Database bottleneck
With Cache → 1ms response, 10 DB queries/sec → 100x faster, 99% less DB load