Part 3: Resilience Patterns - Timeout, Retry, Circuit Breaker, Fallback - Complete Guide

Part 3: Resilience Patterns

The Complete Resilience Stack - Timeout, Retry, Circuit Breaker, Fallback

📌 Quick Summary:
The resilience stack has 4 layers applied in order: Timeout (don't wait forever) → Retry (try again if temporary) → Circuit Breaker (stop calling failing services) → Fallback (provide alternative response). Each layer protects against specific failure scenarios.

The Resilience Stack (Order of Protection)

🔷 Level 1: TIMEOUT - "Don't wait forever"
⬇
🔷 Level 2: RETRY - "Try again if temporary"
⬇
🔷 Level 3: CIRCUIT BREAKER - "Stop calling failing service"
⬇
🔷 Level 4: FALLBACK - "Provide alternative response"

1. Timeout Pattern

What is Timeout?

A timeout limits the maximum time a client will wait for a response from a service before giving up.

📖 Real-World Example: You call a restaurant for delivery. If they don't answer after 30 seconds, you hang up and try another restaurant. You don't wait forever.

Types of Timeouts

Type	Description	Example
Connection Timeout	Time to establish connection	3 seconds to connect to server
Read Timeout	Time to wait for response after connection	5 seconds to read response
Request Timeout	Total time for entire operation	10 seconds max for whole call

Code Example

// Java Spring Boot - Setting Timeouts
@Configuration
public class TimeoutConfig {
    
    @Bean
    public RestTemplate restTemplate() {
        return new RestTemplateBuilder()
            .setConnectTimeout(Duration.ofSeconds(3))   // Connection timeout
            .setReadTimeout(Duration.ofSeconds(5))      // Read timeout
            .build();
    }
}

// Using WebClient with timeouts
webClient.get()
    .uri("/api/payment")
    .timeout(Duration.ofSeconds(5))
    .retrieve()
    .bodyToMono(PaymentResponse.class)
    .onErrorResume(TimeoutException.class, e -> {
        log.error("Payment service timeout!");
        return Mono.just(fallbackResponse());
    });

Best Practices

Every remote call MUST have a timeout - no exceptions
Set timeouts based on Service Level Agreements (SLAs)
Use different timeouts for different operations (fast vs slow operations)
Monitor timeout occurrences - they indicate problems

2. Retry Pattern

What is Retry?

Automatically retry a failed operation if the failure is likely temporary (network blip, momentary overload).

📖 Real-World Example: You try to call a friend. The line is busy. You wait 2 seconds, try again. Still busy. Wait 4 seconds, try again. Connected!

Retry Algorithms

A. Fixed Delay Retry

Attempt 1: Fail → Wait 1 second
Attempt 2: Fail → Wait 1 second
Attempt 3: Success ✓

B. Exponential Backoff (Recommended)

Attempt 1: Fail → Wait 100ms
Attempt 2: Fail → Wait 200ms (2x)
Attempt 3: Fail → Wait 400ms (4x)
Attempt 4: Fail → Wait 800ms (8x)
Attempt 5: Success ✓

Formula: wait = base * (2 ^ attempt) + jitter

C. Exponential Backoff with Jitter (Best)

Add random variation to prevent "thundering herd" problem

Attempt 1: 100ms + random(0-50ms)
Attempt 2: 200ms + random(0-50ms)
Attempt 3: 400ms + random(0-50ms)

Result: 100 services retrying at slightly different times

When to Retry vs When NOT to Retry

Scenario	Retry?	Why
Network timeout	✅ YES	Temporary network issue
HTTP 503 Service Unavailable	✅ YES	Service temporarily overloaded
HTTP 500 Internal Error	✅ MAYBE	Could be transient or permanent
HTTP 400 Bad Request	❌ NO	Client error - retry won't fix
HTTP 401 Unauthorized	❌ NO	Auth issue - retry won't fix
HTTP 404 Not Found	❌ NO	Resource doesn't exist
Non-idempotent operation (POST)	⚠️ CAREFUL	Could create duplicate data

⚠️ Important: Only retry IDEMPOTENT operations (GET, PUT, DELETE) or use idempotency keys for POST requests. Otherwise, you might create duplicate orders or payments!

3. Circuit Breaker Pattern

What is Circuit Breaker?

A circuit breaker monitors failures and "trips" (opens) when failures exceed a threshold, preventing further calls to a failing service. After a timeout, it allows test calls to check recovery.

📖 Real-World Example: Your home circuit breaker trips when there's an electrical fault. You don't keep flipping the switch - you fix the problem first. The circuit breaker protects your house from electrical fires.

Three States of Circuit Breaker

┌─────────────────────────────────────────────────────────┐
│                                                         │
│    ┌─────────┐     Success      ┌─────────┐           │
│    │         │ ───────────────► │         │           │
│    │  CLOSED │                   │  OPEN   │           │
│    │ (Normal)│ ◄─────────────── │  (Failing)│          │
│    └────┬────┘    5 failures    └────┬────┘           │
│         │                            │                 │
│         │                            │ Timeout passes  │
│         │                            ▼                 │
│         │                      ┌─────────┐            │
│         │                      │ HALF-   │            │
│         └─────────────────────►│  OPEN   │            │
│              Success            │ (Test)  │            │
│                                └─────────┘            │
│                                    │                   │
│                                    │ Failure           │
│                                    ▼                   │
│                                ┌─────────┐            │
│                                │  OPEN   │            │
│                                └─────────┘            │
└─────────────────────────────────────────────────────────┘

State Explanations

State	Meaning	Behavior
CLOSED	Normal operation	Calls go through. Failures are counted. When failure threshold reached → OPEN
OPEN	Service is failing	Calls fail immediately without attempting. After timeout period → HALF-OPEN
HALF-OPEN	Testing recovery	Limited calls allowed. Success → CLOSED. Failure → OPEN

Code Example (Resilience4j)

// Configure Circuit Breaker
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .failureRateThreshold(50)                    // 50% failures opens circuit
    .slidingWindowSize(10)                       // Check last 10 calls
    .waitDurationInOpenState(Duration.ofSeconds(30))  // Stay open for 30 sec
    .permittedNumberOfCallsInHalfOpenState(3)    // Allow 3 test calls
    .recordException(e -> e instanceof TimeoutException)
    .build();

CircuitBreaker breaker = CircuitBreaker.of("payment-service", config);

// Use Circuit Breaker
Supplier<Payment> decoratedSupplier = CircuitBreaker
    .decorateSupplier(breaker, () -> paymentClient.process(request));

CompletableFuture.supplyAsync(decoratedSupplier)
    .thenAccept(payment -> handleSuccess(payment))
    .exceptionally(e -> handleFailure(e));

Real-World Configuration Example

payment-service:
  circuit-breaker:
    failure-threshold: 5 failures in 10 seconds
    timeout: 30 seconds (how long to stay OPEN)
    half-open-calls: 3 (test calls to check recovery)
    
Order of events:
1. Payment service fails 5 times in 10 seconds
2. Circuit OPENS for 30 seconds (no calls get through)
3. After 30 sec, circuit goes HALF-OPEN
4. 3 test calls allowed
5. If test succeeds → CLOSED (normal)
6. If test fails → OPEN again (wait another 30 sec)

4. Fallback Pattern

What is Fallback?

When a primary operation fails, provide an alternative response or action instead of showing an error to the user.

📖 Real-World Example: Your favorite restaurant is closed. Instead of starving, you order from your second favorite restaurant (fallback). Or you eat leftovers from the fridge (cached data fallback).

Types of Fallbacks

A. Static Fallback (Default Response)

If payment service fails:
    Return: {"status": "PENDING", "message": "Payment being processed"}
    
User doesn't see an error, just knows it's in progress.

B. Cached Fallback (Stale Data)

If product catalog service fails:
    Return last cached product data (even if slightly stale)
    
User still sees products, just not the absolute latest prices.

C. Secondary Provider Fallback

If Primary Payment Gateway fails:
    Switch to Backup Payment Gateway
    
User doesn't even know one provider failed.

D. Degraded Functionality Fallback

If Recommendation Service fails:
    Show default/top products instead of personalized recommendations
    
Core functionality (buying) works. Nice-to-have features degrade.

E. Async Retry with Queue

If Email Service fails:
    Add to retry queue instead of failing immediately
    Process later when service recovers
    
User gets email eventually, not instantly.

Code Example (Spring Cloud Circuit Breaker)

@Service
public class PaymentService {
    
    @CircuitBreaker(name = "paymentService", fallbackMethod = "paymentFallback")
    public PaymentResponse processPayment(PaymentRequest request) {
        return paymentClient.process(request);
    }
    
    // Fallback method - same signature + Throwable parameter
    public PaymentResponse paymentFallback(PaymentRequest request, Throwable t) {
        log.warn("Payment failed, using fallback. Error: {}", t.getMessage());
        
        // Option 1: Default response
        return new PaymentResponse("PENDING", "Payment queued");
        
        // Option 2: Return cached data
        // return cache.get(request.getOrderId());
        
        // Option 3: Add to queue for later
        // retryQueue.add(request);
        // return new PaymentResponse("QUEUED", "Will retry");
    }
}

Putting It All Together: Complete Payment Flow Example

┌─────────────────────────────────────────────────────────────────────┐
│                    COMPLETE RESILIENCE FLOW                          │
│                                                                      │
│  User clicks "Pay Now"                                               │
│         │                                                            │
│         ▼                                                            │
│  ┌─────────────────┐                                                │
│  │ 1. TIMEOUT      │  ← 2 seconds max wait                          │
│  │    (2 seconds)  │                                                │
│  └────────┬────────┘                                                │
│           │ (Timeout occurs)                                        │
│           ▼                                                          │
│  ┌─────────────────┐                                                │
│  │ 2. RETRY        │  ← Exponential backoff                         │
│  │    (Max 3 times)│    100ms → 200ms → 400ms                       │
│  └────────┬────────┘                                                │
│           │ (All retries fail)                                      │
│           ▼                                                          │
│  ┌─────────────────┐                                                │
│  │ 3. CIRCUIT      │  ← After 5 failures in 10 seconds              │
│  │    BREAKER      │    Circuit OPENS for 30 seconds                │
│  └────────┬────────┘                                                │
│           │                                                          │
│           ▼                                                          │
│  ┌─────────────────┐                                                │
│  │ 4. FALLBACK     │  ← Return "Payment Pending"                    │
│  │                 │    Add to retry queue                          │
│  └─────────────────┘                                                │
│                                                                      │
│  User sees: "Your payment is being processed. We'll notify you."    │
│  (No error message!)                                                │
└─────────────────────────────────────────────────────────────────────┘

Quick Comparison: All Resilience Patterns

Pattern	What it does	When it triggers	What user sees	Real-World Analogy
Timeout	Limits wait time	Call takes too long	Fast failure, not hanging	Hanging up after 30 seconds
Retry	Re-attempts failed call	Temporary failure (network, 5xx)	May see slight delay	Redialing a busy number
Circuit Breaker	Stops calling failing service	Repeated failures (5+ in 10 sec)	Fast failure, no waiting	Tripped electrical breaker
Fallback	Provides alternative response	All else fails	Graceful degradation, not error	Ordering from backup restaurant

💡 Interview Power Phrases

"The resilience stack is timeout first, then retry, then circuit breaker, finally fallback. Each layer protects against a different type of failure."

"Timeouts prevent thread exhaustion. Without them, a single slow service can bring down your entire system."

"Exponential backoff with jitter prevents the thundering herd problem when retrying."

"Circuit breakers are not optional in production microservices. They prevent cascading failures."

"Fallbacks should always provide a degraded but functional experience, never just an error message."

📌 Key Takeaways

Timeout - EVERY remote call needs one. No exceptions.
Retry - Use exponential backoff + jitter. Only retry idempotent operations.
Circuit Breaker - Three states: CLOSED → OPEN → HALF-OPEN. Prevents cascading failures.
Fallback - Always provide graceful degradation. Never show raw errors to users.
Order matters - Apply in sequence: Timeout → Retry → Circuit Breaker → Fallback

Resilence