Part 11: Deployment Models - Zero Downtime Strategies - Complete Guide

Part 11: Deployment Models

Zero Downtime Deployment Strategies for Microservices - Complete Guide

📌 Quick Summary (30 seconds):
Choose deployment model based on risk tolerance and downtime requirements: Rolling (simple, brief downtime), Blue-Green (zero downtime, higher cost), Canary (gradual rollout, lowest risk), Active-Active (highest availability, highest complexity). Blue-Green and Canary are most common for production microservices.

❓ The Problem: Deploying Without Downtime

Traditional Deployment Problems:

Taking down old version = User sees errors
Long maintenance windows = Business impact
Big bang releases = High risk of failure
No rollback strategy = Extended outages
Can't test in production = Unknown behavior

Deployment Strategies Spectrum

Low Risk / High Complexity ←—————————→ High Risk / Low Complexity

[Canary] → [Blue-Green] → [Rolling] → [Recreate]
↓ ↓ ↓ ↓
Very Low Low Medium High Risk
High Cost Med Cost Low Cost Low Cost
Zero Downtime Zero Downtime Some Downtime Full Downtime

📊 Deployment Models Complete Comparison

Model	Downtime	Risk	Cost	Rollback Speed	Complexity	Best For
Recreate (Traditional)	Full downtime	High	Low	Slow (redeploy old version)	Very Low	Dev environments, maintenance windows
Rolling Update	Some (brief per instance)	Medium	Low	Slow (progressive rollback)	Low	Standard deployments, non-critical apps
Blue-Green	None	Low	High (2x resources)	Fast (switch router)	Medium	Critical production apps
Canary	None	Very Low	Medium-High	Fast (reduce % to 0)	High	Large-scale systems, user validation
A/B Testing	None	Low (feature-specific)	Medium	Fast	Medium	Testing features with user segments
Shadow/Replay	None	Lowest (no user impact)	Medium	N/A (traffic replay only)	High	Testing with real traffic without user impact
Active-Active	None	Low	Very High (2 regions)	Fast (DNS failover)	High	Global high-scale, multi-region

🔄 1. Recreate (Traditional)

Recreate Flow

Old Version (Running) → Take Down Entirely → Deploy New Version → Start New Version

[===== DOWNTIME =====]

Pros: Simple, no traffic management
Cons: Full downtime, slow rollback

🔄 2. Rolling Update

Rolling Update Flow (4 instances)

Start: [V1] [V1] [V1] [V1] (All running old version)
Step 1: [V1] [V1] [V1] [V2] (Update instance 4)
Step 2: [V1] [V1] [V2] [V2] (Update instance 3)
Step 3: [V1] [V2] [V2] [V2] (Update instance 2)
Step 4: [V2] [V2] [V2] [V2] (Update instance 1)

Downtime: Each instance has brief interruption during restart
Risk: Mixed versions running simultaneously

✅ PROS

No extra infrastructure needed
Gradual rollout
Works with auto-scaling
Low cost (no extra instances)

❌ CONS

Brief downtime per instance
Mixed versions during update
Slow rollback (must re-deploy)
Version compatibility issues

🔵 3. Blue-Green Deployment (Most Popular)

Blue-Green Flow

┌─────────────────────────────────────────────────────────────────┐
│ │
│ BLUE Environment (Live) GREEN Environment (New) │
│ ┌─────────┐ ┌─────────┐ │
│ │ v1.0 │ │ v2.0 │ │
│ │ v1.0 │ │ v2.0 │ │
│ │ v1.0 │ │ v2.0 │ │
│ └────┬────┘ └────┬────┘ │
│ │ │ │
│ └──────────┬─────────────────┘ │
│ │ │
│ Load Balancer │
│ │ │
│ ▼ │
│ Users │
│ │
│ Step 1: Deploy v2.0 to GREEN (while BLUE serves traffic) │
│ Step 2: Test GREEN environment │
│ Step 3: Switch Load Balancer → BLUE to GREEN │
│ Step 4: Zero downtime! │
│ Step 5: Keep BLUE for rollback (10 min - 1 hour) │
└─────────────────────────────────────────────────────────────────┘

# Kubernetes Blue-Green via Service selector
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    version: blue   # Currently pointing to blue (v1)
  ports:
    - port: 80

# Switch to green (v2)
kubectl patch service myapp-service -p '{"spec":{"selector":{"version":"green"}}}'

# Rollback
kubectl patch service myapp-service -p '{"spec":{"selector":{"version":"blue"}}}'

✅ PROS

Zero downtime
Instant rollback (flip traffic back)
Full testing in production environment
No version mixing
Clear success/failure boundary

❌ CONS

2x infrastructure cost
Database migration complexity
Longer deployment time
Environment drift issues

🐦 4. Canary Deployment (Lowest Risk)

Canary Release Flow

Phase 1: 5% of users → New Version (Canaries)
Phase 2: Monitor metrics for 1 hour (errors, latency)
Phase 3: If healthy → Increase to 20%
Phase 4: Monitor again → Increase to 50%
Phase 5: Monitor → Increase to 100%
Phase 6: Old version decommissioned

If errors detected at ANY phase → Immediate rollback (reduce to 0%)

# Istio VirtualService for Canary Deployment
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: myapp-canary
spec:
  hosts:
  - myapp
  http:
  - match:
    - headers:
        x-canary:
          exact: "true"   # Specific users
    route:
    - destination:
        host: myapp
        subset: v2
      weight: 100
  - route:
    - destination:
        host: myapp
        subset: v1
      weight: 95
    - destination:
        host: myapp
        subset: v2
      weight: 5      # 5% canary traffic

✅ PROS

Lowest risk (small blast radius)
Real user validation
Performance testing with live traffic
Gradual, controlled rollout

❌ CONS

Complex traffic routing
Requires advanced infrastructure (Istio, Spinnaker)
Long rollout time
Version compatibility challenges

🧪 5. A/B Testing Deployment

A/B Testing Flow

User Category A (50%) → Feature Flag ON → New Experience
User Category B (50%) → Feature Flag OFF → Old Experience

Compare metrics: conversion rate, engagement, error rate
Winner: Roll out to 100%

🎭 6. Shadow Deployment (Safest Testing)

Shadow/Replay Flow

Live Traffic → Splitter → 95% → Old Version → Response to User
↓
5% → New Version → Response LOGGED (Not to user)

Compare responses for correctness without impacting users!
Perfect for critical systems like payments, fraud detection.

🌍 7. Active-Active (Multi-Region)

Active-Active Multi-Region Deployment

US-East Region ──┐ ┌── Europe Region
[v2.0] [v2.0] │ Load Balancer │ [v2.0] [v2.0]
├──────── DNS ───────┤
Asia Region ─────┘ (Geo-routing) └── Australia Region
[v2.0] [v2.0] [v2.0] [v2.0]

Benefits: Global low latency, disaster recovery, zero downtime

🎯 Deployment Strategy Decision Matrix

If you need...	Choose...	Why
Simple, low-cost deployment	Rolling Update	No extra infrastructure, works out of box
Zero downtime for critical apps	Blue-Green	Instant switch, instant rollback
Lowest risk for large user base	Canary	Small blast radius, real user validation
Test features with specific user segments	A/B Testing	Compare metrics, data-driven decisions
Test without user impact	Shadow/Replay	No user sees new version
Global scale and disaster recovery	Active-Active	Multi-region, automatic failover

⚠️ The Database Problem in Blue-Green Deployment

Challenge: How to handle database schema changes with Blue-Green?

Database Migration Strategies for Blue-Green

Strategy 1: Database per Environment
Blue DB (v1 schema) ←─── Replication ───→ Green DB (v2 schema)
(Switch app, keep both DBs during transition)

Strategy 2: Backward-Compatible Changes Only
- Add columns (not remove)
- Rename with migration period
- Deprecate old fields gradually

Strategy 3: Branch by Abstraction
- Code works with both schemas
- Deploy code first, then migrate data
- Remove old code after migration

⏪ Rollback Strategies

Deployment Type	Rollback Method	Time to Rollback
Rolling Update	Redeploy previous version	5-15 minutes
Blue-Green	Flip traffic back to BLUE environment	Seconds
Canary	Reduce traffic to new version to 0%	Seconds
Active-Active	DNS failover to healthy region	Minutes

☸️ Kubernetes Native Deployment Strategies

Kubernetes Strategy	Command	Behavior
RollingUpdate (default)	kubectl set image deployment/myapp myapp=v2	Gradually replaces pods
Recreate	kubectl patch deployment myapp -p '{"spec":{"strategy":{"type":"Recreate"}}}'	All pods down, then new pods up
Blue-Green	kubectl patch service myapp -p '{"spec":{"selector":{"version":"green"}}}'	Switch service selector
Canary via Istio	kubectl apply -f virtualservice-canary.yaml	Weighted traffic routing

✅ Production Deployment Checklist

Before Deployment

Unit tests passing
Integration tests passing
Database migrations tested
Rollback plan documented
Monitoring dashboards ready
Feature flags configured

During Deployment

Monitor error rates
Watch latency percentiles
Check resource usage
Verify business metrics
Set alert thresholds

After Deployment

Smoke tests on new version
Validate logs for errors
Keep old version ready (Blue-Green)
Document deployment
Notify stakeholders

💡 Interview Power Phrases

"We use Blue-Green deployment for critical services where downtime is unacceptable — instant rollback with just a traffic flip."

"For large-scale systems, we use Canary deployment. We start with 5% of users, monitor for an hour, then gradually increase to 100%."

"The database is the hardest part of Blue-Green. We design backward-compatible schema changes and use database replication between environments."

"Our rollback strategy is tested before every deployment. For Blue-Green, rollback is instant; for rolling, we have automated redeployment."

📚 Quick Memory Reference

🔑 Key Terms

Blue-Green - Two identical environments
Canary - Gradual % rollout
Rolling - One instance at a time
Shadow - No user impact testing
A/B Test - Feature-specific testing
Active-Active - Multi-region

🧠 Mental Models

Blue-Green = Two runways, switch landing at once
Canary = Send a few birds into coal mine first
Rolling = Change tires while car is moving
Shadow = Practice surgery on dummy first

📌 One-Line Summary

"Blue-Green for zero downtime and instant rollback, Canary for gradual risk reduction, Rolling for simplicity and lower cost."

📚 Complete Reference: All Microservices Patterns

One-Line Summaries - Quick Revision:

• Sync Communication: Immediate response, tight coupling, user-facing ops
• Async Communication: Event-driven, loose coupling, scalable processing
• Saga Pattern: Distributed transactions via compensating actions
• CQRS: Separate read/write models with intentional duplication
• Circuit Breaker: Stop calling failing services, prevent cascading failures
• Retry: Handle transient failures with exponential backoff
• Timeout: Never wait indefinitely for any remote call
• Idempotency: Same operation multiple times = same result
• Rate Limiter: Control request flow, return 429 when exceeded
• Cache Aside: Load on miss, most common caching pattern
• Blue-Green: Two environments, instant switch, zero downtime
• Canary: Gradual rollout, lowest risk
• Rolling Update: One instance at a time, simple

✅ Key Takeaways (Remember These 5 Points)

Blue-Green = Zero downtime + instant rollback (2x cost)
Canary = Lowest risk (gradual, real user validation)
Rolling Update = Simple, low cost (brief downtime)
Database is hardest part = Plan backward-compatible changes
Test rollback before deployment = Deployment isn't done until rollback works

🎯 Remember This:
Low Risk Tolerance → Canary or Blue-Green
High Risk Tolerance → Rolling or Recreate
Critical System → Blue-Green + Canary + Active-Active

DeploymentModels

Part 11: Deployment Models

❓ The Problem: Deploying Without Downtime

📊 Deployment Models Complete Comparison

🔄 1. Recreate (Traditional)

🔄 2. Rolling Update

🔵 3. Blue-Green Deployment (Most Popular)

🐦 4. Canary Deployment (Lowest Risk)

🧪 5. A/B Testing Deployment

🎭 6. Shadow Deployment (Safest Testing)

🌍 7. Active-Active (Multi-Region)

🎯 Deployment Strategy Decision Matrix

⚠️ The Database Problem in Blue-Green Deployment

⏪ Rollback Strategies

☸️ Kubernetes Native Deployment Strategies

✅ Production Deployment Checklist

💡 Interview Power Phrases

📚 Quick Memory Reference

📚 Complete Reference: All Microservices Patterns

✅ Key Takeaways (Remember These 5 Points)