2) Performance math & sizing — detailed guide

• Start with 3 anchors: peak QPS, payload size (p50/p95), and latency SLO (p95/p99).
  Most sizing mistakes come from using averages. Use peak with a multiplier, and speak in percentiles (p95/p99).
• Convert users → actions → QPS: DAU × actions/day ÷ 86,400 → avg; then apply peak multiplier.
  If you’re missing numbers, make reasonable assumptions and say them out loud. The reasoning matters more than precision.
• Use Little’s Law: concurrency = RPS × latency (seconds) to size pools and in-flight limits.
  This is the fastest way to estimate how many requests are in flight and whether pools/queues are safe at peak.
• Turn cache hit into DB load: origin QPS = QPS × (1 − hit).
  A 90% cache hit reduces DB by 10×. This single line often justifies introducing Redis/Memcached.
• Always include headroom: size for peak × (1.3–2.0) and keep utilization under the knee.
  As utilization approaches 1, queueing delay explodes. Keep ρ ≤ ~0.6–0.7 for latency-sensitive paths.

• DAU: 10M; feed opens/user/day: 10 → 100M feed reads/day
  Avg ≈ 1,157 RPS; peak (×8) ≈ 9,259 RPS.
• SLO: GET /feed p95 ≤ 250ms (server-side)
  We will budget latency hop-by-hop and keep retries/hedging within this envelope.
• Payload: 25 items/page, ~800B/item → ~20KB/page (metadata only)
  Media bytes are served via CDN/object store; the feed API returns metadata + URLs.
• Cache hit target: 90% for the feed head
  DB sees ~926 QPS at peak on misses.

• Define SLO in percentiles: p95/p99 latency target + error/availability target for the critical endpoint.
  Percentiles matter because tail latency drives user pain. Pick the handful of endpoints that matter most.
• Budget hop-by-hop: LB/edge → auth → service → cache → DB → serdes/network.
  Always keep a buffer for tail risk. If you spend your entire budget on ‘expected’ work, p95 will miss.
• Tail amplifies with fan-out: overall latency trends toward the slowest subcall.
  This is why we batch and parallelize carefully, and why we cap fan-out in the read path.
• Tail mitigation: per-hop timeouts, bounded retries + jitter, hedged reads for idempotent reads, bulkheads.
  Retries can worsen tail if unbounded. Hedging is a controlled way to cut p99 when replicas are independent.

• SLO: GET /feed p95 ≤ 250ms (server)
  We keep the read path fast using cache hits; slow dependencies trigger fallback to last-good cached pages.
• Budget split: Edge/LB 10ms + Auth 15ms + FeedReadSvc 15ms + Redis 10ms + DB 50ms + serdes/net 25ms + buffer 125ms = 250ms
  If DB p95 rises, the buffer gets eaten first. That’s your early signal to scale cache, reduce misses, or shard.

“We’ll target p95 250ms for feed reads. I’ll budget it across hops and keep ~50% as buffer for tail risk, so we can survive spikes without missing SLO.”

• Little’s Law: L = λ × W — in-flight concurrency = RPS × avg time in system.
  Use this to size thread pools, connection pools, and max in-flight requests at peak.
• Utilization: ρ = λ / μ. As ρ → 1, queueing delay grows superlinearly.
  You don’t need exact queueing math—just explain the knee and why headroom is mandatory.
• Rule of thumb: keep ρ ≤ 0.6–0.7 at peak for latency-sensitive services.
  Higher utilization may be fine for offline/batch; it’s dangerous for interactive APIs like feeds.

• Concurrency: peak λ≈9,259 RPS and W≈120ms (0.12s) → L≈ 1,111 in-flight
  That’s the number that should guide max in-flight limits and pool sizing. If your pools are smaller, you’ll see queueing and p95 spikes.
• Knee intuition: If a DB node sustains μ=2000 QPS at S=50ms, then ρ=0.6 is healthy; ρ=0.85 produces big tail latency.
  This is why feed systems keep DB protected with caches and avoid saturating it during traffic bursts.

• Traffic sizing: DAU/MAU × actions/user → avg RPS; apply peak multiplier (×4–×10).
  Use peak to select storage/caches; use average mainly for cost intuition.
• Fan-out math: backend QPS = front-door QPS × fan-out × (1 − cache_hit) per layer.
  Identify where fan-out happens (sync read path vs async pipelines) and ensure the read path stays bounded.
• Payload sizing: use p50/p95 sizes; use pagination, compression, and CDNs.
  Separate metadata payload from media bytes. Many feed systems return IDs + minimal metadata and hydrate in batch.
• Cache hit target H: origin QPS = QPS × (1 − H); prevent stampede via coalescing + jittered TTL.
  Cache hit drives DB cost and latency. Mention stale-while-revalidate and request coalescing for hot keys.

• Peak feed reads: ~9,259 RPS
  Derived from DAU and opens/day. Peak multiplier accounts for diurnal + bursts.
• Cache hit: 90% → DB sees ~926 QPS
  This single line shows why Redis matters: 90% hit makes DB 10× smaller and keeps tails under control.
• Payload: ~20KB per page (metadata only) → peak metadata egress ~177 MB/s
  Real egress is dominated by media bytes, so use CDN for images/videos and keep API responses small and cacheable where possible.
• Hydration: batch-get posts by postIds to avoid N+1 calls
  Even if each page has 25 items, don’t do 25 RPCs. Do 1–2 batch calls and cache hydrated post summaries.

• Headroom policy: size for peak × 1.3–2.0; keep ρ ≤ 60–70% at peak.
  Headroom covers failure (AZ loss), deployments, and randomness in tail latency.
• Bursts: queues between tiers; autoscale on lag/latency; shed non-critical work; early 429/503 with Retry-After.
  Prefer fast failures over slow timeouts. Protect DB first with pool caps, backpressure, and circuit breakers.
• Diurnal: run batch/compaction off-peak; plan for event spikes.
  If the workload has a daily peak, schedule heavy background jobs away from it.

• Provisioned target: peak × 1.5 = 13,889 RPS
  This includes headroom for spikes and partial failure.
• If one API instance sustains: 400 RPS at ~60% utilization → need ~35 instances
  Then add N+1/AZ and room for deploys. Exact numbers depend on CPU cost of ranking/hydration.
• Burst plan: edge token buckets + brownout mode (fall back to cached/recency feed), autoscale workers on queue lag.
  If ranking pipeline is unhealthy, degrade to recency to keep availability and latency SLOs.

• Compute: instances = count × $/hr × 730 (include spare/LB/NAT). Serverless = requests + duration.
  Give relative drivers rather than exact dollars. Mention you’ll validate with real pricing later.
• Egress: GB egressed × $/GB; CDN can reduce origin egress dramatically.
  For feeds, media egress usually dominates. CDN caching and compression are the big levers.
• Storage: GB-month across hot/warm/cold tiers; include IO/requests; lifecycle old data.
  Feeds and posts grow fast. Use TTL/lifecycle and keep indexes lean.
• Ops/observability: logs/metrics/traces (GB/day), retention policies, sampling.
  Sampling and retention are cost controls. Also avoid logging PII and large payloads.

• Biggest costs: CDN/media egress, cache tier, and DB reads (if cache hit is low).
  Keeping cache hit high reduces both DB cost and tail latency.
• Largest savings: smaller payloads, better cache hit, aggressive CDN for media, and log sampling.
  Return metadata + URLs (not media). Compress responses. Use lifecycle policies for older content and logs.

Compute ≈ instance_count × $/hr × 730
Storage ≈ GB × $/GB-month (+ IO/requests)
Egress ≈ GB_out × $/GB (CDN reduces origin)
Observability ≈ GB/day × $/GB (retention + sampling matters)

• Little’s Law: L = λW (concurrency = RPS × latency)
  Use for pools, in-flight, worker counts.
• Utilization: ρ = λ/μ; keep ≤ 0.6–0.7 at peak
  Avoid the knee where queues explode and tails spike.
• M/M/1 intuition: R ≈ S / (1 − ρ)
  Not exact in real systems, but great for explaining why headroom matters.
• Cache: QPS_origin = QPS × (1 − H)
  Hit rate is the biggest lever for DB load and tail latency.
• Peak planning: peak = avg × multiplier (×4–×10)
  Use peak for sizing, avg for cost intuition.
• Retry budget: retries must fit inside SLO
  Use bounded retries + jitter; idempotency keys for writes.

avg_RPS = actions_per_day / 86,400
peak_RPS = avg_RPS × peak_multiplier
concurrency ≈ peak_RPS × latency_seconds
origin_QPS = peak_RPS × (1 − cache_hit)
keep utilization below ~70% at peak (tails explode near saturation)