Proxies, load balancers and CDNs
Forward versus reverse proxies, layer 4 versus layer 7 balancing, health checks and sticky sessions, where TLS terminates, and how CDN caching rules actually behave.
Who the proxy is serving
| Kind | Serves | Typical use | Sees the request |
|---|---|---|---|
| Forward proxy | The client | Egress control, caching, anonymity | Full URL and headers |
| Reverse proxy | The server | TLS termination, routing, compression | Full request after decryption |
| L4 load balancer | Either | Raw TCP or UDP distribution | Only addresses and ports |
| L7 load balancer | The server | Routing on path and headers | Full HTTP request |
| API gateway | The server | Auth, rate limits, routing | Full request plus policy |
| CDN edge | The server | Static and cacheable content | Full request at the edge |
# headers that let the origin see the real client
X-Forwarded-For: 203.0.113.10, 198.51.100.7
X-Forwarded-Proto: https
X-Forwarded-Host: api.example.com
Forwarded: for=203.0.113.10;proto=https;host=api.example.com
# and the hop-by-hop headers a proxy must not forward
Connection: keep-alive
Transfer-Encoding: chunkedTrust only the forwarded headers that come from your own edge. A client can send X-Forwarded-For itself, so an origin that reads the leftmost value without validating the source is trivial to spoof.
Choosing a balancing strategy
- Round robin spreads evenly but ignores load and connection cost.
- Least connections handles uneven request durations better.
- Consistent hashing keeps a key on the same backend, which helps caches and stateful sessions.
- Least response time reacts to latency but can oscillate under changing load.
- Random with two choices is simple and close to optimal at large scale.
health check design
passive observe real traffic; eject after N failures
active probe a dedicated endpoint on a fixed interval
a good probe checks:
the process is accepting connections
a dependency it needs is reachable
the response is a known body, not any 200
do not probe:
the full user-facing path (too slow, too flaky)
a shared dependency with a short timeout in a way
that lets one slow dependency eject every node at once| Decision | Option A | Option B | Choose B when |
|---|---|---|---|
| TLS | Terminate at the edge | Pass through to the backend | Compliance requires end-to-end encryption |
| Sessions | Sticky sessions | Shared session store | You want to scale and deploy without draining |
| Timeouts | Short and aggressive | Long and lenient | Requests are legitimately slow and retries are safe |
| Retries | Retry at the proxy | Retry in the client | The client knows which operations are idempotent |
What a CDN caches
Cache-Control: public, max-age=31536000, immutable # hashed static asset
Cache-Control: public, s-maxage=300, stale-while-revalidate=60
Cache-Control: private, no-store # per-user response
Cache-Control: no-cache # revalidate every time
Vary: Accept-Encoding, Accept-Language # separate cache entriesmax-ageapplies to browsers;s-maxageoverrides it for shared caches such as a CDN.stale-while-revalidateserves a stale copy while fetching a fresh one, which removes the latency spike at expiry.- A
Varyheader multiplies cache entries; a careless one such asVary: User-Agentdestroys the hit rate. - Only cache a response per user if the cache key includes the user identity, and usually it should not be cached at all.
- Purge by tag or by surrogate key so a content update invalidates every related variant, not just one URL.
⚠️
A shared cache and a
Set-Cookie response are a dangerous combination. If a per-user response is cacheable by the edge, the next visitor can receive the previous visitor's session. Mark authenticated responses private and no-store unless you have designed the cache key carefully.FAQ
Should the load balancer retry failed requests?
Only idempotent ones, and only when the failure happened before the request was processed. A blind retry on a non-idempotent POST can duplicate the effect.
Why is my CDN hit rate low?
Usually query strings or varying headers. Normalise or ignore irrelevant query parameters, and audit every
Vary header — each one splits the cache.Related
DNS deep dive: zones, records and DNSSEC Network debugging toolkit
Last refreshed 2026-09-18.