Load balancers: algorithms, health checks, L4 vs L7

Consider what information the balancer can see at each layer.

Model answer. A load balancer sits in front of a pool of servers and distributes incoming requests among them, improving throughput and availability. It also lets you add, remove or update servers without clients noticing.

Common algorithms

• Round robin: send requests to servers in turn. Simple; works when servers and requests are similar.
• Weighted round robin: stronger servers get proportionally more requests.
• Least connections: send to the server with the fewest active connections — good when request durations vary a lot.
• IP hash / consistent hashing: hash the client IP or a key so the same client goes to the same server — useful for cache locality or simple session affinity.
• Random with two choices: pick two servers at random and use the less loaded one — cheap and surprisingly effective.

Health checks
The balancer periodically probes each server (e.g. GET /health) and stops routing to servers that fail, adding them back once they recover.

Layer 4 vs layer 7

• L4 (transport level): decides using IP addresses and TCP/UDP ports only, without reading the payload. Very fast, protocol-agnostic, and can pass TLS through untouched.
• L7 (application level): understands HTTP, so it can route by URL path (/api to API servers, /images to a media service), headers or cookies; it can terminate TLS, compress responses, rewrite headers and apply rate limits. It costs more CPU per request.
• Examples: AWS Network Load Balancer is L4; AWS Application Load Balancer, Nginx and HAProxy in HTTP mode are L7 (HAProxy and Nginx can also do L4).

Avoiding a single point of failure

• Run load balancers in a redundant pair — active-passive with a floating virtual IP (e.g. VRRP/keepalived) or active-active.
• Use DNS to spread traffic across multiple load balancers or regions.
• Managed cloud load balancers are already distributed across multiple machines and zones.

Common follow-up: "How do you deploy a new version without downtime?" Use connection draining: mark one server as out of rotation, let in-flight requests finish, upgrade it, health-check it, return it, and repeat (a rolling deployment).