# System Design Interview: Load Balancers

Aced (formerly Exponent) publishes a public search API over its indexed pages; the guide is at https://www.aced.io/agents/agent-access.

Source: https://www.aced.io/blog/load-balancers-system-design-interview
Source observed: 2026-11-05T00:17:09.158Z
Last checked: 2026-11-05T00:17:09.158+00:00
Indexed: 2026-11-06T01:33:14.829+00:00

---

# System Design Interview: Load Balancers

We explain how load balancers work and why they matter in system design interviews. A [load balancer](https://www.aced.io/courses/system-design-interviews/load-balancers?ref=exponent-blog.ghost.io) sits between external web traffic and backend servers, receives incoming requests, and distributes them across two or more servers.

![Load balancer illustration](https://storage.ghost.io/c/a5/f5/a5f5c2f0-4c62-43c7-891b-662a0b6d68b3/content/images/2023/02/Load-Balancer-Illustration---System-Design.png)

## Scaling and benefits

Every server has finite resource capacity. When it reaches that capacity, scale in either of two ways:

1. **Vertical scaling:** Add processing power, memory, or RAM to one server, up to a finite limit.
2. **Horizontal scaling:** Add more servers.

Load balancing is used with horizontal scaling when there are two or more servers. Benefits include:

- **Minimize response time** by reducing the time needed to answer requests.
- **Maximize throughput** by increasing the number of requests an application can handle.
- **Scale quickly** by adding or removing servers as request volume changes.
- **Improve availability** by keeping the application running when one server reaches capacity.
- **Avoid overload** of any single resource.
- **Monitor server health** and route traffic away from a failed server.

Our [system design interview course](https://www.aced.io/courses/system-design-interviews?ref=exponent-blog.ghost.io) covers load balancers and related concepts with video examples and practice questions.

## What makes a good load balancer?

A good load balancer is efficient, highly available, and secure. Teams commonly use reverse proxies such as [HAProxy](https://www.jenkins.io/doc/book/system-administration/reverse-proxy-configuration-haproxy/?ref=exponent-blog.ghost.io) and [Nginx](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/?ref=exponent-blog.ghost.io). Cloud providers offer managed services such as Amazon Elastic Load Balancer (ELB).

A single load balancer can sit between external traffic and application servers, but this is not universal. In a microservice architecture, place a load balancer in front of each service so services can scale independently.

Load balancing does not solve every system problem. Adding servers will not fix inefficient calculations, slow database queries, or unreliable third-party APIs; those issues require separate solutions.

## Load balancing versus rate limiting

Load balancing groups multiple servers to distribute traffic, improve scalability and reliability, and reduce latency. [Rate limiting](https://www.aced.io/blog/rate-limiter-system-design?ref=exponent-blog.ghost.io) instead protects an application from abuse by throttling or dropping requests that exceed a threshold for a user or organization. Rate limiting is also commonly used against DDoS attacks that consume server resources.

## Interview concepts

### How does a load balancer choose a server?

Common routing mechanisms are:

1. **Round robin:** Send requests to servers in turn.
2. **Least connections:** Send a request to the server with the fewest open connections.
3. **IP hashing:** Consistently send a client request to the same server based on the client's IP.

### Layer 4 and Layer 7 load balancers

#### Layer 4 (L4)

L4 is the transport layer of the OSI model and is also called a network load balancer. Routing decisions use network-layer information. L4 load balancers can manage **millions of requests per second** and perform Network Address Translation (NAT) on request packets without inspecting the packet contents. They distribute traffic across available IP addresses, switches, and routers.

#### Layer 7 (L7)

An Application Load Balancer (ALB) operates at Layer 7 and routes using request content at the application layer, such as HTTP/HTTPS. It can use HTTP headers, cookies, the uniform resource identifier, SSL session ID, and HTML form data to choose a backend. This enables routing by factors such as user location or device type and can distribute requests efficiently. See [When should you use HTTP or HTTPS](https://www.aced.io/courses/system-design/http-vs-https?ref=exponent-blog.ghost.io).

### Sticky-session load balancing

Sticky sessions use IP hashing to map a client IP to one server. Subsequent requests from that user go to the same server for the session duration. Keeping session data on one server can support session state and consistency while reducing latency.

The approach can overload a server if too many requests map to it. Monitor network topology, traffic patterns, and server performance to identify bottlenecks.

### Reverse proxy servers

A reverse proxy sits between clients and backend servers, forwards requests to one or more backends, and returns responses to clients. It can:

1. **Load balance** traffic across origin servers.
2. **Hide IP addresses** and protect origin servers from external users.
3. **Act as a gateway** to content on multiple servers without exposing each resource directly.
4. **Cache content** for faster, more efficient delivery while reducing origin-server load.

### Round-robin variations

Round robin cyclically assigns requests to backend servers. Variants include:

- **Weighted round robin:** Assign weights to servers. If Server A has weight 1 and Server B has weight 2, Server B receives double Server A's requests.
- **Modified round robin:** Assign weights based on how long servers take to process requests.
- **Dynamic round robin:** Adjust weights based on current server load.

### SSL load balancing

With SSL load balancing, the load balancer decrypts incoming data, determines the backend server, and re-encrypts the data before forwarding it. This secures communication between endpoints but adds latency.

When multiple SSL certificates serve different domains or subdomains, the load balancer must select the correct certificate for each request, which can add management time. SSL load balancing does not directly improve performance; it adds complexity and latency that belong in the architecture design.

## Aced resources

Aced, formerly Exponent, provides free interview guides, tips, courses, and practice. [Create a free account](https://www.aced.io/signup?src=blog).

Related posts:

- [Design Food Delivery System like Uber Eats (Mock Interview)](https://www.aced.io/blog/design-food-delivery-uber-eats-system-design-interview)
- [System Design Interview Prep & Questions (2026 Guide)](https://www.aced.io/blog/system-design-interview-guide)
- [System Design Interview: Design a Rate Limiter](https://www.aced.io/blog/rate-limiter-system-design)
- [The 10 Best System Design Books to Sharpen Your Skills](https://www.aced.io/blog/best-system-design-books)

---

[Read this page’s summary](https://www.aced.io/agents/api/public/v1/page-tldr?url=https%3A%2F%2Fwww.aced.io%2Fblog%2Fload-balancers-system-design-interview&trace=t_679ff9ab98f3)

## Ask a follow-up about this page

Need a more specific answer? Ask the exact question you have. It returns a fresh, cited answer from Aced (formerly Exponent)’s knowledge base. No account or API key required.

Open this URL with your own question, URL-encoded. Include your question in q. All research inputs are optional; no account or setup.

`GET https://www.aced.io/agents/agent/faq?q={question}&trace=t_679ff9ab98f3`

[Tool guide and examples](https://www.aced.io/agents/agent-access) · [OpenAPI](https://www.aced.io/agents/api/public/v1/openapi.json) · [Page TLDR form](https://www.aced.io/agents/agent-tools/page-tldr?url=https%3A%2F%2Fwww.aced.io%2Fblog%2Fload-balancers-system-design-interview). Opening a form does not execute a tool.