System Design — Part 02

Introduction

In Part 01, we started with requirements, traffic, architecture and scalability. Now we go one level deeper.

A real system is not just a collection of servers. We need to understand how requests enter the system, how data is accessed, how servers share traffic and what happens when something fails.

        A scalable system is not simply one with more servers.
        It is a system where the major bottlenecks can be
        identified and scaled independently.
    

1. Designing the API

The API is the contract between clients and the backend. A good API makes the system easier to understand, maintain and evolve.

Example: Blog API

``` GET /api/posts GET /api/posts/:id POST /api/posts PUT /api/posts/:id DELETE /api/posts/:id

```

Request Flow

``` Browser │ │ GET /api/posts/123 ▼ API Gateway / Load Balancer │ ▼ Application Server │ ▼ Cache │ ├── HIT ──────► Response │ └── MISS │ ▼ Database │ ▼ Response

```

Keep the API contract simple. Complexity should be introduced only when there is a real requirement for it.

2. Load Balancing

When one application server is not enough, we can run multiple instances.

┌──────────────┐ │ Client │ └──────┬───────┘ │ ▼ ┌─────────────────┐ │ Load Balancer │ └───────┬─────────┘ │ ┌────────────────┼────────────────┐ ▼ ▼ ▼ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ Server 1 │ │ Server 2 │ │ Server 3 │ └──────────┘ └──────────┘ └──────────┘

Common Strategies

Round Robin

Requests are distributed sequentially across servers.

Least Connections

New requests go toward the server handling fewer active connections.

Weighted

More powerful servers can receive more traffic.

Health Checks

A load balancer should know when a server is unhealthy. It can periodically call an endpoint such as:

``` GET /health 200 OK { "status": "healthy" }

```

If a server stops responding correctly, traffic can be redirected to healthy instances.

3. Database Indexes

One of the easiest ways to make a database query faster is to use an appropriate index.

Without an Index

``` SELECT * FROM users WHERE email = '[user@example.com](mailto:user@example.com)'; Database │ ├── Row 1 ├── Row 2 ├── Row 3 ├── Row 4 ├── ... └── Row 1,000,000 Potentially many rows must be examined.

```

With an Index

``` CREATE INDEX idx_users_email ON users(email);

```

The database can use the index to locate matching records much more efficiently.

Indexes are not free. They consume storage and can make writes slower because indexes also need to be maintained.

4. Database Replication

A single database can become a bottleneck or a single point of failure. Replication allows us to maintain multiple copies of the data.

┌──────────────────┐ │ Primary DB │ │ READ/WRITE │ └────────┬─────────┘ │ Replication │ ┌──────────────┼──────────────┐ ▼ ▼ ▼ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ Replica 1│ │ Replica 2│ │ Replica 3│ │ READ │ │ READ │ │ READ │ └──────────┘ └──────────┘ └──────────┘

This is especially useful for read-heavy systems.

Read / Write Split

``` WRITE → Primary Database READ → Replica 1 READ → Replica 2 READ → Replica 3

```

        Replication improves read scalability and availability,
        but introduces an important question:
        How quickly should replicas reflect new data?
    

5. Consistency

Imagine a user changes their profile name. The write succeeds on the primary database, but a replica has not received the update yet.

``` User updates name │ ▼ Primary DB │ │ replication delay ▼ Replica DB Primary: "Alex" Replica: "John"

```

If the next request reads from the replica, the user may temporarily see the old value.

Strong Consistency

Reads reflect the latest successful write.

Eventual Consistency

Different copies may temporarily disagree, but eventually converge to the same state.

Consistency is a system requirement, not simply a database configuration. Decide what the user experience actually requires.

6. Stateless Application Servers

When running multiple servers, it is useful for each server to be able to handle any request.

Bad Example

``` User │ ▼ Server 1 │ └── Session stored only here Next request │ ▼ Server 2 Session unavailable!

```

Better Architecture

┌──────────────┐ │ Load Balancer│ └───────┬──────┘ │ ┌────────────┼────────────┐ ▼ ▼ ▼ Server 1 Server 2 Server 3 │ │ │ └────────────┼────────────┘ ▼ Shared Storage / Session Store

Shared state can be placed in a database, distributed cache or another dedicated storage system.

7. Handling Failures

Every production system eventually experiences failures. The question is not whether something will fail. The question is what the system does when it fails.

Server Failure

Redirect traffic to healthy instances.

Database Failure

Fail over to a replica or recover from backup.

Network Failure

Use timeouts, retries and graceful degradation.

Dependency Failure

Use queues, fallbacks or circuit breakers.

8. Timeouts and Retries

Imagine your application calls a payment service and waits forever because the dependency never responds.

``` Application │ ▼ Payment Service │ X No response │ ▼ Timeout │ ▼ Retry / Fail gracefully

```

Why Timeouts Matter

Without timeouts, a slow dependency can consume application resources until the entire system becomes unhealthy.

Retries must be used carefully. Retrying an operation multiple times can make an overloaded dependency even more overloaded.

9. Queue-Based Processing

Some work does not need to happen before the user receives the response.

``` User │ ▼ API Server │ ├──────────────► Fast Response │ ▼ Message Queue │ ├──► Worker 1 ├──► Worker 2 └──► Worker 3

```

Examples include:

Email delivery
Image processing
Video processing
Report generation
Notifications
Large background jobs

The Important Advantage

The API does not need to perform all expensive work while the user is waiting.

10. Putting Everything Together

┌───────────────┐ │ Client │ └───────┬───────┘ │ ▼ ┌───────────────┐ │ Load Balancer │ └───────┬───────┘ │ ┌──────────────────┼──────────────────┐ ▼ ▼ ▼ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ Server 1│ │ Server 2│ │ Server 3│ └────┬────┘ └────┬────┘ └────┬────┘ │ │ │ └──────────────────┼──────────────────┘ │ ┌────────────┴────────────┐ ▼ ▼ ┌─────────┐ ┌───────────┐ │ Cache │ │ Queue │ └────┬────┘ └─────┬─────┘ │ │ ▼ ▼ ┌──────────┐ ┌─────────┐ │ Primary │ │ Workers │ │ DB │ └─────────┘ └────┬─────┘ │ ┌──────┼──────┐ ▼ ▼ ▼ Replica Replica Replica

Notice that each component has a reason to exist. We are not adding infrastructure simply because it is popular.

11. The Engineering Trade-offs

Technique	Helps With	Introduces
Load Balancer	Availability / Traffic	Infrastructure complexity
Cache	Latency / DB load	Invalidation problems
Replication	Read scaling	Consistency concerns
Queue	Async workloads	Delayed processing
Retries	Temporary failures	Duplicate requests / load
Indexes	Query performance	Storage / write overhead

Good system design is the process of understanding these trade-offs and choosing the appropriate one for the problem.

12. Interactive Failure Simulator

A system becomes easier to understand when we can simulate what happens when individual components fail.

System is waiting for an event...

13. What We Learned

Design APIs around clear contracts.
Use load balancers to distribute traffic.
Use indexes for important database access patterns.
Use replicas when read scaling requires them.
Understand strong vs eventual consistency.
Keep application servers stateless when possible.
Design explicitly for failures.
Use timeouts when communicating with dependencies.
Use queues for expensive asynchronous work.
Understand the trade-offs of every component.

Conclusion

In Part 01 we learned how to start a system design problem. In Part 02 we went deeper into the components that make a system scalable and reliable.

The next step is to take these individual concepts and apply them to real systems.

        Part 02 takeaway:
        Design the API → distribute traffic → optimize data access →
        replicate when needed → handle failures → understand trade-offs.
    

In the next part, we can take a real-world problem such as a URL shortener, Instagram-like feed, chat application or notification system and design it from requirements to production architecture.

Comments