What System Design Actually Is

System design is the work of deciding how software components, data, and operational processes cooperate to satisfy a product’s requirements under realistic load and failure.

It is not a competition to draw the most boxes. A queue, cache, replica, search index, or microservice is useful only when it solves a named constraint or failure mode better than the alternatives.

By the end of this lesson, you will be able to:

• separate functional requirements from quality constraints;
• distinguish latency, throughput, availability, reliability, and durability;
• connect demand and capacity to architecture decisions;
• use a repeatable eight-step design process;
• explain CAP without the common “choose any two” mistake;
• evolve a simple URL shortener and defend every added component.

Start with the smallest honest system#

Many products can begin here:

Client ──HTTP──> Application ──query──> Database

This design is not embarrassing. At modest scale, one deployable application and one well-operated relational database may be faster to build, easier to test, cheaper to run, and more reliable than a distributed architecture.

The design must change when evidence changes:

• read traffic exceeds database capacity;
• large background work makes request latency unacceptable;
• one process cannot meet availability objectives;
• data no longer fits the storage or access pattern;
• global users cannot tolerate round-trip latency to one region;
• independent teams need different release or scaling boundaries.

Good system design is controlled evolution. Start simple, identify the limiting resource, and introduce the smallest mechanism that improves the required property.

Design for faults so they do not become system failures

DDIA makes a useful distinction:

• a fault is one component deviating from its specification—a disk becomes

unavailable, a process pauses, or one request handler returns corrupt data;

• a failure is the system as a whole no longer providing the required

service to its user.

Fault tolerance does not mean pretending components never break. It means defining which faults the system should absorb before they become user-visible failures. That boundary must be explicit: “survive one application-instance loss” is testable; “highly available” is not.

Not every fault should be hidden. A malformed payment command should be rejected rather than converted into a seemingly successful response, and a corrupted replica may need isolation instead of automatic failover. Reliability means preserving the promised behavior, not returning success at any cost.

Functional and non-functional requirements#

Functional requirements describe capabilities: what users or other systems can do. For a video platform, users may upload, watch, search, and comment.

Non-functional requirements describe the conditions under which those capabilities must work. Examples include latency, throughput, availability, durability, consistency, security, recoverability, operability, and cost.

Averages hide pain. “Average latency below 100 ms” can coexist with a terrible slow tail. State which percentile matters, for which operation, over which window, and at what load.

The six forces you balance#

Scale

Scale includes users, requests per second, concurrency, read/write ratio, item size, retention, bandwidth, geographical distribution, and skew. Ten thousand evenly distributed requests are different from ten thousand requests aimed at one celebrity, tenant, or cache key.

Describe load with the parameters that drive this particular system: reads per second, writes per second, fan-out, active connections, payload size, cache hit rate, or jobs waiting in a queue. Then describe how latency, throughput, error rate, and resource use change as those parameters grow.

That two-part description is more useful than saying a system “scales.” A feed service may handle ten times more readers by adding cache capacity yet still collapse when one author creates extreme fan-out. Scalability is the ability to cope with a changed load parameter while preserving the required behavior—not a property obtained merely by choosing distributed technology.

Latency

Latency is the time one operation takes. End-to-end latency includes client networking, queues, application processing, dependency calls, storage, serialization, and retries.

Consistency

Consistency describes which values an operation is allowed to observe. The correct guarantee depends on the invariant. A profile photo can often propagate asynchronously; an inventory reservation or money movement needs a more carefully defined ordering and conflict policy.

Availability and reliability

Availability asks whether the operation can be used now. Reliability asks whether it performs the correct function over time. A payment endpoint returning 200 OK while charging twice is available but unreliable.

Cost

Cost includes infrastructure, network transfer, storage, third-party calls, engineering time, incident response, and opportunity cost. A technically scalable design can still be a bad business design.

Complexity

Every distributed boundary adds deployment, observability, schema evolution, testing, timeout, retry, and ownership problems. Complexity must purchase a specific capability.

Lower latency often costs more capacity or caching.
Higher reliability requires redundancy and recovery work.
Stronger coordination can reduce availability during failures.
More scale usually increases operational complexity.

Latency is not throughput#

Latency is time per request. Throughput is completed work per unit time. They interact but are not interchangeable.

Imagine one worker takes 100 ms per independent request. Its theoretical sequential throughput is about 10 requests per second. Ten workers might approach 100 requests per second while each request still takes around 100 ms. When demand exceeds total capacity, requests queue; throughput stops growing and latency rises sharply.

Real systems need headroom. Running at 100% theoretical capacity leaves no room for traffic variance, slow dependencies, garbage collection, deployments, or a failed replica. Capacity planning therefore uses measured service curves and a target operating range, not only arithmetic.

Back-of-the-envelope estimation

Rough estimates are not predictions. They identify orders of magnitude and likely bottlenecks.

Use this flow:

Users → actions per user → daily requests → average QPS → peak QPS
      → write volume → retained storage → network bandwidth → hot keys

Useful approximations:

The next estimation lesson will calculate these carefully. For now, remember that “millions of users” is not a load model. Users must perform actions, and actions create compute, storage, and bandwidth.

Availability, durability, and service objectives#

Availability means the service is able to handle an operation. Durability means acknowledged data survives the failures covered by the contract. They can differ: a storage system may temporarily reject reads while preserving every confirmed write.

Production teams make these qualities measurable with indicators and objectives:

• SLI: the measured indicator, such as successful checkout requests or p95 latency;
• SLO: the target for that indicator, such as 99.95% successful checkouts per month;
• SLA: an external agreement that may specify consequences when a target is missed.

An SLO requires an exact population. Does a “successful request” exclude client errors? Does a timeout count at the edge or application? Can retries hide the first failure? Ambiguous metrics produce comforting dashboards rather than operational truth.

CAP theorem without the slogan#

CAP concerns a replicated system during a network partition that prevents some nodes from communicating.

• Consistency in CAP means linearizability: operations behave as if one up-to-date copy exists.
• Availability means every request received by a non-failing node eventually returns a non-error response.
• Partition tolerance means the system continues according to its defined behavior despite dropped or delayed communication between groups of nodes.

During a partition, a system cannot guarantee both linearizability and availability for every operation. It may reject or delay some requests to protect a single order, or answer from reachable state and risk returning stale/conflicting data.

This is not “choose any two of three” during normal operation, and entire databases are not permanently “CP” or “AP.” Real products choose behavior per operation, failure, configuration, and invariant. Even without a partition, latency and consistency involve additional trade-offs.

For example:

• an account balance transfer may refuse service without the required coordination;
• a social feed may serve slightly stale cached posts;
• a shopping cart may accept regional writes and reconcile conflicts using product rules.

The label is less useful than an explicit statement of what a read or write can do when communication fails.

A repeatable eight-step process#

Use this sequence for interviews and production design reviews:

1. Clarify the problem. Identify users, primary workflows, boundaries, and what is out of scope.
2. Define requirements. State functional capabilities and measurable quality targets.
3. Estimate scale. Calculate average and peak traffic, storage, bandwidth, growth, and skew.
4. Design the contract. Define APIs or events, authentication, idempotency, pagination, and errors.
5. Design data around access. Identify entities, invariants, queries, indexes, retention, and ownership.
6. Draw the simplest end-to-end flow. Show where a request enters, where truth lives, and how results return.
7. Deep-dive the limiting paths. Test reads, writes, hot keys, failures, recovery, security, and observability.
8. State trade-offs and evolution. Explain why this design fits today and what evidence would trigger the next change.

Clarify
   ↓
Requirements → Estimates → API + data contracts
                              ↓
                       Simple end-to-end flow
                              ↓
                  Bottlenecks + failure analysis
                              ↓
                    Trade-offs + evolution plan

Do not complete the steps as a rigid waterfall. A storage estimate may expose a missing retention requirement; an API discussion may reveal an invariant. Revisit earlier assumptions explicitly.

Worked example: a URL shortener#

Design a service that accepts a long URL, returns a short URL, and redirects readers from the short code to the original destination.

1. Clarify scope

Core features:

• create a short link;
• redirect a short code;
• optionally choose expiration;
• collect click analytics asynchronously.

Exclude custom aliases, abuse review UI, and billing from the first version. Still note that malicious destinations and open redirects require a safety policy before public launch.

2. Define quality targets

Redirects are more latency-sensitive and frequent than link creation. A confirmed mapping must be durable. Temporary analytics loss must not block a redirect. Hot viral links should not overload one database row or partition.

3. Estimate

Assume 10 million new links per day and 100 million redirects per day:

create average ≈ 10,000,000 / 86,400 ≈ 116 requests/s
redirect average ≈ 100,000,000 / 86,400 ≈ 1,157 requests/s
read/write ratio ≈ 10:1

If each durable mapping plus metadata is roughly 1 KB, logical growth is about 10 GB/day before indexes, replication, and backups. A traffic distribution estimate must include a viral link, not just averages.

4. Define contracts

POST /v1/links
Idempotency-Key: 441c...
Content-Type: application/json

{"url":"https://example.com/long/path","expiresAt":null}

HTTP/1.1 201 Created
Content-Type: application/json

{"code":"aZ91kQ","shortUrl":"https://sho.rt/aZ91kQ"}

Redirect reads use GET /{code} and return a redirect status with the destination. Define whether destinations can change because that decision affects caching and auditability.

5. Model the durable mapping

Link(code PK, destination, owner_id, created_at, expires_at, status)

The primary access pattern is lookup by code. Owner listings require a separate index such as (owner_id, created_at). Click events belong in an asynchronous analytics path rather than the mapping row.

6. Draw the simple design

Create:   Client → Link API → Mapping database

Redirect: Browser → Redirect service → Mapping database → 302 response

Analytics: Redirect service → event queue → analytics workers → analytical store

Start here and measure. The queue isolates redirect latency from analytics work, but it introduces lag, duplicate delivery, retention, and consumer failure behavior.

7. Evolve the bottleneck

Because redirects are read-heavy, cache popular code-to-destination mappings:

Browser → load balancer → stateless redirect services → cache → mapping database
                                                        │
                                                        └── miss → database → fill cache

Now define cache behavior:

• TTL and invalidation when a link is disabled;
• negative-cache policy for nonexistent codes;
• request coalescing on a hot cache miss;
• behavior when cache is unavailable;
• protection against one viral key or destination.

The cache improves latency and database load, but stale data can keep a disabled link active. The product’s abuse and deletion requirements determine whether that risk is acceptable.

8. State the evolution plan

Do not shard because the diagram looks more serious. Partition when measured storage, write load, maintenance time, or availability limits justify it. Choose a partition key after understanding access patterns and hot-key behavior. Retain analytics according to product value and cost rather than forever by default.

Failure questions that improve a design#

Ask these before declaring the diagram complete:

• What happens when a dependency is slow rather than fully down?
• Which retries are safe, and which operation needs an idempotency key?
• Where can duplicate messages appear?
• What happens when only one zone or network path fails?
• Which data is authoritative, cached, derived, or disposable?
• Can one user, tenant, or key receive a disproportionate load?
• How is bad data repaired or restored?
• Which metrics, logs, and traces reveal the failure?
• What is the degraded mode, and how does it recover?

Memory chart#

SYSTEM DESIGN = requirements + scale + failure behavior + trade-offs

Functional: what does it do?
Non-functional: how well, under what conditions?

Latency: time per operation
Throughput: operations per time
Availability: can it serve now?
Reliability: does it behave correctly?
Durability: does confirmed data survive?

Clarify → Requirements → Estimate → Contracts
        → Data → Simple flow → Deep dives → Trade-offs

Every component must answer:
What problem does it solve?
What new failure or cost does it add?

The next lesson turns the eight-step process into a reusable design worksheet. You will practice requirement questions, access-pattern-driven APIs and data models, and a structured way to present trade-offs.

References#

• Kleppmann: Designing Data-Intensive Applications, Chapter 1 — reliability, faults versus failures, load parameters, percentiles, scalability, and maintainability.
• DDIA Chapter 1 source bibliography — the author-maintained primary-source map behind the chapter.
• Google SRE Workbook: Implementing SLOs — defining indicators, objectives, and measurement populations.
• AWS Well-Architected Framework — operational, security, reliability, performance, cost, and sustainability review principles.
• RFC 9110: HTTP Semantics — authoritative HTTP method and response semantics.
• Gilbert and Lynch: Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services — formal CAP result and definitions.
• Google SRE Book: Handling Overload — capacity, queuing, load shedding, and graceful overload behavior.