Back-of-the-Envelope Estimation for System Design

Estimation is not a memory contest about how many requests one server can handle. It is a way to translate product behavior into engineering pressure: operations, bytes, concurrency, skew, and recovery margin. The useful result is not a precise-looking number. It is a decision that changes because of the number.

By the end of this lesson, you will be able to:

• turn users and actions into average and peak operation rates;
• expose amplification from fan-out, retries, indexes, replicas, and derived data;
• estimate retained storage, network throughput, cache working sets, and concurrency;
• model skew and hot keys instead of trusting averages;
• convert demand into provisioned capacity using measurements and headroom;
• express uncertain inputs as ranges and identify the assumptions that matter most.

Prerequisites: measurable requirements, critical workflows, and the design worksheet from the previous lesson. The downloadable lab uses Python's standard library only.

Start with an assumption sheet#

Write each input with a unit and a source:

The source column separates evidence from guesses. Keep units in every intermediate result. users × actions/user/day cancels to actions/day; dimensional analysis catches many mistakes.

Use decimal infrastructure units unless a tool explicitly reports binary units:

1 kB = 1,000 bytes       1 GB = 10^9 bytes
1 MB/s = 8 Mb/s          1 day = 86,400 seconds

Do not spend time memorizing exact powers. Record the convention and calculate consistently.

Product actions are not system operations#

For 20 million daily users performing four send actions:

Average product request rate is:

But suppose one request produces three recipient deliveries. Worker demand is:

This is operation amplification. One product action may cause an API write, an outbox insert, a queue publish, three delivery records, three provider calls, analytics events, and index updates. Estimate the constrained stage rather than applying one QPS number to the whole diagram.

Drag canvas to move−

100%

+FitFull screen

Diagram key and text version

• Entry
• Process
• Data
• Outcome

1. 80M sends/day → 926 API req/s avg
2. 926 API req/s avg → 3 recipients/send
3. 3 recipients/send → 2,778 deliveries/s avg
4. 2,778 deliveries/s avg → records + indexes
5. 2,778 deliveries/s avg → provider calls

Make a small rate table:

Retries are another multiplier. If 5% of provider calls retry twice, extra attempts are roughly 2,778 × 0.05 × 2 = 278/s. Correlated failure can be much worse, so retry budgets, jitter, and backpressure belong in the design.

Average, peak, burst, and concurrency answer different questions#

An average spreads work uniformly across a day. Users do not behave uniformly. If the assumed peak factor is 6:

State what “peak” means: the busiest second, minute, or hour produces different capacity requirements. A one-second burst may be absorbed by a queue; an hour-long peak must be processed sustainably or backlog grows.

For interactive systems, Little's Law gives a useful concurrency approximation:

At 16,668 requests/s and 120 ms average time in the service:

This informs connection pools and in-flight memory. Use the latency relevant to the resource: database time for database connections, full request time for edge concurrency.

Try it yourself

A synchronized peak

A live event produces 12 million requests over 10 minutes. Calculate the average rate during the event. If 40% arrive in the busiest minute, calculate that minute's average rate. Why is the daily average unsafe?

Reveal solution

The event average is 12,000,000 / 600 = 20,000 requests/s. The busiest minute receives 4.8 million requests, or 4,800,000 / 60 = 80,000 requests/s. A daily average would be only 139 requests/s, hiding the synchronized event by more than two orders of magnitude.

Storage: count every retained copy#

Start with logical primary data:

For 240 million delivery records/day, 700 bytes each, retained for 365 days:

That 700-byte estimate should come from representative serialization, not the sum of visible fields alone. Include keys, timestamps, schema overhead, and expected payload distribution.

Then list multipliers separately:

• primary indexes and secondary indexes;
• replication;
• backups and their retention;
• write-ahead logs or change streams;
• compression;
• derived search, analytics, or materialized views;
• temporary migration headroom.

If indexes add 40%, three replicas store roughly:

Do not hide these inside one unexplained “3× safety factor.” Their lifecycles differ. A replica supports availability; a backup supports recovery; neither automatically replaces the other.

Growth matters more than the day-one total. The logical ingest rate here is 168 GB/day. Retention deletion and compaction therefore need comparable sustained capacity. If deletion cannot keep up, “365-day retention” is only documentation.

Bandwidth and egress#

Network throughput is rate multiplied by payload size:

At 16,668 responses/s and 1,200 bytes per response:

Calculate important links independently: client ingress, service-to-database traffic, replication, cross-region transfer, provider calls, and internet egress. Protocol overhead, TLS, retransmission, and request bodies add bytes. Cost often follows monthly transferred bytes, while link saturation follows peak bits per second.

For media systems, separate metadata operations from object bytes. Ten thousand image requests per second may be operationally easy but network-heavy; ten thousand tiny counter updates may be byte-light but consistency-heavy.

Cache estimation means working set plus miss path#

Caching every retained record is rarely the goal. Estimate the working set: distinct data likely to be requested within the useful cache window.

cache bytes ≈ hot objects × average cached-object bytes × overhead

If 12 million objects are hot, each cache entry occupies 1.5 kB after key and allocator overhead, and replication is two:

Capacity alone is insufficient. If 80% of peak operations are cacheable and hit rate is 90%, origin demand becomes:

At 16,668/s, that is about 4,667 origin operations/s. Recalculate at cold start, after eviction, or during cache failure. A design that needs a warm cache merely to avoid collapse has a dangerous recovery path.

Skew breaks average-per-partition arithmetic#

Dividing 16,668 operations/s across 16 partitions suggests about 1,042/s each. That assumes uniform keys. If one tenant or celebrity receives 15% of traffic, its key alone receives 2,500/s. A hash cannot split one indivisible hot key across partitions.

Estimate both:

average partition load = total peak / partition count
hot-key load           = total peak × hottest-key share

Mitigations depend on semantics: shard a hot key with a suffix, batch updates, use hierarchical aggregation, cache replicated reads, isolate heavy tenants, or change fan-out strategy. Each may weaken ordering, freshness, or simplicity. Skew is a product property before it is a database property.

Convert demand into capacity with measurements#

Back-of-envelope estimation forecasts demand. Capacity planning needs a benchmark or production service curve measured at the required latency and error rate.

Suppose one worker sustains 800 delivery operations/s while meeting the SLO. Targeting 65% utilization leaves burst and latency margin:

If two workers may be unavailable during a zone fault or rollout, provision at least 35. This is a simplified model; shared dependencies, uneven assignments, startup time, provider quotas, and correlated failures may require more.

Measure the bottleneck resource under representative request shapes. A “requests per server” constant is not portable across payloads, code paths, indexes, hardware, or SLOs. Load test beyond the expected point to see the service curve: throughput may flatten while latency and errors rise sharply.

Use ranges and sensitivity, not false precision#

Create low, expected, and high scenarios. Do not vary every input arbitrarily; vary uncertain inputs that materially affect decisions. Here, users, recipients per action, and peak factor multiply:

The high scenario is not a prediction. It reveals whether the architecture degrades, queues safely, or fails abruptly. Sensitivity analysis changes one assumption at a time. Doubling retention doubles storage but not QPS; doubling fan-out increases worker QPS, writes, storage, and provider calls; increasing peak factor changes instantaneous capacity but not daily storage.

Keep significant figures honest. Inputs such as “6× peak” do not justify an output of 16,666.6667. Round to a decision-relevant value and retain the formula.

A repeatable estimation worksheet#

1. Scope one critical workflow and its unit of work.
2. List inputs with units, ranges, and evidence.
3. Convert users → product actions → component operations.
4. Calculate average, named peak window, bursts, and concurrency.
5. Calculate logical data, indexes, replicas, backups, and retention.
6. Calculate bandwidth for each important link.
7. Estimate cache working set, hit rate, and cold-cache origin load.
8. Test average partition load against hot-key load.
9. Divide peak demand by measured safe capacity; add explicit failure margin.
10. Run low/expected/high scenarios and state which decisions change.

The accompanying lab implements this worksheet as an inspectable model. It intentionally keeps each multiplier named so that a reviewer can challenge the right assumption.

Common estimation failures#

• Starting with servers: product demand and workload shape are still unknown.
• Using DAU as QPS: users perform different numbers and kinds of actions.
• Ignoring fan-out: downstream operations can dwarf API requests.
• Treating a daily average as a peak: synchronized workloads disappear.
• Counting only payload bytes: indexes, replicas, backups, and logs vanish.
• Confusing bits and bytes: bandwidth becomes wrong by 8×.
• Assuming uniform partitions: one hot key can saturate one owner.
• Assuming cache always works: recovery overload is left undesigned.
• Calling a vendor maximum safe throughput: latency and errors at that point may violate the SLO.
• Reporting one precise answer: uncertainty becomes invisible.

Try it yourself

Estimate a photo feed

Assume 8 million DAU, 20 feed reads per user per day, 15 items per response, a 5× peak, and 900 bytes of metadata per item. Estimate average feed requests/s, peak requests/s, and peak metadata egress. Then identify two important quantities still missing.

Reveal solution

Daily requests are 8M × 20 = 160M; average is about 1,852 requests/s; peak is about 9,260 requests/s. Each response contains 15 × 900 = 13,500 bytes, so metadata egress is roughly 9,260 × 13,500 × 8 ≈ 1.0 Gb/s. Missing quantities include image/video bytes, cache hit rate, protocol overhead, fan-out/write workload, skew, concurrency, retention, and cross-region traffic. Which matter depends on the design boundary.

A product action writes one post and fans it out to 200 followers. Which rate should size the fan-out workers?

The peak follower-delivery operation rate, including retry assumptionsDaily active users divided by 86,400Only the post-create API request rateTotal stored bytes divided by retention

Check answer

Why is measured safe throughput lower than a machine's maximum observed throughput?

It is measured while satisfying the latency and error objective and leaves explicit operating marginDecimal gigabytes are smaller than binary gibibytesReplication makes benchmarks unnecessaryDaily averages already include every burst

Check answer

Study optional flashcards

Load parameter

A quantity that describes workload shape, such as request rate, fan-out, read/write ratio, or hot-key share.

Flip card

Operation amplification

Multiple component operations produced by one product action.

Flip card

Working set

Data likely to be accessed within the relevant time window, rather than all retained data.

Flip card

Service curve

Observed latency, errors, and throughput as offered load changes.

Flip card

Safe throughput

Measured per-unit throughput that still satisfies the target service level.

Flip card

Sensitivity analysis

Changing one assumption to see which outputs and decisions move.

Flip card

Memory chart#

Users × actions → product actions
Product actions × fan-out/retries → component operations
Operations ÷ time × peak factor → peak rate
Records × bytes × retention × copies → stored bytes
Rate × payload × 8 → bits per second
Rate × latency → concurrency
Peak demand ÷ measured safe unit capacity → required units

Always add: ranges, skew, cold-cache behavior, failure margin, and a decision.

The next lesson uses these estimates to design APIs, event contracts, data models, indexes, and retention rules around concrete access patterns and invariants.

References#

• Martin Kleppmann: Designing Data-Intensive Applications — Chapter 1's load-parameter and performance-under-load framework.
• Google SRE: Software Engineering in SRE—Capacity Planning — demand forecasting, resource measurement, uncertainty, and capacity planning.
• Google SRE: Handling Overload — overload behavior, per-customer limits, and graceful degradation.
• Google SRE Workbook: Non-Abstract Large System Design — connecting requirements, resource estimates, component limits, and failure modes.
• AWS Builders' Library: Using load shedding to avoid overload — service behavior near capacity and protecting useful work.
• AWS Builders' Library: Timeouts, retries, and backoff with jitter — correlated retries and multiplicative load.
• AWS Well-Architected Framework: Plan for capacity — monitoring demand, quotas, and provisioned capacity.