Back-of-the-Envelope Estimation for System Design

Fine all week, down at 8:01#

This is part 3 of the System Design Introduction, and it stands on its own. Picture an exam board's results website. All week it's quiet: a few students check dates, a few teachers sign in. Results go live at 8:00 am on results day. At 8:00, hundreds of thousands of students press refresh at once. At 8:01 the site is down.

Afterwards someone says: "But it only gets 21 visits a second." That was true for the day as a whole. It was also useless, because nobody arrives at the day as a whole. Half of the students arrived in the same ten minutes.

A few minutes of rough arithmetic before launch would have shown this. That arithmetic is a back-of-the-envelope estimate: a quick, deliberately rough calculation, with round numbers and powers of ten, that tells you the size of a problem. How many requests a second? How much data? How much network traffic? You want the right order of magnitude (is it 20, 2,000 or 200,000?), not the exact figure, because the order of magnitude is what decides the design: one server or several, memory or disk.

The estimation chain#

Every estimate walks the same chain. Each step takes the previous number and applies one assumption:

1. People: who uses the system? Here, students.
2. Actions per person: what does each one do? Here, the pages each student loads.
3. Requests per day: people × actions.
4. Average per second: requests per day ÷ seconds in a day.
5. Peak per second: requests in the busiest window ÷ the seconds in that window.
6. Storage and bandwidth: how much you keep, and how much you send each second.

The story has already shown you the terms. Requests per second, often called QPS (queries per second), is how hard the servers work. The average spreads the load evenly over the day; the peak is the load in the busiest moment, and the peak is what knocks a site over. Storage is how much data you keep. Bandwidth is how many bytes leave your servers each second.

Three rules of thumb keep the maths in your head:

• A day is about 100,000 seconds. It's really 86,400 (24 × 60 × 60), but 10^5 is close enough and makes dividing easy.
• Each unit step is ×1,000. 1,000 KB is 1 MB, 1,000 MB is 1 GB, 1,000 GB is 1 TB. Memory sizes strictly use 1,024, but the difference is a few per cent and never changes an order of magnitude.
• Round early and say your assumptions out loud. "Assume 3 pages per student" can be challenged and fixed; a hidden assumption can't.

The numbers, one line at a time#

Here are the inputs in this example. They're illustrative, not data from a real exam board:

• Students: 600,000
• Pages each student loads (sign in, results, certificate): 3
• Share who check in the first 10 minutes: 50%
• One stored result record: 2 KB
• One results page sent to the browser: 50 KB

Now walk the chain. Each line is one step:

Look at the two averages. 21 and 18 differ by about 15%, and it doesn't matter: both say "tens per second". The peak says "thousands per second". That gap of nearly two orders of magnitude, not the decimals, is what the site needed to know.

Bandwidth needs one conversion. A 1 Gbps network link carries 1,000 megabits a second, which is 125 megabytes in theory; the handy metrics in the System Design Primer round real-world throughput to about 100 MB/s. So 75 MB/s at the peak would fill most of one server's link, while the average day (21 × 50 KB ≈ 1 MB/s) barely touches it.

What the numbers decide#

An estimate is only worth doing if it changes a decision. Here, each number makes one:

• 1.2 GB of results: keep them all in memory. That fits in the RAM of one ordinary server, so every lookup can come from a cache instead of a disk. The well-known list "Latency numbers every programmer should know" puts a main-memory reference at about 100 ns and a spinning-disk (HDD) seek at about 10 ms: 100,000 times slower.
• 1,500 requests a second: one server is not enough. Suppose, in this example, one server comfortably handles 500 requests a second for this page. 1,500 ÷ 500 = 3 servers, and you'd run about 6 so the site survives one failing and a busier morning than planned. A load balancer spreads requests across them.
• 75 MB/s: move the heavy part to a CDN. Most of the 50 KB page is the same for everyone: styles, logo, scripts. A content delivery network (CDN) serves those shared files from servers close to the students. Assume the personal part is about the size of the stored record, 2 KB, and treat every request as a results page. If only that part comes from your servers, peak traffic drops to 1,500 × 2 KB = 3 MB/s.

Where estimates go wrong, and the interview#

Most estimate mistakes aren't arithmetic. They're a missing step:

• Sizing for the average. The opening mistake. Anything with a deadline, a launch or a daily rush (results, ticket sales, the morning commute) needs its peak worked out separately.
• Forgetting copies and growth. Real systems keep several copies of the data (replication) and keep it for years. In this example, 3 copies of 5 years of results is 1.2 GB × 3 × 5 = 18 GB. Still small, but 15 times the first answer, which matters when you choose where it lives.
• False precision. Writing 20.83 requests a second claims a certainty you don't have. Your inputs are guesses, so round the answer to match.

In a system design interview, estimation usually comes right after clarifying requirements. The interviewer isn't checking your long division. They want to hear your assumptions said out loud, round numbers, an order of magnitude, and then a design choice that follows from it: "About 1,500 requests a second at peak, so several servers behind a load balancer, with every result cached in memory." Spend two or three minutes on it, not ten.

Recap#

The results site was fine on average and broke at the peak. The estimate that would have saved it fits in a few lines: people, actions, requests, average, peak, storage, bandwidth, each rounded, each with its assumption said out loud. Next in System Design: performance vs scalability.

Estimate the peak, not the average: rough numbers decide the design.

Practice

Check your understanding

FAQ

Frequently asked questions