Muhammad Imaz Anwar

Engineering lessons
Learn by building

Free learning playlist

System Design: Foundations to Distributed Data

A practical system design course covering estimation, APIs, storage, distributed data, reliability, consensus, streams, and production case studies.

Audience
Software engineers who want a repeatable path from a simple design to production trade-offs.
Publishing
New System Design lesson every Friday
Progress
4 published · 27 planned
Start with lesson 1 →

COMPLETE CURRICULUM / REFERENCES

Everything this course will cover

The complete curriculum is visible below—including unpublished lessons. Status labels distinguish what is available, what comes next, and what remains planned.

PublishedNext lessonPlanned

Books and primary references

  1. The course follows a repeatable teaching structure: concept, production explanation, interview explanation, memory chart, case study, practice, failure modes, and teach-back.
  2. The supplied Designing Data-Intensive Applications syllabus defines the data-systems spine from reliability and data models through replication, partitioning, transactions, consensus, batch, and streams.
  3. Google SRE books and workbook supply measurable reliability, SLO, overload, incident, monitoring, and operational practices.
  4. AWS Builders’ Library and Well-Architected material supply concrete production patterns, failure analysis, capacity, retries, backpressure, and multi-region trade-offs.
  5. IETF RFCs, official database documentation, and original distributed-systems papers define normative behavior and algorithmic guarantees.

Implementation layer

System Design lessons use executable Python for estimators, simulations, hashing, queues, consistency anomalies, replication lag, partitioning, and stream-processing exercises. Architecture lessons also include accessible SVG system-flow diagrams plus text fallbacks.

Module 01

The system design solving framework

4 lessons

Learn what to ask, calculate, draw, and defend before selecting technology.

What this module covers

  • functional and non-functional requirements, scope, constraints, invariants, and quality targets
  • repeatable clarification-to-trade-off framework and design worksheet
  • DAU, actions, QPS, peaks, read/write ratios, storage, bandwidth, working sets, skew, and hot keys
  • HTTP/event contracts, idempotency, pagination, error semantics, access-pattern-driven data models, indexes, and retention

Reference spine: System Design course framework · Google SRE Workbook · RFC 9110 and RFC 9457 · DDIA Chapter 1 foundations

  1. 01What System Design Actually IsPublished · 13 minRead lesson →
  2. 02The Repeatable System Design FrameworkPublished · 10 minRead lesson →
  3. 03Back-of-the-Envelope EstimationPublished · 11 minRead lesson →
  4. 04API and Data Model DesignPublished · 19 minRead lesson →

Module 02

Core building blocks

5 lessons

Understand the behavior and cost of the components that appear in real system diagrams.

What this module covers

  • networking, DNS, HTTP, TLS, proxies, API gateways, CDNs, and edge delivery
  • vertical/horizontal scaling, load balancing algorithms, health checks, failover, and stateless services
  • cache-aside/read-through/write-through/write-back, TTL, invalidation, stampedes, hot keys, and consistency
  • queues, logs, workers, delivery semantics, ordering, retries, dead letters, backpressure, and load shedding
  • object storage, search indexes, rate limiting, quotas, service discovery, and configuration

Reference spine: System Design course framework · Google SRE · AWS Builders’ Library · official HTTP/Redis/Kafka/cloud documentation

  1. 05Networking, HTTP, DNS, and CDNsNext lessonUpcoming
  2. 06Load Balancing and Horizontal ScalingPlannedUpcoming
  3. 07Caching Strategies and InvalidationPlannedUpcoming
  4. 08Queues, Workers, and BackpressurePlannedUpcoming
  5. 09Object Storage, Search, and Rate LimitsPlannedUpcoming

Module 03

Foundations of data systems

4 lessons

Follow the core DDIA progression from dependable applications through storage and schema evolution.

What this module covers

  • reliability, hardware/software/human faults, scalability, load, performance percentiles, operability, simplicity, and evolvability
  • relational/document/graph models, object-relational mismatch, relationships, declarative queries, MapReduce, Cypher, SPARQL, and Datalog
  • hash indexes, SSTables, LSM trees, B-trees, in-memory systems, OLTP/OLAP, warehouses, column stores, compression, sort order, and materialized views
  • JSON/XML/binary formats, Thrift, Protocol Buffers, Avro, schema evolution, REST/RPC, databases, and message dataflow

Reference spine: DDIA Chapters 1–4 · official database and serialization documentation

  1. 10Reliable, Scalable, and Maintainable SystemsPlannedUpcoming
  2. 11Data Models and Query LanguagesPlannedUpcoming
  3. 12Storage, Retrieval, and IndexesPlannedUpcoming
  4. 13Encoding and Schema EvolutionPlannedUpcoming

Module 04

Distributed data

5 lessons

Develop precise mental models for replicated, partitioned, transactional, and coordinated systems.

What this module covers

  • leader/follower, synchronous/asynchronous replication, failover, logs, lag, read-your-writes, monotonic/prefix reads, multi-leader, leaderless, quorums, conflicts, and version vectors
  • range/hash partitioning, skew, hot spots, secondary indexes, rebalancing, routing, and parallel execution
  • ACID, read committed, snapshot isolation, lost updates, write skew, phantoms, serial execution, 2PL, and SSI
  • partial failures, unreliable networks, timeouts, clocks, process pauses, fencing, fault detection, knowledge, system models, and Byzantine faults
  • linearizability, causality, ordering, total-order broadcast, 2PC, consensus, membership, and coordination

Reference spine: DDIA Chapters 5–9 · original CAP, clock, transaction, and consensus papers · official database documentation

  1. 14Replication: Leaders, Followers, and QuorumsPlannedUpcoming
  2. 15Partitioning, Rebalancing, and Hot SpotsPlannedUpcoming
  3. 16Transactions, Isolation, and SerializabilityPlannedUpcoming
  4. 17Partial Failures, Clocks, and TimeoutsPlannedUpcoming
  5. 18Consistency, Ordering, and ConsensusPlannedUpcoming

Module 05

Derived data and production systems

4 lessons

Connect batch and streaming dataflows to operations, security, recovery, and real architectures.

What this module covers

  • Unix data processing, MapReduce execution, joins, grouping, distributed filesystems, materialization, iterative graphs, and higher-level APIs
  • messaging systems, partitioned logs, database/stream integration, CDC, event sourcing, immutability, event time, windows, joins, and stream fault tolerance
  • capacity and overload, multi-region topologies, disaster recovery, RPO/RTO, backups, restore tests, migrations, and graceful degradation
  • metrics/logs/traces, SLI/SLO, security boundaries, threat modeling, privacy, cost models, ownership, and operations

Reference spine: DDIA Chapters 10–11 · Google SRE · AWS Builders’ Library · Kafka and stream-processor documentation

  1. 19Batch Processing and DataflowsPlannedUpcoming
  2. 20Stream Processing, CDC, and Event SourcingPlannedUpcoming
  3. 21Multi-Region Design and Disaster RecoveryPlannedUpcoming
  4. 22Observability, Security, and CostPlannedUpcoming

Module 06

End-to-end case studies

5 lessons

Apply the framework and expose the failure modes hidden by clean architecture diagrams.

What this module covers

  • URL shortener and distributed rate limiter
  • chat, notification, and presence systems
  • news feed and fan-out strategies
  • file upload, media delivery, and object storage
  • search, autocomplete, indexing, and ranking
  • payments, ledgers, idempotency, reconciliation, and workflows
  • analytics pipelines, batch/stream integration, and a production RAG platform

Reference spine: System Design case-study framework · DDIA concepts · Google SRE and AWS production guidance · relevant protocols and official service documentation

  1. 23Design a URL ShortenerPlannedUpcoming
  2. 24Design a Chat SystemPlannedUpcoming
  3. 25Design a News FeedPlannedUpcoming
  4. 26Design a Payment SystemPlannedUpcoming
  5. 27Design a Production RAG PlatformPlannedUpcoming