Real-Time Chat System Design (WhatsApp-Scale)

WhatsApp moves on the order of 2 billion messages a day. The interesting part isn't sending one message to one online recipient — that's a websocket push. The interesting part is that the recipient might be offline, might have three devices, and might be in a 500-person group, and the system has to handle all three without three different code paths. This post walks through that design the way you'd talk through it in an interview.

Requirements and Goals of the System

Functional Requirements

• Send a message in a 1:1 or group conversation
• Deliver it immediately if the recipient is online; deliver it when they reconnect if they're not
• Support delivery and read receipts
• Keep messages in sync across a user's multiple devices (phone, web, desktop)

Below the line (out of scope)

• Voice/video calling — a genuinely different system (media transport, not message delivery)
• The end-to-end encryption key-exchange protocol itself — we'll assume message content arrives at the server already encrypted, and design around that constraint rather than designing the cryptography
• Media (photo/video) transfer — handled by a separate blob-storage upload path; the message itself just carries a reference

Non-Functional Requirements

• Scale: ~2 billion messages/day, which averages to roughly 23,000 messages/second, with peaks several times higher around regional evenings
• Latency: delivery to an online recipient should feel instant — under 100-200ms
• Ordering: messages within one conversation must be delivered in a consistent order to every participant, regardless of which server handled which message
• Durability: a message can never simply be lost because the recipient happened to be offline when it was sent

Capacity Estimation and Constraints

Traffic: ~23,000 messages/second on average; assume peaks of ~100,000/second during regional evening hours.

Storage: a typical text message is well under 1KB once metadata is included — at 2 billion/day, that's under 2TB/day of message log data, which is a manageable, steadily-growing append-only log rather than anything exotic.

Concurrent connections: the number that actually shapes the architecture. With, say, 500 million daily active users and a large fraction connected at once during peak, the platform needs to hold on the order of 100-200 million concurrent WebSocket connections. If each open connection costs roughly 10KB of memory on a gateway node (buffers, TLS state, routing metadata), that's ~1-2TB of memory needed just to hold connections — meaning this has to be horizontally sharded across many gateway nodes (a node comfortably holding 50,000 connections implies on the order of 2,000-4,000 gateway nodes), which is exactly why a presence layer that tracks which node holds which connection becomes unavoidable rather than optional.

System APIs

client_message_id is generated by the sending device and used for deduplication — if a send is retried after a flaky connection, the server recognizes the duplicate and returns the original server_sequence instead of creating a second message.

High-Level Design

The core requirement to design around: deliver a message instantly if the recipient is online, and just as reliably if they're not — without two separate systems.

Bad solution: clients poll the server every few seconds for new messages.

This is the simplest thing that could work, and it fails the "instant" requirement immediately — delivery latency is bounded by the poll interval, and most polls return nothing, which means the server is doing constant, mostly wasted work at hundreds of millions of users.

Good solution: each client holds a persistent WebSocket connection; when a message arrives for a conversation, the server pushes it down the recipient's open connection immediately.

This solves latency for the online case, but raises the actual hard question: no single server can hold every user's connection, so which server does a given user's connection live on right now — and what happens if they're not connected to any server at all?

Great solution: add a presence layer that maps user_id/device_id → which WebSocket gateway node holds their connection, if any, and make every message write to a durable, ordered per-conversation log regardless of whether the recipient is online. When a message arrives: look up presence for each recipient device. If a device is online, forward the message to the gateway node holding its connection for an immediate push. Whether or not it was online, the message is also durably appended to the conversation log. A device that reconnects — because it just came online, or because a push notification (APNs/FCM) woke a backgrounded app — simply calls sync with its last-known sequence number and catches up on everything it missed.

Durable-log-first is the key idea: online delivery is just an optimization on top of a system that would work correctly even if every push notification failed.

Database Design

• User — an account, which can have several registered Devices at once
• Device — one logged-in client (phone, web, desktop), each tracking its own sync position
• Conversation — a 1:1 or group thread, with a participant list
• Message — content, sender, conversation, and a server-assigned sequence number
• DeliveryReceipt — per message, per recipient device: sent / delivered / read

The load-bearing modeling choice: Message is append-only within a conversation's log, ordered by a server-assigned sequence number, not a client timestamp — clocks drift, and two senders can produce the same millisecond timestamp. Every device tracks its own "last synced sequence number" per conversation rather than the server tracking "has this message been shown to this device" as a mutable flag — that single cursor is what makes reconnect-and-sync, multi-device, and offline delivery all the same mechanism instead of three separate ones.

Detailed Component Design

How do we guarantee message ordering and avoid double-delivery?

Ordering is solved by assigning a monotonically increasing sequence number per conversation at write time, on the server — never trusting the sender's device clock for ordering. Double-delivery (a client retries a send because it never got an acknowledgment, even though the message actually went through) is solved by the client-generated client_message_id: the server checks it against recently-seen IDs for that conversation and returns the existing sequence number for a duplicate instead of creating a new message. This is the same idempotency-key pattern used anywhere a client can't be sure whether its last request succeeded.

How does a message reach a 500-person group without 500 separate writes?

Fan-out-on-write — copying the message into 500 individual inboxes — is the naive answer, and it's the wrong one here specifically because chat, unlike a social feed, has no per-recipient customization: everyone in the group sees the identical message in the identical order. A single shared, append-only log per conversation plus a per-device sync cursor handles a 500-person group exactly the same way it handles a 1:1 chat — the fan-out only happens at delivery time (looking up presence for however many of those 500 are online right now), never at write time.

How does multi-device sync actually work?

Each of a user's devices is a first-class participant in delivery, not a special case: a message to a user with three devices looks up presence for all three, pushes to whichever are online, and each device independently tracks its own last-synced sequence number per conversation. A device that's been offline for two weeks catches up identically to one that's been offline for two seconds — it just asks for everything after its own cursor.

Fault Tolerance and Replication

The recurring theme: nothing on this list can cause message loss, because the durable log write happens independently of every delivery mechanism above it — each failure degrades speed, not correctness.

Extended Requirements

Observability & Monitoring

Security & Compliance

If message content is end-to-end encrypted, the server is deliberately blind to it — which means features like server-side search or content moderation on message text simply aren't available to this system, and any product requirement for them has to be solved client-side or explicitly traded away. Retention policy matters even for opaque, encrypted content: how long undelivered messages are held for an offline device, and how long delivery metadata (who messaged whom, when) is retained, are privacy decisions the architecture has to make room for, not afterthoughts.

Leveled Expectations

Mid-level: Lands on persistent WebSocket connections with server-side push for the online case, and can be prompted into handling the offline case as a fallback.

Senior: Independently proposes a presence layer and a durable per-conversation log as the unifying mechanism for online delivery, offline delivery, and reconnect-sync — rather than three separate features; gets ordering and de-duplication right without prompting.

Staff+: Reasons about the group-fanout trade-off (write-time vs. read-time) explicitly and picks read-time for the right reason; designs multi-device sync as a natural consequence of the per-device cursor model rather than a bolt-on; raises what end-to-end encryption forecloses (server-side search, content moderation) as a real product constraint the design has to respect.

Further Reading

• WhatsApp Engineering: Sharding and Scaling Erlang (architecture retrospectives)
• Discord: How We Scaled Elixir to Handle Millions of Concurrent Users
• Signal: The Sesame Algorithm (multi-device end-to-end encryption)
• Alex Xu — System Design Interview Vol. 1 (chapter on chat systems)