Real-Time Chat, Actually Explained

A real-time chat system's whole job is simple to say: get a message from one device to another, fast, and don't lose it. That's the entire promise — send, deliver, don't drop it.

Everything below this point is about honoring that promise once the recipient might be offline, might have three devices open at once, and might be one of five hundred people in the same conversation — the three complications a demo never shows you.

Before picking WebSocket vs. polling, it's worth being explicit about what this system actually owes every message — the same way you'd scope it out loud in an interview before drawing a single box.

Numbers here are explicit back-of-envelope estimates, not measured production values — but they're what determines that this design is a connection-tracking problem more than a raw-throughput one.

Before drilling into any one piece, here's the system end-to-end: a sender's message goes to a gateway, then a message service durably logs it and looks up presence to decide whether to push it live or leave it for reconnect-sync. The API contract and data model below are what every later section refines — nothing past this point replaces this picture, it only explains why each piece of it exists.

client_message_id, generated by the sending device, is what makes retries safe: if a send is retried after a flaky connection, the server recognizes the duplicate against recently-seen IDs for that conversation and returns the original sequence number instead of creating a second message — the same idempotency-key pattern that shows up anywhere a caller can't be sure its last request landed.

Every device tracks its own “last synced sequence number” per conversation rather than the server tracking “has this message been shown to this device” as a mutable flag — that single cursor is what makes reconnect-and-sync, multi-device, and offline delivery all the same mechanism instead of three separate ones.

Push alone raises the real question underneath it: no single server can hold every user's connection, so which node does a given user's connection live on right now — and what happens if they're not connected to any node at all?

The answer that holds up: add a presence layer mapping a user/device to a gateway node, if any, and make every message write to a durable, ordered per-conversation log regardless of whether the recipient is online. Online push becomes an optimization on top of a system that would still be correct if every push failed — not the primary mechanism with offline bolted on, matching the architecture shown earlier.

This is the part that's hard to picture from a diagram alone. A recipient can have several devices, and each one is either online right now or it isn't — that's the whole story. Toggle devices on or offline, send a message, and watch what happens to each one.

Every device gets the same treatment: the log write happens once, online devices get it pushed live, offline ones pick it up whenever they reconnect by asking for everything after their own last-synced sequence number. A device offline for two weeks catches up exactly like one offline for two seconds — multi-device sync isn't a separate feature, it falls out of the same mechanism offline delivery already needed.

The recurring theme: nothing on this list causes message loss, because the durable log write happens independently of every delivery mechanism above it — each failure degrades speed, not correctness.

If message content is end-to-end encrypted, the server is deliberately blind to it — which means features like server-side search or content moderation on message text simply aren't available, and any product requirement for them has to be solved client-side or explicitly traded away. Retention policy matters even for opaque, encrypted content: how long undelivered messages are held for an offline device, and how long delivery metadata is retained, are privacy decisions the architecture has to make room for, not afterthoughts.