To our users we're a chat app, judged against ChatGPT · WhatsApp · WeChat — but our server sits in Singapore and most users are in China, behind a ~200 ms wire. This note is one place: why the app felt slow (measured), what WeChat & WhatsApp do to stay responsive on bad networks (sourced from their engineering), a scorecard of our app against that playbook, and what's left. It folds together the former Perceived speed study and the Slow-network playbook.
chat.xbbapp.com, v179–v187). The app no longer waits on the wire: a one-round-trip cold open (/api/boot), a persistent per-user cache painted before any network, optimistic mutations + new-chat, prefetch, live-turn recache, access-loss eviction, a reliable send outbox (retry + tap-to-retry, idempotent /say), delta /replay sync, and value-aware/byte-budgeted cache sizing. The one frontier that remains is pure transport — the ~1.4 s cold TLS handshake — see §6 What's left.Timed against the live box (a client on the China path), each request, cold:
| Metric | chat.xbbapp.com (SG) | localhost:8011 |
|---|---|---|
| TCP connect (1 round-trip) | ~205 ms | ~0 ms |
| Cold request incl. TLS handshake | ~1.40 s | ~2 ms |
| Fetching a 1.5 KB static file | 1.38 s | instant |
| Server compute (uvicorn) | ~0 ms | ~0 ms |
A 1.5 KB manifest taking 1.38 s proves the cost is round-trip latency + TLS setup, not payload and not the server. So the only thing that matters is how many times, and how synchronously, the app touches the wire. The original cold open chained ~5 serial round-trips before the first room painted — and fetched /api/rooms twice:
/api/boot) painted over a warm cache — see §4. On localhost the same six calls totalled ≈ 6 ms; only the wire ever changed.| What you felt | What it waited on | The fix (shipped) |
|---|---|---|
| Chat rows load slow | refreshChats() wiped the list, then waited a round-trip | persistent list cache painted instantly, deduped + parallel boot |
| Skeleton → bubbles | switchTo → /state + /replay, in-memory cache lost on reload | persistent room cache + prefetch → cached rooms open instantly |
| Notes load slow | loadRoomNotes → blank pane until the round-trip | notes SWR — cached clips paint at once |
| Delete-note lag | deleteNote awaited the DELETE before removing the card | optimistic — vanishes on tap, reconciles behind |
| "Caches gone after an update" | in-memory Map + no-store GETs; SW cached the shell but bypassed /api/* | data layer mirrored to localStorage per user; survives a reload + deploy |
The same tap, two architectures. The old UI was frozen until the wire answered. Local-first, the UI answers the user's intent immediately and lets the server catch up — this is the shape of every fix below.
Both apps run on hundreds of millions of bad connections. Strip away the branding and they converge on the same moves; this is the model we built toward.
WhatsApp keeps every message in a local SQLite DB — the UI renders from it, works fully offline, reconciles in the background (local-first design). History is an append-only log; ordering is resolved by server timestamps (offline & sync). WeChat's local store plays the same role behind its Mars stack.
The message appears instantly with a local id and a “pending” status, before any confirmation; then WhatsApp's ticks — ✓ the server has it, ✓✓ delivered, blue ✓✓ read — so you're never left guessing on a crawling link. The general pattern (Simon Hearne): animate on tap, run the request in parallel, and on failure queue a retry — only undo after several failures.
When offline, WhatsApp writes the message to a pending queue and drains it in order on reconnect, deduped by a local id (store-and-forward). WeChat's Mars/STN keeps a long-link (persistent socket) for push + a short-link (HTTP) for big requests, with a smart heartbeat (smart_heartbeat.cc) and delta sync on reconnect. The cold-start tax is attacked with 1-RTT/0-RTT handshakes (mmtls / QUIC — Citizen Lab) and “complex connect” IP-racing. And media is decoupled from text — thumbnails first, download-on-tap via CDN, so a big file never blocks message sync.
| Technique | What they do | Us | Where we stand |
|---|---|---|---|
| 1 · LOCAL-FIRST STORE | |||
| Local store = source of truth | read local, render now, reconcile in background | done | per-user localStorage + roomCache, painted before any network on boot; recacheRoom keeps it live on every turn. The core, and it's solid. |
| Append-only + server authority | append-only log, server timestamps order it | done | /replay is append-only; a no-shrink guard stops a stale read shrinking the cache; the server is the clock. |
| Evict on access-loss | (the mirror of caching — un-show what you can't see) | done | evictRoom + a removed event + a 403/404 guard + a reconcile to the authoritative list (v184). |
| Cache sizing by real limits | hold a lot; don't blow the device | done | value-aware eviction (opened rooms protected from prefetch); 64 in-memory; persisted by a ~3.5 MB byte budget (v186–v187). |
| 2 · OPTIMISTIC UI + STATUS | |||
| Optimistic echo + actions | message/edit appears instantly, before the server | done | user echo + the conferring “egg”; note add/delete, rename, archive/trash, member add/remove, new-chat — all optimistic with rollback. |
| Pre-emptive loading | prefetch on hover / touchstart | done | row hover/touch prefetch plus a capped idle prefetch of the most-recent rooms. |
| Send status + retry | pending → sent; failed = a tappable retry | done | v185: auto-retry with backoff → a tappable red “!”, text kept. By choice, no sent/read ticks — failure-only. |
| “Only undo after N failures” | retry-queue an action before reverting | done | sends auto-retry before showing the “!”; the durable outbox re-drains on reconnect. |
| 3 · DURABLE DELIVERY | |||
| Outbox: queue + drain on reconnect | offline sends persist, flush when back | done | v185: a failed send is persisted (survives reload, re-painted with its “!”) and auto-resends on the online event + user-stream re-open. |
| Idempotency / dedup | message ids so a retry can't duplicate | done | /say is idempotent on the nonce (server-side) → recorded exactly once across retries + drain; lines carry a stable lid. |
| Server store-and-forward | hold messages for an offline client | done | turns persist server-side; a missed turn replays via /replay on return (and reconcileRoom on stream re-open). |
| 4 · CONNECTION | |||
| Persistent push connection | long-link socket for real-time push | done | SSE — a per-room stream + an always-on per-user stream (unread dots, removed, activity). |
| Delta sync on reconnect | fetch only what was missed | done | v185: /replay?since=<total> returns only new turns; the client merges onto the cached base (full-fallback if the base diverged). Cuts the payload on long rooms. |
| 0-RTT setup + connection racing | mmtls/QUIC 1-RTT, parallel-IP “complex connect” | to build | the ~1.4 s China→SG TLS cold handshake — the one frontier left. HTTP/2 + h3 advertised but unconfirmed. See §6. |
| Smart / adaptive heartbeat | longest ping that keeps the link alive | n/a | fixed 20 s SSE ping; fine at our scale. Revisit only if battery/idle-drop becomes a real complaint. |
| 5 · PAYLOAD & MEDIA | |||
| Small payloads, compression, batching | protobuf, gzip/brotli, batch small msgs | partial | turns are batched; JSON (fine for our size); gzip only. Quick win: enable brotli in Caddy — free shrink on text. See §6. |
| Bandwidth-aware behaviour | back off on 2G / Save-Data / metered | to build | Quick win: gate the idle prefetch on navigator.connection.effectiveType / saveData so we don't warm rooms on a 2G link. |
| Media decoupled from text | thumbnails first, download-on-tap, CDN | n/a | no heavy media yet — text + client-rendered diagrams (small, already off the text path). Apply the rule when images/files land. |
| 6 · PERCEIVED POLISH | |||
| Skeletons / interstitial | structure shows while the body loads | done | shimmer skeleton for an uncached room body; the “egg” as a live placeholder. |
| Clear failure + retry affordance | a red “!” you can tap to resend | done | the tappable “!” on a failed send (v185). |
A fast cache is worthless if it's wrong. Two consistency problems came with the speed, and both are now closed.
A reply that arrives while you're in the room used to render but never reach the cache, so switching away and back showed a stale snapshot until /replay caught up. Now recacheRoom(rid) re-snapshots from the authoritative /replay (delta, via ?since=) on every content event — for the open room and for an already-cached background room when its activity ping arrives — so the cache is never behind what you've seen.
An owner removes you from a room, but your client still had it cached — so the chat lingered, opened from the snapshot, then went empty / “New chat” the moment the server started 403-ing. Fixed with three layers:
| Event | Server | Your client |
|---|---|---|
| Owner removes you | removed → your user stream | evict from list + cache; bounce out if open + toast |
| Room deleted | removed → each member (captured before the cascade) | same eviction, live |
| Rename / archive / cast / model | activity → all members | list title/status + open view refresh live |
| Any room GET → 403/404 | the membership gate | getRoomJSON throws accessLost → evict |
| Offline removal / missed event | — | reconcileCachesToList: a fresh /api/rooms prunes any cached room it no longer lists |
Defence in depth: a live push (instant), a 403/404 guard on every room fetch (catches a click racing the push), and a reconcile against the authoritative list on every refresh (catches anything missed while the tab was closed).
The local-first half of the playbook is done and live. What remains is pure transport plus a couple of quick wins — none of it changes the architecture, all of it chips at the raw wire.
The biggest remaining cost is the first packet. Two levers: confirm HTTP/3 (QUIC) is actually used (the box advertises alt-svc: h3=":443", but the China path may block UDP/443 — if it falls back to h2 we never get QUIC's 0-/1-RTT + loss recovery); and front the box with an edge/CDN that terminates TLS near the user and pools a warm long-haul connection to Singapore. WeChat solves this with mmtls 0-RTT + IP-racing; an edge is our pragmatic equivalent. (Mind the cert / grey-cloud caveat in deployment topology.)
Brotli in Caddy (free shrink on all JSON/HTML over the wire); bandwidth-aware prefetch — gate the idle room-warming on navigator.connection / saveData so a 2G or metered link isn't flooded.
Media discipline (thumbnails, download-on-tap, CDN) the moment images/files land — today we're text + small client-rendered diagrams. Adaptive heartbeat only if idle-drop / battery becomes a real complaint; the fixed 20 s SSE ping is fine at this scale.
Tencent — Mars (WeChat's cross-platform network component) · STN smart heartbeat · long-link
Citizen Lab — Security analysis of WeChat's MMTLS encryption protocol (1-RTT handshake; “move to QUIC”)
WhatsApp local-first — Your phone is a data center · How WhatsApp works without internet
Optimistic UI & queues — Simon Hearne — Optimistic UI patterns · Message-queue best practices (idempotency, retry)
chat.xbbapp.com) · grounded in lib/room-ui.html, lib/run_room.py, lib/sw.js — see also Rendering & speed · Deployment topology · Roadmap