To our users we're a chat app, judged against ChatGPT · WhatsApp · WeChat — but our server sits in Singapore and most users are in China, behind a ~200 ms wire. This note is one place: why the app felt slow (measured), what WeChat & WhatsApp do to stay responsive on bad networks (sourced from their engineering), a scorecard of our app against that playbook, and what's left. It folds together the former Perceived speed study and the Slow-network playbook.
chat.xbbapp.com, v179–v187). The app no longer waits on the wire: a one-round-trip cold open (/api/boot), a persistent per-user cache painted before any network, optimistic mutations + new-chat, prefetch, live-turn recache, access-loss eviction, a reliable send outbox (retry + tap-to-retry, idempotent /say), delta /replay sync, and value-aware/byte-budgeted cache sizing. The one frontier that remains is pure transport — the ~1.4 s cold TLS handshake — see §6 What's left.Timed against the live box (a client on the China path), each request, cold:
| Metric | chat.xbbapp.com (SG) | localhost:8011 |
|---|---|---|
| TCP connect (1 round-trip) | ~205 ms | ~0 ms |
| Cold request incl. TLS handshake | ~1.40 s | ~2 ms |
| Fetching a 1.5 KB static file | 1.38 s | instant |
| Server compute (uvicorn) | ~0 ms | ~0 ms |
A 1.5 KB manifest taking 1.38 s proves the cost is round-trip latency + TLS setup, not payload and not the server. So the only thing that matters is how many times, and how synchronously, the app touches the wire. The original cold open chained ~5 serial round-trips before the first room painted — and fetched /api/rooms twice:
/api/boot) painted over a warm cache — see §4. On localhost the same six calls totalled ≈ 6 ms; only the wire ever changed.| What you felt | What it waited on | The fix (shipped) |
|---|---|---|
| Chat rows load slow | refreshChats() wiped the list, then waited a round-trip | persistent list cache painted instantly, deduped + parallel boot |
| Skeleton → bubbles | switchTo → /state + /replay, in-memory cache lost on reload | persistent room cache + prefetch → cached rooms open instantly |
| Notes load slow | loadRoomNotes → blank pane until the round-trip | notes SWR — cached clips paint at once |
| Delete-note lag | deleteNote awaited the DELETE before removing the card | optimistic — vanishes on tap, reconciles behind |
| "Caches gone after an update" | in-memory Map + no-store GETs; SW cached the shell but bypassed /api/* | data layer mirrored to localStorage per user; survives a reload + deploy |
The same tap, two architectures. The old UI was frozen until the wire answered. Local-first, the UI answers the user's intent immediately and lets the server catch up — this is the shape of every fix below.
Both apps run on hundreds of millions of bad connections. Strip away the branding and they converge on the same moves; this is the model we built toward.
WhatsApp keeps every message in a local SQLite DB — the UI renders from it, works fully offline, reconciles in the background (local-first design). History is an append-only log; ordering is resolved by server timestamps (offline & sync). WeChat's local store plays the same role behind its Mars stack.
The message appears instantly with a local id and a “pending” status, before any confirmation; then WhatsApp's ticks — ✓ the server has it, ✓✓ delivered, blue ✓✓ read — so you're never left guessing on a crawling link. The general pattern (Simon Hearne): animate on tap, run the request in parallel, and on failure queue a retry — only undo after several failures.
When offline, WhatsApp writes the message to a pending queue and drains it in order on reconnect, deduped by a local id (store-and-forward). WeChat's Mars/STN keeps a long-link (persistent socket) for push + a short-link (HTTP) for big requests, with a smart heartbeat (smart_heartbeat.cc) and delta sync on reconnect. The cold-start tax is attacked with 1-RTT/0-RTT handshakes (mmtls / QUIC — Citizen Lab) and “complex connect” IP-racing. And media is decoupled from text — thumbnails first, download-on-tap via CDN, so a big file never blocks message sync.
| Technique | What they do | Us | Where we stand |
|---|---|---|---|
| 1 · LOCAL-FIRST STORE | |||
| Local store = source of truth | read local, render now, reconcile in background | done | per-user localStorage + roomCache, painted before any network on boot; recacheRoom keeps it live on every turn. The core, and it's solid. |
| Append-only + server authority | append-only log, server timestamps order it | done | /replay is append-only; a no-shrink guard stops a stale read shrinking the cache; the server is the clock. |
| Evict on access-loss | (the mirror of caching — un-show what you can't see) | done | evictRoom + a removed event + a 403/404 guard + a reconcile to the authoritative list (v184). |
| Cache sizing by real limits | hold a lot; don't blow the device | done | value-aware eviction (opened rooms protected from prefetch); 64 in-memory; persisted by a ~3.5 MB byte budget (v186–v187). |
| 2 · OPTIMISTIC UI + STATUS | |||
| Optimistic echo + actions | message/edit appears instantly, before the server | done | user echo + the conferring “egg”; note add/delete, rename, archive/trash, member add/remove, new-chat — all optimistic with rollback. |
| Pre-emptive loading | prefetch on hover / touchstart | done | row hover/touch prefetch plus a capped idle prefetch of the most-recent rooms. |
| Send status + retry | pending → sent; failed = a tappable retry | done | v185: auto-retry with backoff → a tappable red “!”, text kept. By choice, no sent/read ticks — failure-only. |
| “Only undo after N failures” | retry-queue an action before reverting | done | sends auto-retry before showing the “!”; the durable outbox re-drains on reconnect. |
| 3 · DURABLE DELIVERY | |||
| Outbox: queue + drain on reconnect | offline sends persist, flush when back | done | v185: a failed send is persisted (survives reload, re-painted with its “!”) and auto-resends on the online event + user-stream re-open. |
| Idempotency / dedup | message ids so a retry can't duplicate | done | /say is idempotent on the nonce (server-side) → recorded exactly once across retries + drain; lines carry a stable lid. |
| Server store-and-forward | hold messages for an offline client | done | turns persist server-side; a missed turn replays via /replay on return (and reconcileRoom on stream re-open). |
| 4 · CONNECTION | |||
| Persistent push connection | long-link socket for real-time push | done | SSE — a per-room stream + an always-on per-user stream (unread dots, removed, activity). |
| Delta sync on reconnect | fetch only what was missed | done | v185: /replay?since=<total> returns only new turns; the client merges onto the cached base (full-fallback if the base diverged). Cuts the payload on long rooms. |
| 0-RTT setup + connection racing | mmtls/QUIC 1-RTT, parallel-IP “complex connect” | to build | the ~1.4 s China→SG TLS cold handshake — the one frontier left. HTTP/2 + h3 advertised but unconfirmed. See §6. |
| Smart / adaptive heartbeat | longest ping that keeps the link alive | n/a | fixed 20 s SSE ping; fine at our scale. Revisit only if battery/idle-drop becomes a real complaint. |
| 5 · PAYLOAD & MEDIA | |||
| Small payloads, compression, batching | protobuf, gzip/brotli, batch small msgs | partial | turns are batched; JSON (fine for our size); gzip only. Quick win: enable brotli in Caddy — free shrink on text. See §6. |
| Bandwidth-aware behaviour | back off on 2G / Save-Data / metered | to build | Quick win: gate the idle prefetch on navigator.connection.effectiveType / saveData so we don't warm rooms on a 2G link. |
| Media decoupled from text | thumbnails first, download-on-tap, CDN | n/a | no heavy media yet — text + client-rendered diagrams (small, already off the text path). Apply the rule when images/files land. |
| 6 · PERCEIVED POLISH | |||
| Skeletons / interstitial | structure shows while the body loads | done | shimmer skeleton for an uncached room body; the “egg” as a live placeholder. |
| Clear failure + retry affordance | a red “!” you can tap to resend | done | the tappable “!” on a failed send (v185). |
A fast cache is worthless if it's wrong. Four consistency problems came with the speed, and all four are now closed.
A reply that arrives while you're in the room used to render but never reach the cache, so switching away and back showed a stale snapshot until /replay caught up. Now recacheRoom(rid) re-snapshots from the authoritative /replay (delta, via ?since=) on every content event — for the open room and for an already-cached background room when its activity ping arrives — so the cache is never behind what you've seen.
An owner removes you from a room, but your client still had it cached — so the chat lingered, opened from the snapshot, then went empty / “New chat” the moment the server started 403-ing. Fixed with three layers:
| Event | Server | Your client |
|---|---|---|
| Owner removes you | removed → your user stream | evict from list + cache; bounce out if open + toast |
| Room deleted | removed → each member (captured before the cascade) | same eviction, live |
| Rename / archive / cast / model | activity → all members | list title/status + open view refresh live |
| Any room GET → 403/404 | the membership gate | getRoomJSON throws accessLost → evict |
| Offline removal / missed event | — | reconcileCachesToList: a fresh /api/rooms prunes any cached room it no longer lists |
Defence in depth: a live push (instant), a 403/404 guard on every room fetch (catches a click racing the push), and a reconcile against the authoritative list on every refresh (catches anything missed while the tab was closed).
The two above are about rooms. The one-round-trip cold open carries more than rooms, and the rest of it had no story at all: /api/boot hands the client its config — the persona library, the model list, the palette, the admin dials — and applyConfig wrote it once. There was no second read anywhere in the app. Every surface drawn from the library was therefore a photograph of it taken the last time that device cold-started.
Reported on 2026-08-17, and worth keeping because of how it reads from the outside: “I created a persona, I can talk to her, but I can't find her in the contact list, anywhere, on my phone — on desktop I see her in all lists.” That sounds like a contradiction and isn't. A chat is server-truth on every open, so talking to her worked; the lists came from the boot photograph, and the phone's was older than she was. The server was right throughout — the follow row was written at build time, the card was discoverable, and both of her rooms had exactly one member, so the phone was the same account. Nothing was broken except that one payload was never asked for twice.
| Trigger | What it costs | What it catches |
|---|---|---|
Foreground (visibilitychange → visible) | a 304, at most once per 20 s | the reported case: a phone resumed rather than relaunched |
Stream reconnect (esUser.onopen) | forced — the throttle must not swallow it | a push sent while the stream was down was sent to nobody |
| A library page opening (All persona · picker · search · Friends) | a 304, behind the paint | you are about to read the list — the last moment it is cheap to check |
The library push | forced | build · hidden · visibility · clone · delete · discard · purge · restore — near-instant, on every open device |
What makes the cheap triggers affordable is the validator. /api/config now carries an ETag and the client sends If-None-Match by hand — the SW passes /api/* straight through, so leaning on the HTTP cache would put the library's freshness in the hands of heuristic caching. The common answer is a 304, which is the only reason foregrounding can afford to ask at all on the very link that made the bundle attractive. The boot payload ships the same validator, so even the first revalidation after a cold open is conditional.
Two deliberate choices behind it. The ETag hashes the whole payload rather than keying on the persona roots' mtimes: a root's mtime does not move when a profile.md inside it is edited, and half these fields are DB settings that touch no file — hashing what we are about to send is exact by construction and cannot be forgotten by whoever adds the next field. And the push is a broadcast carrying no card: the payload is per-user filtered by may_discover, so deciding whose library a change touched would re-derive that predicate in a second place. Everyone hears something moved and asks for their own copy; the ones it didn't touch get a 304.
Client-side the repaint is a registry (onLibraryChange), not a list of call sites: the surfaces live in their own closures, and naming them from the refresher is how the sixth one gets left out. Each page registers its own repaint next to its own render, and answers “am I even open?” itself.
Reported on 2026-08-20: “is FP working in room d233? why can't I see the FP bubble even when I turned it on in the admin console?” The floor producer was working — sixteen staged turns on record, arm v1f, ~$0.0003 each — and the console switch was on. The bubbles were missing from a chat that had already been opened, and no amount of reloading brought them back.
fp_bubbles decides whether fp rides at all (_fp_visibility gates it server-side, on the live turn and the replay alike).Why nothing caught it. Flipping the switch doesn't change a single turn's identity — so every test we had agreed with the stale copy, and each was right on its own terms:
| The test | What it asked | Its answer, and why it was wrong |
|---|---|---|
replaySince | how many turns do I already have? | all of them → ask for a delta. The server truthfully returned none. |
mergeReplay | splice the new tail on | there was no tail → the old, field-short bodies stand. |
changed (in switchTo) | did the count or the title move? | neither → don't repaint. The stream was never even rebuilt. |
So the bubbles never appeared on any turn that predated the flip, for the life of the cache — and the cache outlives a reload, which is why reloading, the one thing anyone tries, didn't help either. Only a brand-new turn carried its fp, arriving live.
The fix: the snapshot carries the shape it was written under. cacheShape() is one line naming every payload-shaping switch; it is persisted beside the room bodies and compared on every config apply. A mismatch marks those rooms, and a marked room's replaySince returns 0 — so the next open reads the room whole instead of its tail, and changed is forced true because nothing else can know the paint is owed. The mark is cleared at the single write point, cacheRoom, whenever a full replay lands.
Two choices worth keeping. It marks, it does not drop: the bodies are still the right thing to paint instantly on a cold open — the whole point of the cache on the China→SG wire — they are just a field short, and the full replay landing a moment later repairs them in place. And a snapshot written before this existed carries no stamp at all, which reads as unknown — a mismatch — so the first load after the ship re-reads once and self-heals, with no cache-version bump and no lost instant paint.
The room actually on screen is re-read on the spot rather than on the next switch: an admin who flips the switch in the console and comes back to the tab triggers the §5 foreground revalidation, and is owed the change there. Every other room repairs lazily, on open, off the mark alone. Verified on :8016 against a room with four staged turns: switch off → 0 cards and a snapshot stamped -; switch on + reload → 4 cards; a snapshot whose stamp lies that it is current → back to 0 (the bug, reproduced); an unstamped legacy snapshot → 4; and a mid-session flip, both directions, with no reload at all.
The local-first half of the playbook is done and live. What remains is pure transport plus a couple of quick wins — none of it changes the architecture, all of it chips at the raw wire.
The biggest remaining cost is the first packet. Two levers: confirm HTTP/3 (QUIC) is actually used (the box advertises alt-svc: h3=":443", but the China path may block UDP/443 — if it falls back to h2 we never get QUIC's 0-/1-RTT + loss recovery); and front the box with an edge/CDN that terminates TLS near the user and pools a warm long-haul connection to Singapore. WeChat solves this with mmtls 0-RTT + IP-racing; an edge is our pragmatic equivalent. (Mind the cert / grey-cloud caveat in deployment topology.)
Brotli in Caddy (free shrink on all JSON/HTML over the wire); bandwidth-aware prefetch — gate the idle room-warming on navigator.connection / saveData so a 2G or metered link isn't flooded.
Media discipline (thumbnails, download-on-tap, CDN) the moment images/files land — today we're text + small client-rendered diagrams. Adaptive heartbeat only if idle-drop / battery becomes a real complaint; the fixed 20 s SSE ping is fine at this scale.
The owner's rule, and it is the whole section: loading is fine as long as it is expected and explicit — unexplained loading reads as a bug. The local-first work above closed the blank window; what was left was the half-dressed one, where the app is up, drawing, and drawing the wrong thing. Three of them, all shaped the same way — a control whose position or presence is decided by data that has not landed yet.
Dictation is gated on config.stt, which arrives with /api/boot. Without the class the mic is display:none and send stands in its slot; with it, send is hidden while the box is empty and the mic stands there instead. So every cold open drew a send button, then swapped it for a microphone a beat later, with nothing to explain the swap — worst on the long SG link, where the beat is seconds. stt-on now rides the same pre-paint mirror that already kills the flash of admin-hidden tabs, so the composer boots in the shape it is going to keep.
The composer drops its buttons to a second row once text passes one line — a width judgement. An unlaid-out row measures zero, at which point a two-word draft "wraps", the two-deck engages, and it stays engaged: the only path that took it back off ran on the next keystroke. A chat opened while its pane was still being sized therefore sat in the wrong shape until you typed into it. It now refuses to decide without a width (falling back to the single deck rather than inheriting the previous room's), and re-decides when the width arrives — a held ResizeObserver for the pane transition and the split drag, with resize/orientationchange beside it as an event-driven belt.
The Me hero's name, monogram and handle are painted by setCurrentUser, which cannot run until boot answers — and its own cache could not help, because that paint is gated on ME being known. Identity is the one thing about Me that is stable between sessions, so it now paints from the last one in the instant-paint pass, and the wire corrects it. Identity only, never the numbers: energy, activity and voices are exactly the part that may have changed while you were away, and a stale number shown as current is a worse lie than a blank one.
Tencent — Mars (WeChat's cross-platform network component) · STN smart heartbeat · long-link
Citizen Lab — Security analysis of WeChat's MMTLS encryption protocol (1-RTT handshake; “move to QUIC”)
WhatsApp local-first — Your phone is a data center · How WhatsApp works without internet
Optimistic UI & queues — Simon Hearne — Optimistic UI patterns · Message-queue best practices (idempotency, retry)
chat.xbbapp.com) · grounded in lib/room-ui.html, lib/run_room.py, lib/sw.js — see also Rendering & speed · Deployment topology · Roadmap