← Design notes

Chat-grade rendering & speed

To our users we are a chat app. Fluency, speed, and render quality are judged against ChatGPT, Gemini, and Claude — not against a research tool. This is a survey of how the best chat apps render output and feel fast, what they do about the $ math-vs-money problem, and a grounded, prioritized plan to close the gap in lib/room-ui.html. Two fronts: ① render quality and ② real + perceived speed.

The bar isn't "looks fine in most cases." It's nothing ever renders wrong — a price isn't eaten by a math parser, a code block keeps its indentation, an equation is an equation, and a structural tag like <speak> is never seen by a human. And it's first words in under a second — not a clean answer after a long silence.

① Render quality — the SOTA pipeline

Every top chat app renders model output through the same four-stage pipeline. Ours is hand-rolled and covers only the first stage partially.

Stage SOTA tool What it does 1 · Markdown markdown-it / marked / remark GFM: headings, lists, tables, blockquotes, **bold**, `code` 2 · Math KaTeX (not MathJax — 10× faster) $…$ / $…$ → rendered equations 3 · Code highlight.js / Shiki / Prism syntax colors + copy button + lang label 4 · Sanitize DOMPurify strip any <script>/onclick the model emitted — treat model output as untrusted, always —

Where we are (as built, v52+): all four stages run, through one shared renderRich() + enhanceRich() pipeline used by host bubbles, the artifact pane, and dispatch briefs — mdToHtml for block markdown, KaTeX for math, highlight.js for code (theme-mapped), DOMPurify to sanitize. The libraries are vendored same-origin and lazy-loaded (China-safe, and the chat still works if one fails to load). What's not done is streaming — see the speed half below.

The recommended stack for a single-file app

We're vanilla JS in one HTML file — but the md-viewers already load marked.js from a CDN, so the pattern is established. Add, all via pinned CDN: KaTeX (+ its auto-render extension), highlight.js, and DOMPurify. This is exactly the Gemini/ChatGPT client stack; none of it needs a build step.

The `$` problem — math vs. money

This is the one you flagged. A single $ is ambiguous: $5 and $10 is money; $x^2 + 1$ is math. Naive renderers treat the first $…$ span as an equation and turn "5 and 10" into garbled italics. ChatGPT, Gemini and Claude all hit versions of this. The fix is three layers, belt-and-suspenders — don't pick one, do all three:

Layer 1 — prefer UNAMBIGUOUS delimiters (prompt the model). Instruct the panel in the system prompt to write math as $ … $ (inline) and \[ … \] (display) — never bare $. These have no money collision. This is what ChatGPT/Gemini do server-side. Render them as the PRIMARY math path. Layer 2 — accept $ only under STRICT rules (KaTeX's own rule). A $ opens math only if a non-space follows it; closes only if a non-space precedes it AND it is not immediately followed by a digit. So "it costs $5 and $10" → NOT math (digit after the closing $), but "$x^2$" → math. $$…$$ is always display math. Layer 3 — a literal dollar is \$ , and normalize the dialects. Rewrite the model's  \[ \] into the renderer's delimiters before parse; honor \$ as an escaped, literal dollar sign.

Net: you stop struggling over whether a $ is KaTeX or money by not relying on bare $ in the first place — the model is steered to bracket delimiters (unambiguous), and the conservative $ rule is a safety net for pasted content, tuned so prices survive. KaTeX plugins (markdown-it-katex, texmath) already implement the Layer-2 rule; the Layer-1 prompt line is one sentence in the actor system prompt.

our angleOur hosts are characters, not assistants — most won't write LaTeX. So Layer 1 matters less for, say, Confucius, but a lot for Euler, Tao, Wolfram, Lovelace. Add the bracket-delimiter instruction to the actor SP once (it's persona-agnostic); it costs nothing for hosts who never do math and saves the math-heavy ones from the $ trap.

Never expose `<speak>`

<speak who="…"> is our structural transport, not content — it multiplexes one model turn into per-host bubbles. A user must never see the raw tag. Today the parser extracts the inner text, so it's usually clean, but two paths can leak a stray tag:

The salvage path — when a turn has no parseable structure, we surface raw prose; a half-emitted <speak… can ride along.
Streaming (when we add it) — a tag arrives in pieces (<spea … k who=), and a mid-stream render would paint the fragment.

Recommendation: a defense-in-depth scrub at the render boundary — strip any residual </?speak…> and </?confer…> from bubble text right before it reaches the DOM, so no path can leak the tag even if upstream parsing slips. And when streaming lands, parse <speak> boundaries before rendering (materialize the bubble on the open tag, stream text into it, finalize on close) rather than rendering the raw buffer.

② Speed — real and perceived

The single most important fact about chat-app speed:

A 10-second answer that starts streaming immediately feels faster than a 3-second answer that appears all at once after 3 seconds of silence. The metric users feel is time-to-first-token (TTFT), not total time. Below ~200 ms TTFT an app feels instant; a multi-second blank feels broken.

Real speed — the levers, and what we already pull

Lever	Status in our app
Prompt caching (warm prefix → low TTFT)	✓ §8 append-only, 1h TTL — the prefix is warm after turn 1
Reasoning off by default	✓ fast variants are the default; greeting never reasons
Fast model tier available	✓ v4 Flash + the fast/reasoning split
Smaller / capped output	~ multi-host turns are inherently long; a floor producer would trim who speaks
Wire size (shell bytes over the China link)	✓ serve-time minified shell (828 → 514 KB source; `MAD_MINIFY=0` kill-switch, dev serves raw) + Caddy zstd/gzip → ~140 KB wire; shell ETag → 304, so SW revalidations of an unchanged build cost 0 B
Boot cost (server work per open)	✓ per-room meta.json sidecar (the chat list stops parsing full transcripts; boot 150–580 ms → ~80 ms) + a popularity memo; delta boot ships only the landing room’s tail (`land_since`)
Repeat + background loads	✓ `Cache-Control` 30 d on the vendored libs/fonts; idle prefetch is byte-aware (skips >80-round transcripts — they warm on tap/hover)
Streaming (token-by-token to the client)	✗ we block — the whole turn renders at once via the egg-hatch

We pull almost every real-speed lever already. The glaring exception — and the biggest single gap to SOTA — is streaming.

Perceived speed — the techniques

Technique	What it buys	Us
Token streaming	first words in <1s; the core "feels alive" signal	✗
Skeleton / "alive" placeholder	structure shows instantly; beats a spinner	✓ the egg (conferring bubble)
Optimistic echo of the user's own line	input feels instant, no round-trip wait	✓ we render the user bubble immediately
Fade-in / cross-fade on new content	no jarring jump or layout shift	✓ egg-hatch reveal animation
Stop button	control = perceived responsiveness	✗ (no mid-turn cancel)

We've quietly done most of the perceived-speed work — the egg is a good skeleton, the optimistic user echo is right, the hatch is a clean fade. But all of it still resolves only when the whole turn lands. The egg says "alive," then makes you wait for everything. Streaming would replace "egg → whole turn" with "first host's words within a second, the rest flowing in."

The hard part: streaming a multi-speaker turn

Single-assistant apps stream trivially — one text box, append tokens. Ours is harder because a turn is several hosts + a backstage confer + sometimes an artifact, currently delivered as one structured blob (the panel_turn tool JSON, or <speak> tags). Two honest sub-problems:

Structured output resists streaming. A tool-call's JSON arguments are only valid when complete — you can't render half a JSON object. The <speak> text format, by contrast, can be stream-parsed: when <speak who="euler"> arrives, open Euler's bubble; stream text into it; close it on </speak>; move to the next. So streaming favors the <speak> path over the tool — a real tradeoff against the tool's parse-reliability.
Streaming markdown is its own problem. Tokens arrive mid-syntax: an unclosed ``` fence, a half-written $$, a dangling **. Rendering the raw buffer each chunk flickers and breaks. The fix the field has converged on:

Streaming-markdown rules (Vercel Streamdown / Chrome AI guide): • Buffer + auto-close for rendering only — temporarily close an open **, `, or $$ so the partial parses; never mutate the real text. • Defer code blocks until the closing fence — or show an open code container with a "streaming" shimmer; unclosed fences are the ugliest break. • Multi-pass highlight — show code as plain text as it streams, then syntax-highlight once the block completes (zero perceived latency). • Append, don't re-set — use append()/incremental DOM, never re-assign innerHTML each chunk (kills layout-shift and re-highlight cost). • Sanitize the accumulated buffer every chunk (DOMPurify); stop on removal.

Where we stand vs. SOTA

shipped · productionSteps 1 + 2 below are shipped to production (KaTeX math, highlight.js code blocks, DOMPurify, full markdown in host bubbles, the $ money-vs-math fix, the <speak> scrub). Render quality is now at par with SOTA; the one remaining gap is streaming. Vendored same-origin (China-safe), not CDN.

Dimension	ChatGPT / Gemini / Claude	Us now	Gap
Streaming / TTFT	token-by-token, <1s	blocking whole-turn (egg)	★★★
LaTeX / math	KaTeX	KaTeX, vendored + lazy	✓
Markdown coverage	full GFM via lib	full block markdown in bubbles	✓
Code blocks	highlight + copy + lang	highlight.js + copy + lang (theme-mapped)	✓
Sanitization	DOMPurify	DOMPurify + esc()	✓
The `$` problem	delimiter rules	pairing rule — math & money both safe	✓
Skeleton / optimistic / fade	yes	egg + echo + hatch + timers	on par
Structural-tag hygiene	n/a (single assistant)	`<speak>` scrubbed at render boundary	✓

Roadmap

Ordered by value-per-effort. The first two are done (render-quality wins, no architecture change); the third — streaming — is the remaining perceived-speed transformation.

1 · The render stack✓ shipped · production

Done (v52). Vendored KaTeX + highlight.js + DOMPurify same-origin (China-safe, lazy, graceful — not CDN), behind one shared renderRich() + enhanceRich() pipeline for host bubbles, the artifact pane, and dispatch briefs. Host bubbles now render full block markdown (was inline-only); a unified .cb code block with language label + copy + theme-mapped highlighting. Math via KaTeX. A # FORMATTING note in the actor SP asks hosts to fence code + write math in LaTeX.

2 · the $ fix & <speak> scrub✓ shipped · production

Done (v52–v53). The $ money-vs-math problem is solved by the delimiter-pairing rule (markdown-it-katex's): a $…$ is math only when the closing $ sits tight against non-space and isn't followed by a digit — so $1$ / $x=\pi$ render while $5 and $10 stay money (an earlier digit-heuristic mis-paired and swallowed prose — fixed). $$…$$/$…$/\[…\] also render; \$ is literal. And stripStructural() removes any <speak>/<confer> at the render boundary on every path.

3 · Streaming the panel turnthe remaining one · high effort

Not started. Stream-parse the <speak> format: materialize each host's bubble on its open tag, stream text in, finalize on close — replacing "egg → whole turn" with progressive bubbles and sub-second first words. Requires the streaming-markdown rules above, a per-token SSE path from the model (both DeepSeek and Anthropic support it), and accepting the <speak>-text path over the structured tool for streamed turns. Pairs naturally with a stop button. This is the change that makes us feel like ChatGPT.

what's leftRender quality is at par — math, code, tables, sanitization, the $ trap, and structural-tag hygiene all shipped to production. The one frontier is streaming: a genuine architecture change to the turn-delivery path (the egg-hatch was built around whole-turn arrival). Plan it deliberately, and note the structured-tool tradeoff (streaming favors the <speak>-text path) before committing.

Sources

Chrome for Developers — Best practices to render streamed LLM responses
Vercel — Streamdown: streaming markdown renderer
Preventing flash of incomplete markdown when streaming (HN)
markdown-it-katex · NextChat — the mixed LaTeX/$ dollar-sign bug
AI Chat UI best practices (2026) · NN/g — Skeleton screens
Why faster first tokens matter more than total response time · Time to first token (TTFT)

Dialogue · research note · updated 2026-06-21 (steps 1–2 shipped to production; streaming remains) · grounded in lib/room-ui.html (renderRich/enhanceRich, egg-hatch) — see also Function map · Roadmap · Caching