Jev, the decision model — can it seat the company? bench · 2026-09-22

Short answer: yes, for the shortlist; not for the whole job. Jev is real — TypeSafe AI's「decision model」, out 2026-09-15 — and it does one thing very well here: shown the whole roster of 105 personas and one brief, it ranks who the story belongs to, in under a second, for a hundredth of a cent. It cannot write the licence (it returns numbers, never words), it cannot find「the outsider」(asked for a far lens it names the watch repairer and Roger Federer on every story), and its yes/no on「does this touch the reader's own money, home, health or safety」is the best proxy gate we have tried. So: Jev makes the shortlist and gates the proxy seat, Flash makes the pick and cites the passage. Measured on the twelve 09-22 dev briefs, ≈0.25¢ a brief, ≈3¢ a morning.

1 · What Jev is

2 · The task, and the three ways tried

One brief in, two to four seats out: the expert (1–2, whose field this is), the outsider (0–1, another field with a lens on this one), the reader's proxy (0–1, the person on the street, only when the story touches the reader's own money, home, health or safety). Every seat must be licensed by a passage of the profile. The roster: the 92 repo personas plus 13 app-built ones (ink_brew.roster(), both roots), Tess excluded.

Judged three ways: (a) the seats the owner named in §1b (the crater: Hawking, Einstein; the mortgages: Xiao Pang, Max Rockbank, an outsider Dan Wang or Feddo, the proxy Xiao Pang) and, for the other ten, the seat a reader would expect (my read, marked as such); (b) whether the pick's licence is a passage that truly speaks to the subject, not merely a string found in the profile; (c) cost and seconds per brief. Everything below is one run, cached; the total spend was about six cents.

3 · The bench — twelve briefs, 2026-09-22

4 · What Jev did well, and what it did not

The expert ranking is the win.

One Choice over all 105 cards — name, role, tagline, eight tags — put the field's own persona first or second on every brief: Hawking on the crater, Xiao Pang on both housing stories, Max Rockbank on the three property listings, Moritz on both funding stories, Karpathy and Hinton on both Anthropic stories, Ambika Kumar second on the antitrust suit, Feddo second on Moscow. The tag shortlist, by contrast, put Shreyas Doshi, Jack Dorsey and Jony Ive on the mortgages, Stan Lee and Jeff Koons on Moscow, and no lawyer on the lawsuit — a word-overlap on 105 sets of tags is noisy, and Flash can only pick from what it is shown. Where Jev's list was clean, Flash's pick was clean (arm C).

The outsider question fails.

Asked「which persona from a field this story does NOT belong to would see it differently」, Jev names the retired watch repairer, Roger Federer and Rafael Nadal on nearly every brief, at confidence 0.1. It reads the question literally — the most unrelated card — and「a lens on this one」is not a thing it can weigh. Liu Cixin on the crater (0.14) was the one hit. A far seat needs the far candidates drawn in code and Flash asked whether any of them would add.

The reader-touched Noul is the best proxy gate tried.

0.86 on both housing stories, 0.08 on the crater, 0.11 on the Moon city, 0.20 on both funding rounds; the misses are Miami's $10 m sales (0.75) and the antitrust suit (0.76 — the subscription price, arguably). At ≥ 0.8 it switches the seat on for exactly the two stories it should. The code test in the baseline (a「my」or「should I」in the questions) switched it on for nine of twelve, and Flash then sat Xiao Pang on the Moon, on Moscow and on the Anthropic IPO.

The proxy Choice is weak.

Among the everyday personas Jev has no ground to stand on: a Tokyo student tops most briefs, and on the Bay Area mortgages Xiao Pang (0.30) ties with a lingerie company secretary (0.29). Keep the seat's candidates a curated list and let Flash pick, or default to the reader's own persona.

The passage-finder is half a tool.

One Noul per profile paragraph (24–34 a persona, $0.0003, 0.7 s) finds a real licence when one exists — Karpathy「at Anthropic since 19 May 2026」0.80, Musk's own trial 0.53, Xiao Pang「what changed him was the market」0.57 — and scores honestly low when none does: Churchill on Moscow 0.13, Hawking on a crater 0.35 (his top passage is the wheelchair over the toes). It also likes biography tables (Moritz's birthplace row, 0.71). As a second gate under Flash's quoted licence it would catch the far-fetched ones; as the sole licence it cannot, because it returns a number and the rule wants the words.

The string check is not the licence.

Both Flash arms pass the code check almost always because the model copies from the passages it was shown — and still cites「he cannot stand or walk unaided」to seat Roosevelt on drone warfare, or「the only child of a country store」to seat Charlie Rose on a duck club. Found in the profile ≠ speaks to the subject. The passage-finder's number is the cheapest second opinion.

Two smaller things.

Jev is not deterministic: the same four-option probe returned Hawking 0.53 / Einstein 0.47 twice and Einstein 0.51 / Hawking 0.49 the third time — take the top three, never only the top one. And a Choice caps at 255 options, so the roster fits in one question until it passes 255 personas; after that, chunk and merge.

5 · Cost and seconds, per brief

Flash measured at the off-peak sheet ($0.22 / M in, $0.66 / M out; the calls ran at 17:26 CST, inside the 2× window, so the morning's real figure is what is shown). Jev's cost is OpenRouter's own usage.cost. The Jev pick call carries the roster twice (once per Choice); dropping the outsider Choice halves it to ~9k tokens. Nothing here is a cost question: any arm is under a nickel a morning.

6 · The recommendation

Why not Jev alone: it cannot write the licence, the rule the owner set so a match is never a stretch; and it cannot find the outsider. Why not the baseline alone: its tag shortlist is the weak link — it never showed Flash the lawyer, or Feddo, or Max Rockbank on the Bay Area — and Flash cannot seat who it does not see. The hybrid is the same one Flash call the plan already budgets, with a cleaner list in front of it, for six-hundredths of a cent more.

7 · Open questions

8 · Status

Sources

TypeSafe: the launch post · System One · the API reference · models (limits, languages) · jev-1.13 rough edges · the 182-option cookbook. Independent: Simon Willison · liteLLM's bench · awesome-typesafe-jev (the ordering and calibration audits) · DataCamp. Gateways: OpenRouter · Vercel.