Plansmith
PlanSmith Benchmark

Helpdesk & ticketing — the model build-off

We dogfood every PlanSmith planner across multiple coding models. We lock the vertical, the discovery answers, and the feature ledger, then audit what each model builds. The planner is the product — these are the support desks it produced.

2026-07-314 models · 4 builds audited
The build-off · 01

How the builds ranked.

01

Alpen & Oak Support Desk

Best Information Isolation
by Claude Fable 5
88/100

The strictest handling of private information.

Best at
Private stays private, even when we went looking for ways in. An internal note never reaches the customer's browser at all, a different customer gets nothing, and — the detail most builds miss — a file attached to an internal note is refused even to the customer whose ticket it is, while staff can still open it. A missed response deadline stays on the record permanently instead of quietly clearing itself once someone finally replies.
Watch for
It did not compile for production — an error that only appears outside development, so its own browser testing could never have caught it. Deploying it also exposed a database location that only worked from its own folder.Fixed since auditboth were corrected for the live demo; it now compiles and runs from a normal deployment.
Open live demo
02

Signal Desk

Most Complete Workspace
by Grok 4.5
86/100

The broadest working agent workspace.

Best at
The most finished day-to-day workspace — inbox queues, saved views, bulk actions, a knowledge base, satisfaction scores and manager reporting — with permissions enforced on the server for every action rather than by hiding buttons. Customer separation and internal notes both held up when we tested them directly.
Watch for
A missed response deadline was wiped the moment an agent finally replied, so a manager's report could show zero breaches after a month of them. A customer could also attach a file to another customer's internal note. Its own report mentioned neither.
Open live demo
03

Mira Support

Most Disciplined Engineering
by GPT-5.6 Sol
85/100

Strongest engineering; honest about what it had not finished.

Best at
Private notes are filtered out in the database query itself, so an internal note never even reaches a customer's browser — the strongest version of that protection. Live chat, email threading, protection against two agents overwriting each other, and an AI assistant that switches itself off and says why rather than inventing an answer when nothing is connected.
Watch for
At audit you could not sign in to it anywhere except a developer's own machine, and only two-thirds of its chosen features were finished — both of which it reported honestly rather than claiming to be done.Fixed since auditfull password sign-in with server-side sessions was added and independently re-verified, and the remaining features were reconciled.
Open live demo
04

Northstar Support Desk

Fast-Tier Build
by Gemini 3.6 Flash
62/100

A working desk from a small model — but it shipped with no sign-in.

Best at
For a small, fast model this is a genuinely working support desk: real ticket queues, a manager dashboard built from live data, working exports, and — notably — the permission rules themselves were written correctly.
Watch for
It shipped with no sign-in at all. Adding a word to the web address granted administrator access, which made every one of those correct permission rules unenforceable. It was labelled ready for production regardless.Fixed since auditpassword sign-in, protected sessions and a real sign-out were added after our audit and independently re-verified before this demo was published.
Open live demo

Models tested: Claude Fable 5, Grok 4.5, GPT-5.6 Sol, and Gemini 3.6 Flash. These scores are ranked on the axis a helpdesk lives or dies by: keeping information where it belongs. For every build we planted a real internal note, then checked as the customer who owns that ticket — in the served HTML, not just the visible page — and as a different customer entirely. We also checked whether a missed first-response SLA survives in the manager's report once an agent finally replies, and whether an attachment on an internal note stays private. Scores are as-found at audit; later fixes are noted as “fixed since audit” but do not change the recorded number. Note that Gemini 3.6 Flash is a fast, small-tier model and is not directly comparable to the larger models here. These grade correctness, not visual design.