Online store — the model build-off
We dogfood every PlanSmith planner across multiple coding models. We lock the vertical, the discovery answers, and the feature ledger, then audit what each model builds. The planner is the product — these are the storefronts it produced.
How the builds ranked.
Harbor & Pine
Best OverallThe top build — fifteen lifecycles, no failures found.
- Best at
- We drove all fifteen workflows end to end without a single failure. The most careful correctness work of the session: going back to a paid checkout page never charges you twice, two shoppers cannot both buy the last item, price changes are locked to the owner in three separate places including spreadsheet import, and unpaid cash-on-delivery orders are never counted as money received.
- Watch for
- Its own hostile testing found seven correctness and permission bugs — cancelled orders could still be shipped, and catalogue staff could change prices by importing a spreadsheet.Fixed since audit — all seven were fixed before release and held up under repeat attack — shipping a cancelled order is now refused by the server, and price changes smuggled in via import are ignored.
Alder & Stone
Best Payment EngineeringStrongest money handling and the tightest permissions.
- Best at
- The most rigorous money handling here — a test payment provider covering confirmed, declined, pending and abandoned outcomes, where a confirmed order is recorded exactly once and a repeated confirmation cannot charge it again. It also has the tightest permissions we tested: catalogue staff cannot reach customers, orders, payments or exact prices, spreadsheet imports refuse to carry prices or stock at all, and accepting a return never quietly restocks or refunds by itself.
- Watch for
- It was built in stages and initially stopped three steps in, so discounts, returns, reviews, imports and payments simply did not exist at first audit — which it reported honestly rather than claiming to be finished.Fixed since audit — the remaining batches were authorised and completed; all fifteen features are now implemented and the payment loop closes end-to-end.
Northstone Goods
Widest CheckoutBest when payment-method breadth matters.
- Best at
- The widest choice at checkout — card, PayPal and pay-on-delivery — with a card simulator that accepts and declines on the real test numbers, and orders that end up honestly marked paid, failed, abandoned or part-refunded. Stock only leaves the shelf once payment actually succeeds, and a discount code recalculates tax on the reduced price rather than the original.
- Watch for
- Its customer list — names, emails, addresses and order history — was readable by catalogue and warehouse staff, contradicting the build's own permissions table. It also has no reports page; sales live in a dashboard panel and an export instead.Fixed since audit — the customer list is now owner-only and enforced on the server — other staff are turned away with nothing revealed. The missing reports page is still outstanding.
Models tested: Claude Fable 5, GPT-5.6 Sol, and Grok 4.5 — three builds of the same fifteen-feature storefront planner (catalog through payment, returns and imports), each run to full scope in batch mode. Scores come from driving the builds live in a browser and measuring each against its OWN selected scope, not one build's architecture imposed on the others. Every build here takes a real order end-to-end; all three keep payment in an honest test mode, because no live card credentials were issued for a demo. This is not a universal ranking of coding models; it is a PlanSmith benchmark for these specific storefront dogfood runs. These scores grade correctness and completeness — not visual design.