Research

What AI models believe, and how they reason

I hope to bridge the gap between AI and the real world. Each study judges AI by real outcomes, and a finding that holds up against the real world stands on its own, however it was found. Next is interpretability: using what the models provide to explain why the best and worst performers decide the way they do. AI is just the means.

01 · Belief Stability02 · Agentic Checkout03 · Legal Reasoning

01 · Belief Stability

In progress

Do AI assistants have stable brand preferences?

When a buying question is reworded, the recommended brand often changes. This study asks whether the model has a stable preference underneath the wording.

A · IF WORDING IS ALL THERE ISB · IF THERE IS A BELIEFOne buying question, asked 15 waysThe same question, asked 15 waysFour different brands, no signalOne brand keeps winning, a beliefeach dot = one answer · color = brand named
Ask the same buying question 15 ways and count how often the same brand comes back. Scattered picks mean the wording decides, a steady pick means there is a belief underneath
Results · October 2026
68 to 75%of a question's picks go to its most-picked brand, when the assistant is asked for one pick
About 3brands make up each assistant's shortlist for a buying question
Under 0.10change in consistency from asking for the pick in a follow-up turn, the preregistered test

See the resultsOne-page PDF

02 · Agentic Checkout

Paper under review

Can AI agents buy from online stores?

This study focuses on one core question: can AI agents actually make purchases at popular online stores? We sent four AI shopping agents (Claude, GPT, Gemini and Jev) to buy one item at each of 500 well-known US online stores (Jev at 250 of them).

STORES THAT STOPPED AN AI AGENT BEFORE PAYMENTTop 50k sites, Shopify0 of 17Top 50k sites, other63 of 8850k to 200k, Shopify4 of 6450k to 200k, other16 of 29First 250 stores, Claude · October 2026 · traffic rank from the Tranco top-million list
Shopify stores rarely stopped the agent, while stores on other platforms usually did: 8 of 218 against 167 of 251.
Results · October 2026
37%of the stores we tested stopped the agent before payment
413 of 427stores gave all three big models the same outcome
195 of 197checkouts set up through UCP; each one stopped before payment

Explore the resultsOne-page PDF