Research · Study 03 · Paper under review
Can AI shopping agents buy from online stores?
Where stores let them in, AI agents can shop, but there's no trusted way for them to pay.
Getting in
37% of the stores we tested stopped the agent before it reached payment (175 of 469 stores).
Changing the model
Changing the AI model did not help an agent get in. Stores treated each agent largely the same: 413 of 427 stores gave Claude, GPT and Gemini the same result.
Paying
At Shopify stores that offer UCP, the agentic checkout route, the agent could set up a checkout almost every time (195 of 197), but every checkout stopped before payment. 181 came back requires_escalation, which UCP tells the agent to hand to the buyer. The other 14 came back ready_for_complete, but completing a checkout needs Shopify's permission to call complete_checkout, which the script did not have and never tried.
Add-ons
Of the 162 runs with a paid add-on on the payment page, 47 (29%) ended with it in the cart, such as package protection, returns coverage, a protection plan, or a membership. We ran a follow-up test on a checkout we built, with a $3.98 add-on already ticked. GPT and Gemini paid for it every time, and Claude removed it every time.
Three runs
Stitched from the screenshot saved at every step. Captions show what the agent did next.
Method and limits
Stores. We picked 500 popular US online stores, including large national retailers and smaller well-known brands. Just under half run on Shopify (222) and the rest on other e-commerce platforms (278), like Salesforce Commerce Cloud or their own custom builds. Our selection also includes all 55 of the NRF Top 100 retailers that sell online. When measuring results, we split results into 4 tiers ranked by popularity, using the Tranco list (Le Pochat et al. 2019).
No purchases were completed: no agent completed submitting payment, so this measures how often agents reached the payment step rather than completed purchases; the test-checkout follow-up covers the card step only.
Undeclared automation, independent agent: the agents drove a standard Chrome browser through automation software and did not identify themselves, and we tested with no platform partnership. Agents that run inside a person's own browser or sign their requests were not tested, and routes where approved agents can pay exist but none is open to an unaffiliated agent.