Fact sheet
| What | Independent test of consumer AI shopping agents available in the US: same tasks word for word, scored against the evidence |
|---|---|
| Agents (Round 1) | Muse (Meta); Amazon Alexa for Shopping (incl. Buy for Me); Google Gemini app and AI Mode in Search (scored separately); Microsoft Copilot; Perplexity; ChatGPT (confirmation screen only) |
| Tasks | 8 (T01, T02, T03, T04, T06, T07, T08, T10) |
| Real orders | T02, T03, T10, each with one planned second pass |
| Scoring | S0–S4; weights 0/1/3/10/25; raw counts published |
| Second scorer | Random 30% of runs; sample locked by a pre-published SHA-256 seed fingerprint |
| Testing | Pre-registered window October 9–22, 2026; openly extended at 7:13 AM ET on October 23 to 11:59 PM ET that day (to finish T10, T02 and T03); price watch to October 28. All changes are dated in the changelog |
| Results | Mid-November 2026 |
| Right of reply | Each company gets 7 days with its results and evidence before publication |
| Data | Run log (CSV) and redacted evidence, CC BY 4.0. Credit "Agent Mistakes Index" |
| Funding | Self-funded. No sponsors, affiliate links or freebies |
| Tester | Round 1 was run by one person in the United States. Identity not published |
| Contact | agentmistakesindex@gmail.com |
Not claimed: that it's the first test of AI shopping tools, or that it ranks a "best" agent.