Test Your Store With an AI Shopping Agent Before Black Friday

AI shopping agents now pick the store, not just the product. Run this test on your Shopify store before BFCM, then fix what the agent cannot confirm.

Cover: Why did the agent pick that store? A dark grey agent chat where the agent could not confirm freight cost or delivery date, with the Ambaum logo

AI shopping agents now pick the store, not just the product. When a buyer asks an assistant for brake pads or a patio table, the agent decides which store to send them to, and it often checks out right in the chat. We covered how that checkout works in Any AI Agent Can Now Buy From Your Store, and how Shopify Catalog decides what the agent reads in our Shopify Catalog Mapping tool post. This piece is the practical part: a test you can run on your own store in a few minutes.
‍

Video: Test Your Store With an AI Shopping Agent

With BFCM a few weeks out, this is the cheapest way we know to find out what an agent sees when it looks at your store.
‍

The test: shop like a stranger
‍

Open the AI assistant you use most. ChatGPT, Claude, Gemini, and Meta's Muse all work. Shop for your best seller without naming your store, the way a buyer would. For a parts store, that might be “front brake pads for a 2019 Ram 1500.”
‍

Four step AI shopping agent store test: shop without naming your store, ask the agent why it picked that store, give it your URL, then check the same query in Shopify under Sales channels > Agentic
The test in four steps.

Then ask the agent why it picked the store it picked. If it is not yours, give it your store's web address and ask whether it ran into any issues, like the shipping cost or the delivery date. Agents tend to name the exact thing they could not confirm, and that answer is your fix list.
‍

Check it inside Shopify too

You can run a similar check in your admin. Go to Sales channels > Agentic and enter a shopper style query in the search preview. It shows the raw Shopify Catalog results and which of your products could rank. Shopify's Help Center calls it a directional signal, since each AI channel re-ranks results with its own logic.
‍

Frame from the Ambaum video: the Shopify admin Agentic search preview for an example cedar furniture store, next to three follow up questions to ask the agent
From the video: the Agentic search preview in Shopify admin for an example store.

Fix one: let the agents in
‍

Start with anything that blocks agents before they reach your store. If you use a bot blocking or country blocking app, ask the vendor whether it blocks AI agents. Many of these tools were set up to stop scrapers, and some treat a shopping agent the same way.
‍

Next, open yourstore.com/robots.txt and look for a Disallow: /policies/ line. Shopify's current managed robots.txt no longer blocks policy pages. But if your theme has a robots.txt.liquid template that prints the older default rules, that line can still be there, per Shopify's robots.txt guide. Agents need to read your shipping and return policies, so remove the rule from the template, or delete the template if you do not need custom rules.
‍

Frame from the Ambaum video: an example Shopify store's robots.txt with agent instructions, policy pages crawlable and no Disallow: /policies/ line
From the video: an example robots.txt with no Disallow: /policies/ line.

Fix two: write your rules as numbers
‍

An agent cannot confirm “fast shipping” or “easy returns.” It can confirm numbers. Put these in your store policies in plain figures:
‍

  1. Freight charges: what oversized or heavy items cost to ship, and where.
  2. Delivery times: business days by region or method.
  3. Cancellation windows: how long a buyer has to cancel an order.
  4. Core return windows: for parts stores, how many days a buyer has to return a core and how the deposit is refunded.
    ‍
Frame from the Ambaum video: three cards for freight charges, delivery times, and return windows, with a reminder to set up pickup and ask the agent if it sees it
From the video: write your policy rules as numbers.

Fix three: keep stock and pickup current
‍

Agents favor stores that can show the item is in stock. Keep inventory counts accurate in Shopify, especially on your best sellers going into November. If you offer local pickup, set it up in Shopify, then run the test again and ask the agent whether it sees pickup as an option.
‍

Two things to avoid
‍

Do not replace your agents.md file unless you have a clear reason. Every Shopify store serves a managed file at /agents.md that tells AI agents how to find your products, policies, and checkout. Shopify keeps it current for you. Adding an agents.md.liquid template replaces that managed file, and from then on you maintain it yourself.
‍

Avoid vague promises. “Fast shipping” and “hassle free returns” are claims no agent can verify. If it cannot confirm a promise, it moves on to a store that states the number.
‍

Ambaum fix list for AI shopping agents: let agents in, write rules as numbers, keep stock current, and avoid replacing agents.md or vague promises like fast shipping
The fix list before BFCM.

What to do before BFCM
‍

  1. Run the test on your top three products with two different assistants.
  2. Ask each agent why it picked the store it picked, then give it your URL.
  3. Check your bot blocking app and your /robots.txt for Disallow: /policies/.
  4. Rewrite shipping, delivery, cancellation, and return rules as numbers.
  5. Confirm stock counts and pickup settings, then rerun the test.
  6. Leave Shopify's managed /agents.md in place unless you have a reason to change it.
    ‍

At Ambaum we run this test for Shopify merchants and tell you what the agents see. If you want that done before Black Friday, reach out.
‍

No pitch. No pressure. Just perspective.
‍

Video transcript
Read the full transcript

As a Shopify merchant, you should know that AI shopping agents send buyers to a store, not just to a product. So here is a test you can run on your own store today and what to fix if you fail it.

The test. In your AI assistant of choice, Claude, Gemini, Muse, or any other, shop for your best seller without naming your store. Something like front brake pads for a 2019 Dodge truck. Then ask the model why it picked that store. If it is not yours, give it your web address and ask if there were issues like the shipping cost or the delivery date. You can run a similar check inside Shopify under Sales channels, Agentic, Test a query.

Now the fixes. First, ask your bot blocking or country blocking app's vendor whether it blocks AI agents, and remove any Disallow: /policies/ line from your robots.txt.liquid file. Agents need to read your policies. Second, write your rules as numbers in your policies. Freight charges, delivery times, cancellation windows, core return windows. Third, keep stock current. And if you offer pickup, set it up and ask the agent if it sees it.

Two things to avoid. Do not replace agents.md, Shopify's file for AI agents, unless you know what you are doing. Shopify sets it up for your store by default. And avoid vague promises like fast shipping that no agent can confirm.

Ambaum can run this test on your store before Black Friday. Reach out and we will tell you what the agents see.

Frequently Asked Questions
How do I test whether AI shopping agents recommend my store?
Ask an assistant such as ChatGPT, Claude, Gemini, or Meta's Muse to find your best selling product without naming your store. Then ask why it chose the store it did. If it was not yours, give it your store URL and ask what it could not confirm, such as shipping cost or delivery date.
Can I preview AI agent search results inside Shopify?
Yes. In your Shopify admin, go to Sales channels > Agentic and use the search preview. It shows raw Shopify Catalog results for a query. Shopify notes that each AI channel re-ranks results, so treat it as a directional signal.
Does Shopify's robots.txt block AI agents from my policy pages?
Shopify's current managed robots.txt does not block /policies/. Stores with a robots.txt.liquid template that outputs the older default rules can still have Disallow: /policies/. Check yourstore.com/robots.txt and remove that rule if it is there.
What is agents.md on a Shopify store?
It is a file Shopify serves by default at /agents.md that tells AI agents how to discover your products, policies, and checkout endpoints. /llms.txt and /llms-full.txt mirror it. Adding an agents.md.liquid template replaces it, so most stores should keep the managed default.
Why should store policies use numbers instead of promises?
AI agents compare stores on facts they can confirm. A policy that states freight charges, delivery days, cancellation windows, and return windows gives the agent something to verify. Phrases like "fast shipping" do not.
No pitch. No pressure. Just perspective.

Get the strategic input you’ve been missing