Somewhere in the last few months, Google quietly added a new category to Lighthouse and PageSpeed Insights. It sits below the familiar Performance, Accessibility, Best Practices, and SEO scores, and it is called Agentic Browsing. Most site owners have never seen it. Most SEO agencies have never mentioned it. And yet it is already live, already scoring real websites, and already exposing a gap that almost every major e-commerce brand shares.

This post walks through an investigation we ran across four Shopify storefronts — three of the biggest names in direct-to-consumer retail, and one small technical SEO agency site, alneeko.com. What we found was consistent, repeatable, and, frankly, a little surprising: every major brand we tested failed the exact same check, for the exact same underlying reason, while a much smaller site passed with a perfect score.

Along the way, we also revisited two foundational technical SEO files — robots.txt and sitemap.xml — clarified a common misreading of how they work on Shopify, and used real Core Web Vitals field data to show why a scary-looking lab performance score can be almost meaningless in isolation. If you run or advise an e-commerce store, this is the kind of audit that is quickly becoming table stakes as AI agents start to browse and shop on behalf of real customers.

Part 1: Getting the Fundamentals Right — Sitemap vs Robots.txt

Before getting to Agentic Browsing, it is worth being precise about two files that are often confused with each other: robots.txt and sitemap.xml. They look similar — both are small, plain-text-adjacent files sitting at the root of a domain — but they do fundamentally different jobs.

Robots.txt Is a Permission File

robots.txt lives at yourdomain.com/robots.txt and tells crawlers which parts of a site they are allowed to request in the first place. It is a gate, not a map. It uses simple directives — User-agent to specify which bot a rule applies to, Disallow to block a path, and Allow to carve out exceptions inside a blocked folder.

VERIFIED: Google’s own documentation confirms that a Disallow rule blocks fetching of a page’s content, but does not guarantee the URL stays out of the index if it is linked from elsewhere — Google can still index the bare URL with no description.

This distinction matters because it is the source of one of the most common technical SEO myths: that blocking a page in robots.txt is the same as de-indexing it. It is not. If you genuinely want a page out of search results, robots.txt is the wrong tool; a noindex directive or authentication is the right one.

Sitemap.xml Is a Discovery File

A sitemap does not ask permission. It hands crawlers a list of URLs the site considers important, along with metadata like lastmod timestamps. On a real Shopify store, this is rarely a single flat file — it is a sitemap index that fans out into segmented sub-sitemaps, typically split by products, collections, and pages, with large catalogs further split by ID ranges.

VENDOR CLAIM: Some SEO plugins imply that the priority and changefreq fields inside a sitemap directly influence ranking or crawl frequency.

DISPUTED: Google has stated for years that it largely ignores priority and changefreq values in practice, even though the fields remain part of the sitemap protocol and other crawlers may weigh them differently.

The practical takeaway is simple: stop tuning priority values, and instead make sure the URLs listed in your sitemap are URLs you genuinely want indexed. That is the real lever.

Where the Two Files Actually Conflict

The mistake we see constantly on Shopify and WordPress stores alike is a sitemap listing product or collection URLs that robots.txt simultaneously blocks — often because a theme, app, or filtered navigation path was disallowed after the sitemap was already generated. The result is Google Search Console’s familiar warning, Submitted URL blocked by robots.txt, which represents wasted crawl budget and a conflicting signal sent to every crawler that reads both files.

VERIFIED: Google Search Central explicitly recommends not including URLs in a sitemap that are blocked by robots.txt, since it creates conflicting signals for crawlers.

A five-minute audit — pulling the live sitemap.xml and the live robots.txt side by side and cross-checking every Disallow rule against every sitemap URL — catches this reliably, and it is one of the highest-leverage, lowest-effort checks in a technical SEO process.

Part 2: What Shopify Actually Automates — and What It Does Not

A fair question, and one worth answering honestly: on Shopify specifically, how much of this do store owners actually need to worry about?

Shopify automates the sitemap almost entirely. It is generated by the platform, updates itself as products and collections change, and cannot even be edited directly — there is no sitemap.xml.liquid template to override. For the overwhelming majority of stores, the sitemap is genuinely set-and-forget.

Robots.txt is a different story. Shopify ships a sensible default that blocks cart, checkout, account, and admin paths, and allows everything else. But since 2021, merchants have been able to override this default with a custom robots.txt.liquid template. That flexibility is exactly what creates risk: theme developers, apps, and well-meaning store owners can accidentally block collections, blogs, or entire site sections. Multiple apps touching the same file over time also tends to produce duplicate, redundant Disallow lines — harmless to crawlers, but a clear sign that a file has been patched rather than maintained.

Faceted navigation is the other recurring Shopify-specific issue: sort and filter parameters on collection pages can generate a near-infinite number of crawlable URL combinations, which is a genuine crawl-budget problem regardless of how clean the base robots.txt file is.

So the honest framing is this: Shopify removes roughly eighty percent of the manual sitemap and robots.txt work compared to a hand-rolled WordPress setup. The remaining twenty percent — custom robots.txt edits, app conflicts, and faceted navigation bloat — is exactly where technical audits still catch real, revenue-relevant problems.

Part 3: The Lighthouse Score That Lies (and the Field Data That Tells the Truth)

Running a live Lighthouse audit against Allbirds’ homepage produced a jarring result: a Performance score in the twenties to thirties, alongside Accessibility, Best Practices, and SEO scores all in the nineties or better. On the surface, that looks like a badly broken site.

It is not. Pulling the same URL through PageSpeed Insights and looking at the Chrome User Experience Report field data — the actual, anonymized experience of real visitors over the last 28 days — tells a very different story: Largest Contentful Paint at 1.9 seconds, Interaction to Next Paint at 73 milliseconds, Cumulative Layout Shift at 0.01, First Contentful Paint at 1 second, and Time to First Byte at 0.4 seconds. Every single metric green. Core Web Vitals: Passed.

VERIFIED: Google’s own guidance distinguishes lab data, which is based on a simulated load under fixed throttled network and device conditions, from field data, which reflects real-world anonymized performance data from actual users — and explicitly notes the two can differ.

The lab score is a synthetic worst-case stress test; the field data is what real customers on real devices actually experience. Both have a place in an audit, but reporting a scary lab number to a client without also checking field data is, at best, incomplete, and at worst, actively misleading.

So What Is Actually Slow?

Digging into the mobile diagnostics explains the gap. The flagged issues were not server-side: Time to First Byte was green throughout. Instead, the culprits were almost entirely client-side JavaScript — Reduce JavaScript execution time, flagged at 3.1 seconds; Minimize main-thread work, flagged at 5.4 seconds; Reduce unused JavaScript, with an estimated 637 KiB of savings available; Avoid enormous network payloads, with a total page weight of roughly 5.5 megabytes; and fifteen separate long main-thread tasks.

This is the pattern behind almost every heavily-throttled Lighthouse score on a mature e-commerce homepage: analytics tags, personalization engines, review widgets, chat tools, A/B testing scripts, and upsell apps all competing for the same main thread. None of it is a hosting problem — Shopify’s infrastructure is the same CDN-backed platform regardless of plan tier, which is exactly why server response time was never the issue. It is a front-end and third-party-script bloat problem, and it is a decision made by marketing and app-stacking choices, not by an infrastructure budget.

Part 4: Agentic Browsing — The New Category Almost No One Is Auditing

This is where the investigation turned up something genuinely new. Buried beneath the familiar four Lighthouse categories, PageSpeed Insights now includes a section labeled Agentic Browsing.

VERIFIED: PageSpeed Insights describes this category as checks that ensure high-quality, browsable websites for AI agents, and validate the correctness of WebMCP integrations, while explicitly noting that the category is still under development and subject to change.

In plain terms, it is asking a very different question than the other four categories. Performance asks how fast a page loads. Accessibility asks whether a human using assistive technology can use the page. SEO asks whether a search crawler can index it. Agentic Browsing asks whether an autonomous AI agent — the kind increasingly built into products like ChatGPT, Gemini, and various shopping assistants — can actually parse the page’s structure, find its interactive elements, understand what they do, and act on them.

Concretely, it checks things like whether the page’s accessibility tree is well-formed, whether buttons and links carry discernible, readable text or labels, and whether any WebMCP integration on the page is implemented correctly. It is a fairly small checklist today, and Google is explicit that it will change. But it is live, it is scoring real production sites right now, and it exposed something worth paying attention to.

Part 5: The Test — Four Sites, One Repeating Failure

We ran the Agentic Browsing check across four live Shopify storefronts: alneeko.com, and three major, high-traffic direct-to-consumer brands — Allbirds, Gymshark, and Skims.

Site	Agentic Browsing Score	Root Cause
alneeko.com	3 / 3 — Pass	None — accessibility tree well-formed
Allbirds	2 / 3	Unlabeled carousel nav button (icon-only, no aria-label)
Gymshark	1 / 2	Accessibility tree not well-formed
Skims	1 / 2	Carousel slide link with no discernible text (div/role=”link” instead of real <a>)

alneeko.com passed with a clean 3 out of 3 — accessibility tree well-formed, zero cumulative layout shift, and llms.txt following Google’s stated recommendations. Every other site tested, including three brands with substantial marketing budgets, dedicated development resources, and Shopify Plus infrastructure, failed to reach a perfect score. And crucially, they did not fail for three unrelated reasons. They failed for the same reason, wearing three slightly different outfits.

Allbirds: The Unlabeled Arrow

Allbirds scored 2 out of 3. The single failed check, Buttons must have discernible text, traced to a specific element on the homepage promotional banner: a swiper navigation arrow rendered as an icon-only button, with no aria-label, no visible text, and nothing else in the markup that would let an accessibility tree — or an AI agent reading that tree — understand what the button does. A sighted human recognizes the left-pointing chevron intuitively. A screen reader, or an autonomous agent, sees an empty, unnamed control.

Gymshark: A Structurally Malformed Tree

Gymshark scored 1 out of 2 — note the denominator itself changes between sites, most likely because Gymshark’s site does not implement an llms.txt file, so that particular check simply did not apply. The failure here was more fundamental than a single missing label: the accessibility tree itself was flagged as not well-formed, meaning the underlying structural relationships an agent relies on to understand the page hierarchy were broken, not just a single element within it.

Skims: The Fake Link

Skims also scored 1 out of 2, and again the accessibility tree was flagged as not well-formed. The specific failing element was a carousel slide built as a div carrying role equals link and tabindex equals zero, rather than a genuine anchor tag with a real href. Visually and functionally, it behaves like a clickable link to a collection page. Structurally, it is a simulation of a link — the kind of JavaScript-driven interactivity that looks identical to a human but is invisible, or at best ambiguous, to anything parsing the DOM’s actual semantics rather than its rendered appearance.

The Pattern

Three different brands. Three different Shopify themes. Three different app stacks, development teams, and marketing budgets. And in every single case, the failure traced back to the same category of UI component: an image carousel or slider, almost certainly built on a library like Swiper.js, implemented without the accessibility attributes that would make its controls legible to anything other than a mouse and a pair of human eyes.

This is not a coincidence, and it is not a coding-skill problem specific to any one team. Carousel and slider components are near-universal in e-commerce theme design, and the JavaScript libraries that power them typically ship without accessible defaults — developers have to explicitly add them, and in practice, most do not.

Part 6: Why This Matters — Separating the Real Stakes from the Hype

It would be easy to overclaim here, and it is worth resisting that temptation directly.

DISPUTED: Whether Agentic Browsing affects search rankings today is not confirmed by Google. It is presented as a diagnostic signal for AI-agent compatibility, not a stated ranking factor, and the category itself is labeled as still under development.

VENDOR CLAIM: Some agencies are already marketing Agentic Browsing fixes as a guaranteed traffic or ranking boost.

Treat that kind of promise with real skepticism. The honest case for fixing these issues rests on two separate, more defensible arguments, not on an unproven ranking claim.

The Emerging-Channel Argument

AI shopping agents are moving from novelty to genuine purchase channel. Products like conversational shopping assistants increasingly browse, compare, and in some cases complete purchases on a user’s behalf. If an agent cannot parse a site’s carousel, navigation, or product links, the practical consequence is not a lower search ranking — it is that the agent skips the site entirely and completes the transaction somewhere else. That is a direct loss of a transaction the brand never had a chance to compete for, independent of anything Google’s algorithm does.

The Accessibility and Legal-Risk Argument

The exact same broken button that confuses an AI agent also fails a human using a screen reader or keyboard-only navigation. Large e-commerce brands are frequent, recurring targets for web accessibility litigation, and an unlabeled button or a fake link built from a div are precisely the kind of concrete, easily-documented failures that show up in that kind of claim. Fixing these issues is not speculative future-proofing — it closes a real, present-day legal and usability exposure, with the AI-agent benefit arriving as a bonus rather than the primary justification.

Part 7: The Fix — and Why It Is Genuinely Simple

The reassuring part of this entire investigation is that the fix, in every case we examined, is small, cheap, and fast to implement.

Step 1: Audit and Document

Run the target site through pagespeed.web.dev on both mobile and desktop, and record every failed Agentic Browsing and accessibility audit along with its exact CSS selector and outer HTML. This becomes both the before-state proof and the fix checklist. Cross-checking with an actual screen reader, such as VoiceOver or NVDA, will typically surface the same gaps an AI agent hits.

Step 2: Locate the Component in the Theme

In Shopify admin, under Online Store, Edit Code, search the theme’s section and snippet files for the slider or carousel component — most Shopify themes rely on Swiper.js, Flickity, or a custom-built equivalent, and the broken markup lives inside that one shared file.

Step 3: Label Icon-Only Controls

For navigation arrows like a swiper previous or next button, add a descriptive aria-label — Previous slide, Next slide — directly on the button element. This single-line change resolves the exact issue found on Allbirds.

Step 4: Replace Fake Links With Real Anchor Tags

Where a slide or card is built as a div using role equals link and tabindex equals zero rather than a genuine anchor element, replace it with a real anchor tag carrying an actual href and either visible text or a descriptive aria-label. Semantic HTML is inherently more robust for both AI agents and assistive technology than JavaScript-simulated interactivity, and it typically requires no additional scripting to remain fully functional.

Step 5: Re-Test and Extend Sitewide

After deploying the change — ideally tested on a duplicate, unpublished theme before going live — re-run PageSpeed Insights on both mobile and desktop to confirm the Agentic Browsing score reaches a full pass and that the accessibility tree now reads as well-formed. Because most themes reuse the same slider snippet across the homepage, product galleries, and upsell carousels, a single fix to the shared component typically resolves every instance sitewide at once.

Step 6: Package the Before-and-After as Proof

The most durable output of this process is not the fix itself but the documentation of it: a failing score and its exact HTML alongside the passing score after the change, tied to a small, specific code diff. That artifact is a far stronger piece of audit evidence than a generic claim of technical expertise, because it is concrete, verifiable, and repeatable against any other prospective client’s storefront.

A site can pass one layer and fail the other. A perfectly labeled, fully accessible carousel pointing to a catalog full of inconsistent product titles will still produce an agent that gives confusing answers. Conversely, a catalog with pristine, consistent data sitting behind an unlabeled div-based carousel may never get parsed by the agent at all. A genuinely complete AI-readiness audit has to check both layers, not just the one that happens to have a convenient Lighthouse score attached to it today.

In practice, this means a proper pre-build audit for any Shopify AI integration — whether that is a customer-facing shopping assistant, an internal support agent, or a third-party AI shopping tool crawling the catalog — should include a pass over product title and variant consistency, a check of policy documents for contradictory or outdated language, and a review of customer history fields for the same kind of formatting drift that shows up in product data. None of this requires new tooling; it is closer to a data-quality audit than a technical SEO one, but it belongs in the same conversation, because the business impact — an agent giving a customer a wrong or confusing answer — looks identical regardless of which layer actually caused it.

Part 9: A Working Checklist

Pulling the investigation together into something usable, here is a practical, in-order checklist for auditing any Shopify storefront’s readiness for both search crawlers and AI agents.

• Confirm robots.txt contains a Sitemap: directive with the full, absolute URL.
• Cross-check every Disallow rule in robots.txt against the live sitemap.xml to catch conflicting signals.
• Check for duplicate or redundant Disallow lines as a sign of unmaintained, app-patched robots.txt files.
• Verify sort, filter, and other query-parameter URLs are blocked to protect crawl budget on collection pages.
• Run PageSpeed Insights on both mobile and desktop, and read the field data (CrUX) section, not just the lab score.
• If the lab Performance score is poor but field data passes, look at JavaScript execution time, unused JavaScript, and total page weight before blaming hosting.
• Check the Agentic Browsing score specifically, and open every failed audit to see the exact failing HTML element.
• Search the theme’s carousel or slider snippet first — in every case examined here, this is where the failure lived.
• Add aria-label attributes to icon-only navigation controls, and replace div-based fake links with real anchor tags.
• Separately audit product title and variant-name consistency across the catalog, since this is a data-layer issue Agentic Browsing does not measure.
• Re-test after each fix and document the before-and-after scores as concrete proof of the work performed.

Part 10: Frequently Asked Questions

Is Agentic Browsing a confirmed Google ranking factor?

No. As of this writing, Google describes it as a diagnostic category still under development, aimed at assessing whether a page is well-suited to AI agents. It is not listed alongside Core Web Vitals as a stated ranking signal, and it may change significantly before it does — if it ever does.

If it is not a ranking factor, why fix it now?

Two reasons hold up independent of any future ranking impact. First, AI shopping agents are an emerging purchase channel in their own right — a site an agent cannot navigate simply gets skipped, which is a direct commercial loss regardless of search rankings. Second, the exact same failures this category surfaces are also accessibility failures for human users of assistive technology, which carries its own usability and legal relevance today.

Does a low Agentic Browsing score mean the site is broken?

Not necessarily, and this is worth stating plainly given how alarming a red score can look. In every case examined here, the rest of the site functioned normally for ordinary visitors. The score reflects a specific, narrow gap in how certain interactive elements are marked up — not a broader outage or defect.

Is this specific to Shopify?

The underlying issue — carousels and sliders built without accessible markup — is a near-universal pattern across e-commerce platforms, not a Shopify-specific defect. Shopify’s theme ecosystem simply makes it easy to observe at scale, since so many stores rely on the same handful of slider libraries.

Conclusion: A Small Test Worth Running Today

Agentic Browsing is not yet a confirmed ranking factor, and Google says as much directly. But it is already live, already grading production e-commerce sites, and it already exposed a consistent, structural gap across three of the largest direct-to-consumer brands tested here — a gap that traces back, in every case, to the same underused component: the image carousel.

The broader lesson sits alongside a related idea worth keeping in view: structural AI-readiness, the kind Agentic Browsing measures, is only half of the picture. The other half is data-layer AI-readiness — whether product titles, variant names, return policies, and customer records are consistent enough for an agent to reason over correctly once it has successfully navigated to them. A site can pass one test and fail the other. A genuinely complete AI-readiness audit checks both: can an agent find and operate the page, and can it trust what the page actually says.

For now, the practical takeaway is narrow and actionable. Run any storefront through pagespeed.web.dev, check the Agentic Browsing score, and if it is not a perfect pass, look first at the carousel. In every case examined here, that is exactly where the problem — and the five-minute fix — was waiting.