What is llms.txt and what does it mean for AI visibility?

We believe it is essential to add an llms.txt file to your website. Google’s own Lighthouse tool now checks for it, and after we added one to quadweb.uk our AI citations increased by over 75% in one month.

The file still has critics. Google has compared it to the keywords meta tag, but we think the evidence has moved on since then, and our own numbers back that up.

What is llms.txt?

llms.txt is a Markdown file that sits at the root of your site, at yoursite.co.uk/llms.txt. It gives AI tools a short summary of who you are and links to the pages that matter most, so they can skip the menus, cookie banners and scripts on a normal web page.

It sits alongside robots.txt. robots.txt decides which crawlers are allowed in. llms.txt tells the ones that come in what to read first and helps to guide AI engines around your website.

Google’s own Lighthouse now checks for llms.txt

Lighthouse 13.3, released in May 2026, added an Agentic Browsing category to Chrome’s site audit. It tests whether AI agents can read and use your site, and llms.txt is one of its checks. Google’s documentation calls the file an emerging convention for giving LLMs and AI agents a machine-readable summary of a site. Without one, it says, agents may spend more time crawling to work out your site’s structure and main content.

Our site passes all three checks. The audit expects a Markdown file with at least one H1 heading, and that’s the minimum we build every client file to.

Doesn’t Google say it ignores llms.txt?

For Google Search, yes. Google’s guide to optimising for its AI features says you don’t need llms.txt to appear in AI Overviews or AI Mode, because Google Search doesn’t use it. In June 2026 Google added a line saying it’s fine to keep the file for other services that use it, and that it won’t help or harm your Google rankings.

So one Google product ignores the file and another audits it. Search is deciding which pages to rank. Lighthouse is checking whether an AI agent filling in a form or comparing quotes for a customer can find its way around your site. Both matter to a business that wants enquiries from AI.

Google isn’t the only AI search engine

ChatGPT, Claude and Perplexity run their own crawlers and their own indexes, so Google’s view of its own search engine settles nothing for them. OpenAI says sites that block OAI-SearchBot won’t be shown in ChatGPT search answers. Anthropic says blocking Claude-SearchBot can reduce a site’s visibility in Claude’s search results.

Neither company calls llms.txt a ranking signal. OpenAI does publish an llms.txt index for its own developer documentation, though. When the people building these models write the file for their own sites, we’d rather copy what they do than what commentators guess.

llms.txt works best alongside the fundamentals

The file gets an AI tool to your best pages. Those pages still have to earn the citation, which means getting the rest right too:

• Crawler access: check robots.txt and your firewall allow OAI-SearchBot, Claude-SearchBot and Claude-User.
• Clean structure: proper headings and semantic HTML. Lighthouse’s accessibility tree check, in the same Agentic Browsing category, tests exactly this.
• Consistent business details: the same name, services and location on your site, Google Business Profile and directories.
• Useful content: pages that answer the questions your customers ask, with real prices, examples and local detail.
• Genuine mentions: reviews, press and directory listings that confirm who you are.
• Tracking: run the questions your customers ask through ChatGPT, Perplexity and Google AI Mode each month, and check that you’re named and described correctly.

Should you add llms.txt to your website?

Yes. Google confirms it can’t hurt your rankings, Chrome’s own audit looks for it, the AI companies use it on their own sites, and our citations went up after we added ours. We now build one into every site we launch.

The bigger lesson

The llms.txt debate is a useful example of where AI search optimisation currently stands. The technology is changing faster than the evidence, which leads to new terminology appearing, and businesses are understandably looking for an advantage. But the safest strategy isn’t to chase every new “AI search hack”. It is to understand how the underlying systems actually discover and retrieve information, follow the evidence where it exists, clearly label uncertainty where it doesn’t, and measure what happens on real websites.

In other words: don’t optimise for what someone says an AI wants. Optimise for being genuinely useful, accessible, understandable and authoritative, then measure whether the AI systems agree.

FAQs

Does llms.txt help SEO?

Not for Google Search rankings. Google says the file won’t help or harm your rankings because Google Search ignores it. Its value is with AI agents and with AI search tools outside Google, which is where we saw our citations rise.

Is llms.txt the same as robots.txt?

No. robots.txt controls which crawlers can access your site, and llms.txt has no say over access at all. You need both.

Should you add llms.txt to your website?

Yes. Google confirms it can’t hurt your rankings, Chrome’s own audit looks for it, the AI companies use it on their own sites, and our citations went up after we added ours. We now build one into every site we launch.

If you’d like to know how AI assistants currently describe your business, book a Referral Accelerator audit. We’ll check your llms.txt, your crawler access and how you show up in ChatGPT, Perplexity and Google AI Mode.