llms.txt v2 Allows Path Scoped Files: What Should B2B Sites Do?

Should you update your llms.txt now that the proposal is on v2?

Only if you have a docs section, a help center, or a product area that deserves its own summary. The v2 proposal on llmstxt.org lets a file sit at any path and cover the pages under it. For a small marketing site, one root file is still enough, and Google says it is not needed for AI Overviews.

The llms.txt proposal has been around since Jeremy Howard published it in September 2024. The page at llmstxt.org now calls itself v2 and shows a modified date of August 10, 2026. In the author's words, it was "updated based on what I learned from two years of adoption." That is worth a fresh look, because the changes affect how B2B sites with several distinct sections might use it.

This piece covers what the current proposal says, what it means for a typical SaaS or services site, and how much effort I think it deserves.

What is llms.txt in plain terms?

llms.txt is a markdown file that gives AI tools a short, curated guide to a website. According to the proposal, it offers brief background information, guidance, and links to detailed markdown files. The only required part is an H1 with the site or project name. Everything else, like a summary and link lists, is optional.

The idea is simple. A website built for humans carries navigation, scripts, banners, and layout that an AI agent has to wade through. A clean markdown file lets the site owner say "here is what matters, in order." It is closer to a reading list than to a technical control.

That distinction is important. llms.txt does not block or allow crawlers. That is still the job of robots.txt. It also does not guarantee that any AI system reads the file at all. It is an offer of a cleaner map, and whether a given tool uses it is up to that tool. If you are new to the file, my earlier piece on what llms.txt is and whether you need it covers the basics.

What does the v2 proposal say about where the file can live?

The current proposal says the file can be placed at the site root, or at any path within it, covering the pages under that path. Where more than one file applies, it says agents should use the most specific one. That means a docs folder could carry its own llms.txt separate from the marketing site.

For most B2B companies, this maps neatly onto how their sites are actually built. The marketing pages, the blog, the documentation, and the help center often serve different readers and change on different schedules. One root file that tries to summarize all of them either gets too long or leaves things out.

With path scoped files, you could keep a short root file for the company and product overview, and a separate file under your docs path that lists the guides, API references, and setup pages a developer or agent would need. Each file stays focused, and each can be owned by the team that owns that section.

What else does the current proposal recommend?

Two things stand out. The proposal suggests providing a clean markdown version of important pages at the same URL as the original, either with .md appended or with the extension replaced. It also recommends standard link relations so clients can find the llms.txt files. Both aim to make discovery and reading easier.

The markdown page idea is the more demanding one. It asks you to serve a second, text only version of key pages. On a hosted platform like Webflow, serving a second format of each page depends on what the platform supports, so check its current docs before promising this to anyone.

The link relation recommendation is cheaper. Adding a link element in the page head that points to your llms.txt is a small change on most platforms, including Webflow through custom code. It costs little and helps any tool that looks for it. I would do that part whenever I touch the file.

Does Google need llms.txt to show your site in AI Overviews?

No. Google's documentation on AI features says you do not need to create new machine readable files, AI text files, or markup to appear in AI Overviews or AI Mode. It says a page must be indexed and eligible to show in Search with a snippet. There are no additional technical requirements.

That one sentence settles a lot of confusion. If your goal is Google's AI features, your effort belongs in the basics: pages that can be crawled and indexed, clear content, and snippet eligibility. Google's page also points to nosnippet, data-nosnippet, max-snippet, and noindex as the controls for limiting what shows, and to robots.txt for managing crawling.

So llms.txt is not a Google play. Its possible value is with other AI tools and agents, especially coding assistants and agents that read documentation, where a clean map of a docs site is genuinely useful. That is a narrower audience than "all of AI search," and it should shape how much time you spend on it.

Which B2B sites benefit most from path scoped llms.txt files?

Software companies with real documentation benefit most. If developers or AI coding agents need to understand your API, SDK, or integration steps, a scoped file under your docs path is a clean way to point them at the right pages. Services firms and small marketing sites gain far less.

A good test is to ask whether an agent would ever need to work through your site to complete a task. A developer asking an AI assistant to build an integration with your product would benefit from a precise list of the relevant docs pages. A buyer asking which agency to hire would not, because that answer comes from reputation, reviews, and your main pages, not from a link list.

For my own site, the stance is simple. My robots.txt explicitly allows crawlers such as GPTBot, ClaudeBot, PerplexityBot, and Google-Extended, because I want AI tools to read the content. A single root llms.txt would be plenty for a site like mine. Path scoping makes sense when a site has sections that serve genuinely different jobs.

What mistakes should you avoid with llms.txt?

The common mistakes are treating it as a ranking tool, letting it go stale, listing every page instead of curating, and contradicting what your main pages say. A stale or bloated llms.txt can point tools at outdated material, which is worse than having no file at all.

Staleness is the big one. If your pricing, plans, or product names change and your llms.txt still links to old pages, any tool that trusts the file gets the old story. I would tie llms.txt updates to the same process that updates your sitemap or navigation. When a page is retired, it leaves the file too.

Curation matters as much. The point of the file is to say what matters most. Dumping every URL into it defeats the purpose and makes the "Optional" section, which the proposal describes as secondary links an agent can skip, meaningless. For a checklist of what to review, see my guide on what to check in an llms.txt audit.

How does llms.txt fit with robots.txt and your other crawler controls?

They do different jobs. robots.txt tells crawlers what they may access. Meta robots tags and snippet controls tell search engines what they may show. llms.txt suggests what an AI tool should read first. You need the first two to be correct before the third adds anything.

I always audit in that order. First, confirm robots.txt is not accidentally blocking the crawlers you want, or allowing ones you do not. Second, check that important pages are indexable and not hidden behind scripts. Only then is it worth curating an llms.txt, because a beautiful reading list is pointless if the pages it links to cannot be fetched.

If you have not reviewed your crawler rules recently, start there. My piece on robots.txt and AI crawlers walks through which user agents to think about and how to decide what to allow. Getting that right does more for AI visibility than any llms.txt file.

What should you do next?

Check whether your site has distinct sections, like docs or a help center, that serve different readers. If it does, consider a short root llms.txt plus a scoped file for that section, and add a link relation pointing to each. If it does not, keep one small root file current and move on.

My honest view is that llms.txt is a low cost, low certainty bet. The v2 proposal makes it more practical for sites with several sections, but Google is clear it is not needed for AI Overviews. Spend an hour on it, not a week, and spend the week on content and crawlability.

If you want help auditing your crawler controls, deciding whether a scoped llms.txt is worth it, or setting it up on Webflow, reach out. Let's chat about what AI tools should find when they read your site.