Does anything actually read llms.txt?

Some tools do and the major search crawlers do not. Google has said plainly that it does not use llms.txt, and our own August 2026 crawl of live sites found roughly 11.5% serving a real file once you discount the servers returning a soft 200 for a page that does not exist. Treat it as cheap and speculative: it costs an hour, it breaks nothing, and it is not a substitute for the page being readable in the first place.

What is the correct structure?

An H1 with the site name, an optional blockquote summary, optional prose, then H2 sections containing Markdown link lists where each link may carry a short description after a colon. A section titled Optional has a defined meaning: it marks links a model may skip when it is working within a tight context budget.

Where does the file go?

At the root of the domain, served as text or Markdown at /llms.txt, the same placement convention robots.txt uses. It is a curated index rather than a crawl instruction, so it does not replace robots.txt and it does not control access to anything.

How many links should I include?

Enough to cover what you would want quoted, which for most sites is between ten and forty. The file competes for a model's context window, so a list of every URL on the site is self-defeating — it is the curation that carries the value, and a complete list is the absence of curation. If you want everything listed, you already have a sitemap.

llms.txt Generator — build a valid llms.txt file for your site

What the file is for

llms.txt is a proposal from 2024 for a Markdown file at the root of a domain that points language models at the pages worth reading, in a form that is cheap to parse and cheap to hold in a context window. It is a curated index. It is not an access control, it does not replace robots.txt, and nothing about publishing one changes what a crawler is permitted to fetch.

Be clear-eyed about adoption

Google has said directly that it does not use llms.txt. When we crawled a large sample of live sites in August 2026, about 27% returned a 200 status for the path and only around 11.5% were serving a real file — the gap is servers answering every unknown path with a soft 200 and a copy of the homepage, which is worth checking on your own domain before you assume yours is working.

So the honest position is that this is cheap and speculative. It takes an hour, it breaks nothing, some tools do read it, and it may matter more later than it does now. What it cannot do is compensate for pages that a model cannot read once it follows the link. If the linked page renders its content with client-side JavaScript and returns an empty shell to a fetch, a beautiful llms.txt just routes a model efficiently to nothing.

The structure

The specification is short and this generator follows it exactly. An H1 with the site name, which is the only genuinely required element. An optional blockquote holding a one-sentence summary. Optional prose after that. Then H2 sections, each containing a Markdown list of links, where each item may carry a short description after a colon.

One section name is reserved: a section titled Optional marks links that a model may skip when it is working with limited context. Everything else is yours to name, and the names are doing real work — Docs, Pricing and Policiestell a model what kind of question each group answers.

What to link, and how much

Between ten and forty links suits most sites. The file competes for the same context window as the conversation a model is having, so listing every URL you have is self-defeating: it is the curation that carries the value, and a complete list is the absence of curation. If you want everything enumerated, that is what your sitemap is for, and the two files should not be the same.

Write the descriptions for a reader who cannot see the page. "Pricing" tells a model nothing it could not guess from the URL. "Pricing: three plans, per-seat, with the self-hosted tier free under five users" is a sentence a model can answer a question with, and answering questions is the entire reason the file exists.

Checking it once it is live

Publish the file at /llms.txt at the root of the domain and confirm that the path returns the file itself with a text content type — not a 200 carrying your homepage, which is the failure mode behind most of that adoption gap. Thellms.txt checker on visibility100x.comfetches the live path and reports what is actually served, which is the one step this generator cannot do from inside a browser tab.