llms.txt

What an llms.txt file is

An llms.txt file is a plain-text file at your site’s root that hands large language models a clean, markdown map of your most important pages and facts. Jeremy Howard proposed it in September 2024. The idea mirrors robots.txt, which guides search crawlers, except llms.txt is written for the AI systems that read and summarize your content.

What goes in it

The file is a short, structured outline, not a data dump. A typical llms.txt names the site, gives a one-line description, and links the pages that matter most, each with a sentence of context. The principle is signal over volume: point a model at the few sources you want it to rely on, in the order you would rank them yourself.

Does it actually work yet

Honestly, the evidence is thin. As of late 2025 the major AI crawlers (GPTBot, ClaudeBot, PerplexityBot) do not request llms.txt while answering a question, and one audit found 8 of 9 sites saw no measurable traffic change after adding one. No AI company has confirmed it uses the file during inference. Treating llms.txt as a guaranteed ranking lever would be dishonest, and we will not pretend otherwise.

Why we still ship one

We add llms.txt to every site because the cost is near zero and the downside is none. It takes minutes, it cannot hurt your search or AI visibility, and if the crawlers do adopt it, which is the direction the ecosystem keeps moving, your site is already prepared. We treat it as cheap insurance against a likely future, not a magic switch, and we tell clients exactly that.