.png)
Let’s talk
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
llms.txt is a plain Markdown file that points AI systems at your best content. It is not an access-control file: it cannot restrict any crawler, block any AI system, or stop your content being used. It also has no effect on Google rankings. It is a helpful signpost, worth ten minutes, and nothing more than that.
llms.txt has been sold to a lot of marketers this year as the AI equivalent of robots.txt. That comparison is wrong in the one way that matters, and brands are making decisions based on it.
llms.txt is a Markdown file at the root of your domain listing your most important pages with short descriptions. It exists so that a system with a limited context window can find the parts of your site worth reading, instead of crawling everything and guessing.
That is genuinely useful. A large site with deep navigation is hard for a retrieval system to summarise fairly, and a curated index helps.
It is not access control. This is the misconception worth correcting plainly:
llms.txt is a table of contents, not a lock. If you want to control access, that is robots.txt and your server – and neither of those is what this file does.
– The Antimony position
The filename invites the comparison, and the comparison leads somewhere bad. We have seen brands treat publishing an llms.txt as the moment they addressed AI visibility, and stop there – the box ticked, the actual work untouched.
Worse is the inverse: a brand believing that omitting pages from llms.txt protects them. It does not. Anything publicly reachable is publicly reachable. If content genuinely must not be ingested, that is a robots.txt, authentication and terms-of-service question, and it belongs nowhere near this file.
Treat it as an editor's brief to a reader with limited time. A short statement of what the organisation actually is – plainly, without positioning language, because a model is trying to classify you. Then your genuinely important pages, each with one line saying what it contains and who it is for. Keep it current: a file listing pages that no longer exist is worse than no file, because it advertises dead URLs.
Our own first llms.txt listed journal articles that were scheduled but not yet published. Every one of those URLs returned a 404. We had built a curated index of pages that did not exist, and only caught it by checking every link in the file by hand.
The general rule: any file whose whole purpose is to point at your best content becomes actively harmful the moment it points at nothing. That applies to llms.txt, to your sitemap, and to the internal links inside your articles. Verify on a schedule, not once at launch – and especially after any change to a publishing calendar.
The version we run now lists only pages that resolve, and points at the material worth reading first: the case study library, the journal, and the results. If a model only reads three things about your business, those should be the three.
Definitions. robots.txt: a crawl-directive file that compliant crawlers honour. llms.txt: a proposed Markdown convention for pointing AI systems at key content. Neither restricts what a non-compliant system does.
