llms.txt: What It Is and How to Create One

llms.txt is a proposed convention for a plain Markdown file, usually served at /llms.txt, that gives AI agents a short, structured guide to your website: what the site is, and which pages matter most. It was proposed by Jeremy Howard of Answer.AI in September 2024. It is not an official standard, and no AI company has to read it. Writing one is quick and free, so you can publish one as a low-risk bet.

What is llms.txt?

llms.txt is a file that follows the format described at llmstxt.org. Its stated purpose is to “provide information to help agents use a website.” The reasoning: a language model can only read so much at once, and a typical page is padded with menus, scripts and banners. A short Markdown file of clean links gives an agent a quick way in. The proposal says the file is meant to be read on demand, and its authors expect it to be useful mainly when an assistant answers a question (inference) rather than for training, though they do not rule out training use.

Who proposed llms.txt, and is it a standard?

Jeremy Howard of Answer.AI published the original proposal on 3 September 2024, and the specification lives at llmstxt.org. It is a community proposal. It has not been adopted as a formal internet standard. Publishing the file does not oblige anyone to use it.

What goes in an llms.txt file?

The format is Markdown, with sections in this order:

• An H1 with the name of the site or project. The proposal calls this the only required section.
• A blockquote with a short summary.
• Optional paragraphs or lists with more detail, without headings.
• H2 sections with link lists, each item written as [name](url), optionally followed by a colon and a note.
• An “Optional” H2 section for secondary links an agent can skip when it needs less context.

Here is a minimal example for a fictional bakery site:

# Rosa's Bakery

> Family bakery in Leeds selling sourdough, pastries and custom cakes. Orders by pickup only.

## Key pages

- [Menu](https://example.com/menu): current breads and pastries with prices
- [Custom cakes](https://example.com/cakes): how to order, lead times, allergen policy
- [Opening hours](https://example.com/visit): address and hours

## Optional

- [Our story](https://example.com/about)

How do you create an llms.txt file?

1. List your key pages. Pick a short selection of the pages a newcomer would need, not the whole site: products, pricing, key guides, policies, contact. The proposal sets no number, so keeping it short is our own suggestion.
2. Write the summary. One or two plain sentences saying who you are and what you offer. This is the blockquote.
3. Group the links under H2 headings such as “Products,” “Guides” or “Policies,” and add a short note after each link saying what the page contains.
4. Move secondary pages under “Optional.”
5. Save the file as llms.txt and upload it to the root of your site, so it loads at https://yourdomain.com/llms.txt.
6. Check that it loads and that every link returns a normal page. Our free llms.txt checker tests the file’s presence and structure.
7. Keep it current. When a page moves or disappears, fix the file. A list of broken links does more harm than no file.

The proposal also recommends serving clean Markdown versions of important pages at the same URL with .md added, for example page.html.md. That is optional and takes more work.

How is llms.txt different from robots.txt and sitemap.xml?

Each file has a different job, and none replaces another.

File	Format	What it does
robots.txt	Plain text rules	Tells crawlers which URLs they may access. Google describes it as mainly a way to avoid overloading your site, not a way to keep pages out of search.
sitemap.xml	XML	Lists the URLs of your site for search engines, each with a required loc tag.
llms.txt	Markdown	Gives AI agents a short, readable guide to your site and its best pages. It neither allows nor blocks anything.

If you want to control which AI crawlers can visit, that is a robots.txt matter. OpenAI, for instance, documents separate crawlers for ChatGPT search and for training, each controlled independently in robots.txt.

Does AI actually use llms.txt?

We cannot tell you that, and neither can anyone outside the AI companies. We found no primary source from an AI provider confirming that its assistant reads the file. Google’s documentation on its AI features states that you do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode.

What is verifiable is adoption by publishers. Anthropic publishes an llms.txt for its developer documentation, which lists the docs as Markdown links and points to a fuller file. A site publishing one is not evidence that an assistant uses it, only that the convention is in use.

So treat llms.txt as a cheap extra on top of the fundamentals: crawlable pages, clear answers, a sound sitemap. Do not expect it to move your visibility on its own. For the bigger picture see What is GEO? and SEO vs AEO vs GEO.

How can you check whether your site is ready?

Run the free llms.txt checker to see if your file exists and is well formed. A GraphoRank scan goes further: it includes llms.txt among its GEO readiness checks, alongside sitemap, AI crawlability and answerability, and the free audit covers the first 10 pages with no credit card. See How it works, or start a free scan.