Illustration of an llms.txt file being read by an AI assistant

An llms.txt file is a proposed, simple way to tell AI agents what your site is and where they should look for clean information. It is a Markdown file, usually published at https://yoursite.com/llms.txt. The idea comes from a public proposal by Jeremy Howard, maintained at llmstxt.org. This article summarises the v2 proposal as it stood on the page we opened, and then walks through how to write a small, honest file for a blog or a documentation site.

Why the file exists

Web pages are built for people. They come wrapped in navigation, ads, and scripts, and they can be awkward for an automated agent to turn back into clean text. Context windows are also limited, so dumping an entire site into a model is often wasteful. The llms.txt idea is to put a short, curated overview in one known place, then link out to more detailed, ideally Markdown, pages that an agent can fetch only when it needs them.

The proposal is modelled in spirit on /robots.txt and /sitemap.xml: a conventional filename, at the site root or at a subpath such as /docs/llms.txt. It does not replace those files. A sitemap lists pages for search engines. A robots.txt file says what automated access is acceptable. An llms.txt file is meant to be used on demand, when an agent is actually trying to answer a question about your content.

The format, in order

According to llmstxt.org, a file that follows the spec contains these parts, in this order:

  1. An optional byte-order mark.
  2. An H1 with the name of the project or site. This is the only required section.
  3. A blockquote with a short summary, including key facts needed to understand the rest of the file.
  4. Optional Markdown (paragraphs, lists, and so on, but not extra headings) with more detail about the project.
  5. Optional H2 sections that contain “file lists”: Markdown bullet lists of links. Each item has a required link, then optionally a colon and a short note.

By convention, a section titled “Optional” is used for secondary links that an agent can skip if it only needs a short context. Links should point to LLM-friendly content where possible, such as a Markdown version of a page. The proposal also suggests that important pages can offer a clean Markdown copy at the same URL with .md added or with the extension replaced, and that HTML or HTTP Link headers can point to those files and to the llms.txt that covers them.

A small example for this kind of site

You do not need a documentation platform to write a useful file. For a blog, a short version might look like this (edit the details so they stay true):

# Informative Sisters

> A family-run site with articles on science, technology and marketing, and free tools such as calculators and passport-photo helpers. Educational content only; not professional medical, legal, or financial advice.

Start with the most specific page for the question. Official rules (for example, government passport requirements) should be checked on the issuing authority’s website, not only on this site.

## Site
- [Home](https://informativesisters.com/): Latest posts
- [About](https://informativesisters.com/about/): Who runs the site
- [Free tools](https://informativesisters.com/free-tools/): Calculators and photo helpers
- [Disclaimer](https://informativesisters.com/disclaimer/): Limits of the information

## Optional
- [Privacy policy](https://informativesisters.com/privacy-policy/)
- [Contact](https://informativesisters.com/contact/): enquiries@informativesisters.com

Keep the tone factual. Do not stuff the file with slogans or with every post you have ever published. The point of the format is to stay small enough to fit in context and to send the agent to the right deeper page.

How to publish it on WordPress

  1. Write the file in a plain-text editor, using a .txt extension, in UTF-8.
  2. Upload it so that it is served at /llms.txt on your domain (many hosts allow this in the site’s root, or via a small plugin). This site already has a file at that address; update it rather than creating a second copy.
  3. Open the URL in a browser and confirm that you see Markdown, not an HTML error page, and that the file is not blocked in robots.txt.
  4. Ask a question about your site in an assistant that can fetch URLs, giving it only the llms.txt file as a starting point, and see whether it finds the right pages. The official proposal recommends that kind of test.
  5. Revisit the file when you add a major section, such as a new tools collection or a disclaimer page, so the links do not go stale.

llmstxt.org notes that some documentation platforms and some WordPress SEO plugins can generate a file automatically. Automatic generation is fine if you then read the result. A generated file that lists hundreds of thin URLs, or that misdescribes the site, is less useful than a short file you curated.

This format is a community proposal, not a law, and it will not by itself improve your search ranking. Used modestly, it is still a clear way to tell agents what the site is for and where the reliable pages are.

How it sits next to other site files

The official proposal is explicit about what llms.txt is not. It is not a substitute for a sitemap, because a sitemap may list more pages than will fit in a model’s context, may omit Markdown versions, and generally does not include useful off-site references. It is not a substitute for robots.txt, which is about permission, not explanation. And it was not designed as a well-known URI under /.well-known/, because many authors can publish a file in their own directory (for example, a project site) but cannot write to the host’s well-known path. Like index.html, an llms.txt file describes the path where it sits.

When you write yours, prefer concise descriptions, avoid unexplained jargon, and keep the links you would want an agent to follow first at the top. If you maintain documentation, consider offering Markdown copies of those pages and pointing to them from the file, as the proposal recommends. For a general-interest blog, honest short summaries and a small set of important URLs are enough.

Sources

The pages below were opened and checked while this rewrite was prepared (21 November 2026, Asia/Calcutta). Official figures and rules can change. If a live page differs from what is summarised here, follow the official page.

Ashok Kumar

Ashok Kumar is the marketing lead at Informative Sisters. He brings campaign and strategy experience to the site’s business and marketing content.