llms.txt: What It Is, How to Create One, and Whether It’s Worth It

llms.txt is a Markdown file at the root of a website that summarizes what the site offers for AI agents. It is an open proposal, not a standard: Google says Search ignores the file, and no other AI company has officially confirmed that it uses it.

The topic gained momentum alongside GEO, or optimization for AI answers, and many people started treating the file as a shortcut. The llms.txt specification reached version 2 in August 2026 and defines the format, location, and intended use.

This guide, part of the marketing automation series, covers what Google and other AI companies say, how the file relates to robots.txt and sitemaps, how to create it in WordPress and Yoast SEO, an annotated template, common myths, and a checklist.

Key Takeaways

• llms.txt is a proposal by Jeremy Howard, published in September 2024 and revised in August 2026, to give AI agents a curated Markdown summary of a site.
• Google says Search ignores llms.txt, so it neither helps nor hurts. The official crawler pages from OpenAI, Anthropic, and Perplexity do not mention the file.
• The file is cheap to maintain and may help agents that read it, but it does not replace robots.txt, sitemaps, indexing, or useful content.

What Is llms.txt?

llms.txt grew out of a reading problem: pages built for people carry menus, ads, and scripts that get in a machine’s way. The file offers a plain-text shortcut written by the site owner.

A Proposal, Not an Official Standard

Jeremy Howard published the proposal on September 3, 2024, and revised it on August 10, 2026, in version 2 of the specification. The text itself presents the file as a proposal for standardization, open to community contributions.

No standards body stands behind it, unlike robots.txt, which is described in an IETF RFC. That is why no crawler is required to read llms.txt, and each service decides whether to use it.

What the File Was Designed For

According to the specification, llms.txt is read on demand, when an agent needs information to help a user, more than for training models. Its strongest use is in software documentation read by coding agents.

The same structure works for businesses that want to explain who they are, what they sell, and which policies they follow. It acts as a public, condensed version of an AI knowledge base, aimed at assistants like the ones covered in the guide to ChatGPT for business.

The llms.txt Format in Specification v2

llms.txt uses Markdown but follows a fixed section order, so it can be read by language models and by simple programs alike. Version 2 also defined where the file can live and how it can be found.

Required and Optional Sections

The only required section is an H1 heading with the name of the site or project. Then come, in this order, a summary in a blockquote, untitled paragraphs or lists, and H2 sections with link lists. The main elements are:

• H1: the site name, the only required part of the file.
• Blockquote (>): a short summary with the essentials needed to understand the rest.
• H2 sections: link lists in the [name](url) format, with an optional note after a colon.

The section named “Optional” is a convention for secondary links that an agent can skip when it needs to save context. In version 2, it no longer has a mechanical meaning and remains only as a good practice for organizing the file.

Folders, Markdown Versions, and Discovery Links

The file can sit at the root or in a folder, such as /docs/llms.txt, and covers the pages below it. If there is more than one, the most specific one applies. The proposal also suggests .md versions of pages at the same address, with the extension replaced or appended.

To help agents find these files, the version 2 changes recommend rel="alternate" with type="text/markdown" for the Markdown version and rel="describedby" for llms.txt, in HTML link tags or in the HTTP Link header.

What Google Says About llms.txt

Google has addressed llms.txt directly in two places. Search says it ignores the file. Lighthouse, Chrome’s auditing tool, now checks whether it exists, as an optional item.

Google Search Ignores the File

In its guide to optimizing for generative AI features, updated on July 10, 2026, Google lists llms.txt among the things that can be ignored. Creating and maintaining the file neither helps nor hurts visibility or rankings in Google Search.

The guide says there is no need to create files for machines to appear in Search, including its generative AI features such as Google AI Mode. Google may crawl other file types, but that does not mean they get special treatment.

Lighthouse: An Optional Audit in Chrome

The Lighthouse llms.txt audit sits in the Agentic Browsing category and looks for the file at the domain root. The documentation describes llms.txt as an emerging convention and says that, without it, agents may take longer to understand a site.

If the file does not exist and the server returns a 404, the result is marked not applicable, because the file is optional for now. The audit only flags a failure when the server errors out while trying to deliver it, and it measures agent readiness, not rankings.

What Other AI Companies Officially Say About llms.txt

Outside Google, the right question is whether any AI company has stated in official documentation that it reads other sites’ llms.txt files to answer or cite. In the pages checked on October 4, 2026, the answer was no.

Crawler Documentation Talks About robots.txt

OpenAI’s page about its crawlers explains OAI-SearchBot and GPTBot and how to control them through robots.txt, with no mention of llms.txt. Anthropic’s help center says its crawlers respect robots.txt directives, also without citing the file.

Perplexity’s crawler documentation follows the same path: it describes its agents and control through robots.txt. None of the three pages promises more citations or a better brand description to sites that publish an llms.txt.

Publishing llms.txt Is Not Reading llms.txt

OpenAI, Anthropic, and Perplexity each publish an llms.txt in their own developer documentation. That shows they use the format to guide coding agents through their manuals. It does not show that their search tools read other sites’ llms.txt files when building answers.

The honest position is to treat the file as a low-cost bet. To find out whether a brand appears in AI answers, the way is to measure with real questions, as the AI visibility monitor does.

llms.txt, robots.txt, and Sitemaps: Different Jobs

All three files live at the site root and are often confused, but they answer different questions. robots.txt says what can be crawled, the sitemap lists what exists, and llms.txt selects what is worth explaining first.

robots.txt Controls Crawling

robots.txt is described in RFC 9309, from September 2022, and must sit at /robots.txt, in lowercase, at the root. The standard itself warns that these rules are not a form of access authorization: they depend on the crawler choosing to follow them.

Google explains that robots.txt is mainly for managing crawler traffic and is not a way to keep a page out of Google; that is what noindex is for. llms.txt changes none of this and blocks no crawler.

A Sitemap Lists, llms.txt Selects

A sitemap tells search engines about a site’s pages, videos, and other files and how they relate, so they can crawl more efficiently. The llms.txt specification says a sitemap does not replace it, because a sitemap lists everything and often exceeds a model’s context.

In practice, robots.txt does not need to reference llms.txt, and llms.txt does not need to repeat the sitemap. The key is to list only pages that robots.txt allows and that can be indexed, so the file never points to a path the crawler cannot follow.

How to Create an llms.txt File Step by Step

Creating an llms.txt takes less time than choosing well what goes in it. The work is editorial: deciding which pages explain the business accurately and writing a short, verifiable description for each one.

Choose the Pages to Include

Start with the pages that answer the most common questions about the business: what it does, for whom, how much it costs, how to buy, and which policies it follows. Product, pricing, help, and core guides usually come first.

Use the same rule as when you train an AI on company data: include only current, checked information. A link to an outdated page can lead an agent to repeat wrong data, a risk similar to AI hallucination without a reliable source.

Write, Publish, and Test

Once the pages are chosen, building the file takes four steps, with or without WordPress:

1. Write the H1 with the company name and a one- or two-sentence blockquote about what it does.
2. Group the links into H2 sections, with a short note after each link.
3. Save it as llms.txt in UTF-8 and publish it at the root, at https://www.example.com/llms.txt.
4. Open the URL in a browser and confirm the server returns 200 with readable text.

The specification suggests a simple test: give an agent only the llms.txt and ask questions about the site’s content. If the answers come out incomplete or wrong, the problem usually lies in the descriptions or the choice of links.

llms.txt in WordPress and Yoast SEO

In WordPress, there are two paths: generate the file with Yoast SEO or upload an llms.txt written by hand. Both work. The point is not to let two generators compete for the same URL at the site root.

Generate llms.txt with Yoast SEO

According to the Yoast help center, the feature is under Yoast SEO, Settings, Site features, in the AI tools section, as the llms.txt option. In dashboards set to other languages, the labels may appear translated. Just turn on the toggle and save the changes.

According to Yoast’s technical specification, the file is updated every week and lists the 5 most recently updated pieces of content published in the last 12 months, with priority for cornerstone content, plus the 5 categories or tags with the most content.

Manual Files and Conflicts with Other Plugins

Pages can also be chosen by hand with the manual selection option. To write your own file, Yoast recommends turning off the feature, creating llms.txt at the root, and following llmstxt.org. An existing file is not overwritten.

The feature is not available on multisite, and the plugin adds the UTF-8 byte order mark (BOM) by default, which the specification allows as optional. A manual file gives control over every link; the automatic one favors whatever changed last.

An Annotated llms.txt Template

The template below uses a fictional company and generic URLs. It follows the order of specification v2 and can be adapted by replacing the bracketed fields with real data, without adding promises the site cannot back up.

The Template Line by Line

# [Company Name]

> [Company Name] offers [what the company does] for [audience], with service in [region or channel].

Important information:
- Official prices and terms are only on the pricing page.
- [Rule an agent should follow when talking about the company.]

## Products and services
- [Service name](https://www.example.com/service/): what it solves and for whom
- [Plans and pricing](https://www.example.com/pricing/): current prices and terms

## Help and policies
- [Help center](https://www.example.com/help/): FAQs and setup
- [Privacy policy](https://www.example.com/privacy/): how personal data is handled (LGPD, GDPR)

## Optional
- [Blog](https://www.example.com/blog/): guides and educational articles

The first line is the H1, the only required part. The blockquote right below should fit in one or two sentences. The untitled lines that follow record rules the agent needs to know before following the links, such as where the official prices are.

Each H2 section groups links by topic, always in the [name](url) format, followed by a colon and a short note. The Optional section holds what can be skipped, and its name follows the specification’s convention.

Common Mistakes When Filling In the Template

The most frequent mistakes happen when the file turns into advertising instead of an index:

• Marketing copy: adjectives and promises in the summary instead of verifiable facts.
• Broken or blocked links: pages with noindex, redirected, or blocked in robots.txt.
• A bloated file: dozens of links without notes that just repeat the sitemap.

The summary should use the same name, description, and data the company shows on its site and profiles. This entity consistency is one of the pillars of the guide on how to show up in ChatGPT.

llms.txt Myths

Much of what circulates about llms.txt mixes the proposal with promises no official source supports. The most common myths fall into two groups: rankings and citations, and access control.

Ranking and Citation Myths

Myth: llms.txt improves Google rankings. Google’s Search guide says the opposite: the file is ignored and neither helps nor hurts. The comparison of GEO, AEO, and SEO shows why the groundwork is still SEO.

Myth: with llms.txt, ChatGPT starts citing the business. None of the official crawler documentation reviewed promises that. Citations depend on access, useful content, and reputation, and what counts is what shows up in the answers, not the file you publish.

Control and Security Myths

Myth: llms.txt blocks AI from using your content. It has no blocking rules. Access is a robots.txt matter, and RFC 9309 notes that even those rules are not authorization. Confidential content needs a login.

Myth: an N/A result in Lighthouse is a penalty. Chrome’s documentation says that without the file the audit is marked not applicable, because llms.txt is optional. It only raises an alert when the server errors out while trying to deliver it.

llms.txt Publishing and Maintenance Checklist

The checklist below sums up publishing and maintaining the file. It applies to a hand-written llms.txt and to one generated by Yoast, whose automatic selection does not always match the priority pages.

Before Publishing

Before publishing, check each item below:

• Format: H1 with the name, a blockquote summary, and links in the [name](url): note format.
• URL: file at /llms.txt, plain text in UTF-8, a 200 response, and no redirect.
• Links: all on HTTPS, indexable, allowed in robots.txt, and with current content.
• Content: verifiable facts, no promises, and prices only as a link to the official page.

Also make sure only one generator is active. Yoast warns that if another plugin serves the file dynamically, that plugin’s file takes priority at the URL. Conversion pages, such as landing pages for WhatsApp, belong in the file only if they are indexable.

Maintenance and Measurement

The file has to keep up with the site: an important new page, a price change, or a discontinued service calls for a review. A monthly routine, together with a review of local SEO and the company’s profiles, may be enough for small sites.

To see whether any crawler accesses the file, check the server or CDN logs. The effect on answers is measured with real questions, as in the method to measure AI visibility, not by whether the file exists.

Is llms.txt Worth It?

For most businesses, the answer is that it is fine to have one, as long as the file does not replace the work that actually matters for AI answers. The cost is low, and so far, so is the proven benefit in search.

When llms.txt Makes Sense

It makes the most sense on sites with documentation, a help center, or many guides, where agents need to find the right page fast. The difference between a chatbot and an AI agent explains why agents look up sources instead of following menus.

Businesses that already use AI agents in their own customer service have this knowledge organized. Anyone who has built an AI business brain to answer customers has much of the llms.txt ready: the pages and rules the internal agent checks every day.

What to Prioritize Before the File

Before llms.txt come a correct robots.txt, indexable pages, original content, consistent data, and off-site reputation, such as Google reviews. Google notes that no third-party tool has access to its internal ranking or AI systems.

ConverZAP, a Tech Provider verified by Meta, works both sides of this journey in its AI marketing solutions: the monitor tracks how the brand appears in AI answers, and the AI agents for WhatsApp serve the people who arrive through them, on the official WhatsApp Business API. The AI visibility solution shows how the monitoring works, and the free trial lets you start with your own brand.

llms.txt: Frequently Asked Questions