What is llms.txt?
llms.txt is a plain text file that sits at the root of a website, at an address like yoursite.com.au/llms.txt. It gives AI tools a short, clean summary of who you are and a list of links to the pages that matter most. It's written in Markdown, the simple formatting used in plain text notes.
The idea was proposed by Jeremy Howard, co-founder of Answer.AI, on 3 September 2024. His reasoning was simple. Websites are full of menus, scripts, pop ups and ads. An AI model reading a page has to dig through all of that to find the useful part, and it can only hold so much text at once. A tidy file that says "here's what this site is, and here's where the good stuff lives" saves it the digging.
It's a community proposal, not an official web standard. Nobody has to support it, and as you'll see below, plenty of AI companies haven't said whether they do.
What goes in an llms.txt file?
Only one thing is required: a heading with the name of your business or site. Everything else is optional but useful. Here's how an llms.txt file is laid out, part by part, using a made up business called Example Electrical as the sample.
| Part of the file | What it is | Example |
|---|---|---|
| Title line | A heading, written as one hash symbol then your business name. The only required part | A hash, then Example Electrical |
| Summary | A short quote block, a line that starts with a greater than symbol, saying what you do and where | Licensed electricians in Perth's northern suburbs, from Joondalup to Yanchep. Emergency call outs, switchboards, solar and EV chargers. |
| Key details | A few plain sentences or a short list of facts you want AI tools to get right | Licence number, service area, hours, phone number, starting prices |
| Section heading | A heading written with two hash symbols, grouping your links | Services |
| Link lines | One line per page: the page name in square brackets, its address in round brackets, then a colon and a short note | Switchboard upgrades, linked to that page, with the note: what's included and typical prices |
| More sections | Other groups of links | About, Pricing, Service areas, FAQ |
| Optional section | A last section actually named Optional, for less important links AI tools can skip | Blog posts, older news, policies |
Keep it short. For a small business, 20 to 40 lines is plenty. It's a map of your site, not a copy of it.
How is llms.txt different from robots.txt and a sitemap?
robots.txt tells crawlers what they may and may not visit. A sitemap lists every page so search engines can find them. llms.txt is a short, hand-written summary that points AI tools at your best pages. They do different jobs.
| File | What it does | Who reads it | Required? |
|---|---|---|---|
| robots.txt | Allows or blocks crawlers from parts of the site | Major search engine and AI crawlers check it | No, but almost every site should have one |
| sitemap.xml | Lists the pages you want indexed | Google, Bing and other search engines | No, but strongly recommended |
| llms.txt | Summarises the business and links to key pages for AI tools | Some AI tools and agents, support varies | No, entirely optional |
The one that matters most for AI search is robots.txt. OpenAI and Perplexity both say you need to allow their crawlers, OAI-SearchBot and PerplexityBot, to appear in their search answers. If robots.txt blocks them, an llms.txt file won't change that.
Which AI tools actually read llms.txt?
Honest answer: support is uneven, and most companies haven't said clearly either way. Here's where things stand as of November 2026.
| Company | What they've said or done |
|---|---|
| Says Google Search doesn't use llms.txt. Its guide to generative AI features, updated July 2026, says the file won't help or harm your Search visibility, and that it's fine to keep one for other services. Google's Gary Illyes also said in July 2025 that Google doesn't support it. Separately, Google's Lighthouse testing tool added an optional llms.txt check in May 2026, in a new section about AI browsing agents. | |
| OpenAI | No public statement that ChatGPT reads other sites' llms.txt files. OpenAI's crawler documentation focuses on robots.txt and its OAI-SearchBot crawler. |
| Anthropic | Publishes an llms.txt file for its own developer documentation. Some industry reports say Claude's tools use llms.txt, but treat that as unconfirmed for ordinary business sites. |
| Perplexity | Also publishes an llms.txt file for its own documentation. Its crawler guidance for site owners is about robots.txt and allowing PerplexityBot. |
| Coding assistants and AI agents | The clearest real use today. Some developer tools and agents read llms.txt on documentation sites to pull in clean context. |
So llms.txt is a low cost bet, not a proven ranking factor. Industry studies so far disagree on whether it changes AI citations at all, and many come from companies selling AI visibility tools. Treat any claimed uplift with caution.
Is llms.txt worth adding to a small business website?
Yes, if it's cheap or included, because it takes about an hour and Google says it does no harm. No, if someone is charging you hundreds of dollars for it or selling it as the fix for AI search.
- Worth doing: you already have a solid site with a page per service, and you want every AI tool and agent that does read the file to get your facts right.
- Not a priority yet: your robots.txt blocks AI crawlers, your pages don't state prices or areas, or your Google Business Profile is half empty. Fix those first. They matter far more.
Think of llms.txt like a business card left at reception. Useful for whoever picks it up. It won't bring visitors in by itself.
How do you write an llms.txt file?
Write it by hand from your own pages, keep it short and keep it current. Here's the process I follow:
- List your 10 to 20 most important pages. Home, each service, pricing, service areas, about, contact and FAQ.
- Write a one or two sentence summary of what you do, where and for whom. Plain words, no slogans.
- Add the facts you want quoted correctly: phone number, licence or ABN, service area, hours and starting prices.
- Group the links under short headings, like Services, Pricing and About, with a short note after each link.
- Put less important links under a final Optional heading.
- Save it as plain text named llms.txt and upload it to the root of your site, so it loads at yoursite.com.au/llms.txt.
- Check robots.txt doesn't block AI crawlers, and update the file whenever prices, services or pages change.
Some website builders don't let you upload a file to the root of your site. If you're on one of those, it may need a workaround or a developer, so ask before you pay anyone for it.
What mistakes should you avoid with llms.txt?
Most llms.txt problems come from treating it as a keyword dump or forgetting it exists. Watch for these:
- Blocking AI crawlers in robots.txt while publishing llms.txt. The crawler rules win.
- Linking to pages that don't exist or redirect. Check every link loads.
- Stuffing it with keywords. It's read by models that understand plain language. Long lists of suburbs and search phrases add nothing.
- Saying something your site doesn't. If llms.txt says one price and your pricing page says another, you've given AI tools two answers.
- Letting it go stale. Old prices and dead services in llms.txt are worse than no file.
- Treating it as your whole AI search plan. It's one small file. Clear pages, real numbers and crawler access do the heavy lifting, which our guide to generative engine optimisation covers.
We publish llms.txt on every build
Every website we build ships with an llms.txt file, written by hand from the site's own pages, alongside robots.txt rules that let the main AI crawlers in and structured data that matches the page.
Two examples. Nailed It Creations, a Brisbane custom home builder, has llms.txt alongside 87 questions in FAQ schema across 19 pages. Koorta Boodjar, an Aboriginal-owned labour hire and workforce business in Balcatta, has llms.txt and 41 FAQ schema questions across its 12 page site. Both launched in September 2026.
If you want one added to your current site, it's part of our AEO implementation work, which you can read about on our AI SEO agency page. Or see how it fits into a new build with our website design service.
