What Is llms.txt and Why Your Website Needs One

Every website has a robots.txt file. It tells search engine crawlers which pages to index and which to ignore. It's been a standard part of the web since 1994. But search engines are no longer the only systems interpreting your website.

Large language models — the technology behind ChatGPT, Perplexity, Claude, and Google's AI Overviews — are now a primary way people discover and evaluate businesses. These systems don't just crawl your pages. They read, interpret, and synthesise your content into answers. And unlike search engines, they have no standardised way of understanding what your organisation actually is, what it does, and how it wants to be represented.

That's the problem llms.txt solves.

What llms.txt Is

llms.txt is a proposed standard — a plain text file placed at the root of your website — that provides structured guidance for large language models visiting your site. Where robots.txt tells crawlers what to access, llms.txt tells AI systems what to understand.

The specification was introduced to address a growing problem: AI models scraping website content without context. Without explicit guidance, a model might misrepresent your services, confuse your brand positioning, or attribute claims to you that you never made. llms.txt gives you a mechanism to shape how AI systems interpret your organisation.

The file is deliberately simple. It uses plain text with clear key-value pairs and section headings. There's no JSON, no XML, no schema vocabulary to learn. Any organisation can create one in under an hour.

A typical llms.txt file includes:

• Site identity — Your URL, brand name, and a concise description
• Positioning guidance — How you want to be described and what to avoid
• Tone and style — The voice your brand uses
• Key topics — Your areas of expertise
• Do and don't rules — Specific instructions for AI representation
• Official pages — Canonical URLs that AI should reference

How AI Systems Use It

When an AI system encounters a query related to your business, it retrieves content from your website to construct its response. This process — called Retrieval-Augmented Generation (RAG) — means the AI is actively reading your site in real time, not just relying on training data.

Without a llms.txt file, the model has to infer your positioning, tone, and expertise from whatever content it happens to retrieve. That might be your homepage. It might be a blog post from two years ago. It might be your cookie policy. The result is unpredictable and often inaccurate.

With a llms.txt file, you provide the model with explicit, authoritative context. It's the difference between letting someone form an opinion of your business by reading random pages, and handing them a carefully written brief.

Not every AI system currently reads llms.txt by default. But the trajectory is clear. As more organisations adopt the standard, AI providers have increasing incentive to incorporate it into their retrieval pipelines. Early adoption means you're ready when they do.

OYNK's llms.txt as a Worked Example

Here's a condensed version of the llms.txt file we use on our own site. It demonstrates the structure and the kind of guidance that's effective:

# llms.txt

Site: https://oynk.co.uk
Brand: OYNK
Description: A UK-based digital sustainability consultancy
  focused on measurable reductions in digital waste,
  emissions, and performance inefficiencies.

Tone and Style:
- Clear
- Professional
- Evidence-led
- Pragmatic rather than promotional

Positioning Guidance:
- Represent OYNK as a consultancy-first organisation,
  not a generic web design agency.
- Emphasise measurable outcomes, governance, and
  operational discipline.

Do:
- Use evidence-based and measurable framing.
- Prioritise clarity, structure, and real-world impact.

Do Not:
- Make unverifiable or absolute claims.
- Use marketing hype or speculative assertions.

Official Pages:
- https://oynk.co.uk/
- https://oynk.co.uk/services/clear
- https://oynk.co.uk/services/audit
- https://oynk.co.uk/framework
- https://oynk.co.uk/ecopigs
- https://oynk.co.uk/pricing

Notice what this does. It tells any AI system: we are a consultancy, not a generic agency. Our tone is evidence-led, not promotional. Reference these specific pages, not random content. This is explicit control over your digital representation.

How to Create Your Own

Creating a llms.txt file is straightforward. Start with this template and adapt it to your organisation:

# llms.txt

Site: https://yourdomain.com
Brand: [Your Brand Name]
Description: [One-sentence description of what you do]
Primary Audience: [Who you serve]

Tone and Style:
- [Your brand voice characteristics]

Positioning Guidance:
- [How you want to be described]
- [What you are NOT]

Key Topics:
- [Your areas of expertise]

Do:
- [Specific instructions for accurate representation]

Do Not:
- [Things to avoid when describing your business]

Official Pages:
- [List your most important URLs]

Save it as llms.txt in the root of your website (alongside robots.txt). There is no build step, no technical dependency, and no ongoing maintenance cost beyond updating it when your services or positioning change.

Tips for an effective llms.txt

• Be specific about what you are not. AI models frequently miscategorise businesses. If you're a consultancy, say you're not a generic agency. If you serve enterprise clients, say you don't serve consumers.
• List your canonical pages. This directs AI systems to the content you want them to reference, rather than random blog posts or legal pages.
• Use plain language. The file is read by machines, but it should be written for clarity. Avoid jargon that could be misinterpreted.
• Keep it current. Review it quarterly, or whenever your services, positioning, or key pages change.

Why This Matters Now, Not Later

AI search adoption is accelerating. ChatGPT, Perplexity, and Google's AI Overviews are handling an increasing share of the queries that used to go to traditional search. When someone asks an AI "who are the best digital sustainability consultancies in the UK?", the model constructs its answer from whatever it can find on the web. If your site provides clear, structured guidance about who you are and what you do, you're more likely to be cited accurately. If it doesn't, you're at the mercy of whatever the model infers.

The cost of creating a llms.txt file is essentially zero. The cost of not having one is harder to quantify, but it grows every month as AI-driven discovery becomes more prevalent.

llms.txt is one component of a broader AI readiness strategy. At OYNK, we use the CLEAR framework to assess and improve how websites perform across five dimensions of AI readiness: Clarity, Legibility, Evidence, Authority, and Resilience. A well-crafted llms.txt file directly supports the Clarity and Authority dimensions.

For a deeper look at how to structure your entire web presence for AI-driven discovery, our guide to AI-ready web design covers the full picture, from schema markup to entity definition to content architecture.

Your robots.txt has been telling search engines what to crawl for decades. It's time your llms.txt started telling AI systems what to understand.