How to Add llms.txt for AI Search Optimization in Record Time

Learn how to add llms.txt & llms-full.txt to boost your website’s visibility in AI search engines like ChatGPT, Perplexity, and Deepseek. A step-by-step guide to structuring your content for AI indexing in under 60 minutes.

Mathias Ptacek•
llms.txt guideAI website optimizationLLMOGEOLarge Language ModelsAI content structuringenhancing AI interaction
How to Add llms.txt for AI Search Optimization in Record Time

How I Added llms.txt & llms-full.txt in No Time - And Why You Should Too

Rankscale is all about fast and agile improvements that sets it apart in the ever-changing AI landscape. Recently, I rolled out llms.txt and llms-full.txt!

Here’s a behind-the-scenes look at what they are, how they work, and a quick rundown of the implementation process inspired directly by the guidelines at llmstxt.org [2].

What’s the Deal with llms.txt?

llms.txt is a markdown-formatted text file that lives in your site’s root directory. Its job? Give AI a structured summary of your site’s content—without all the extra clutter. With llms.txt, you tell AI exactly what your site is about, ensuring it picks up the right information.

Key Takeaway: Traditional crawlers (like Google) are great at indexing websites, but AI-based search engines and their simple crawlers often ignore JavaScript-rendered content. That means sites built on React, Vue, Angular, or similar frameworks might not show up properly in AI-generated responses. So in LLMO terms, by using llms.txt and llms-full.txt you can boost your relevance in AI-driven search results dramatically.

Why This Matters

If you want AI-powered search engines and chatbots (like ChatGPT, Perplexity, DeepSeek or Claude) to pull accurate information from your site, llms.txt is essential. Here’s why:

  • AI Search Engines Are on the Rise
    AI search results are becoming the more and more relevant in 2025 (e.g. ChatGPT has just unlocked Web Search for free, nonregistered users [3]) — and they work differently from traditional search engines. While Google indexes all pages, most LLMs don’t process JavaScript-rendered content. If your site relies on client-side rendering, it’s effectively invisible to AI search tools in the field of Generative Engine Optimization (GEO).

  • Your Competitors are already optimizing for AI
    Looking at the directory.llmstxt.cloud, more and more companies are adapting to the standard [2]

  • Accurate Information = Better Visibility
    Without a structured file, LLMs might misinterpret or completely miss your content. You don’t want AI chatbots making assumptions about your business, products, or services - as your customers already use AI answers heavily.

  • Faster AI Indexing:
    With our new llms.txt files, AI systems can immediately pick up what our site is about. This means better visibility and more accurate search results for client-rendered content that might otherwise be overlooked.

  • It’s Fast, Easy, and Future-Proof
    Creating llms.txt is ridiculously simple - it’s just a well-structured markdown file. But that small step future-proofs your site for AI-driven search and discovery. A few minutes of work, a couple of new files, and we’re ahead in the digital game!

How to do it: Following the llms.txt Format

I based my implementation on the straightforward, effective format specified at llmstxt.org [1] while looking at other implementations at directory.llmstxt.cloud [2]

Here’s the step-by-step breakdown:

  1. Create a llms.txt file in your root directory

    • Open a plain text editor (VS Code, Notepad++, or any IDE).
    • Save the file as llms.txt in UTF-8 encoding.

  2. Follow the Recommended Format, here my Rankscale example (simplified):

    • Start with an H1 header ("# YourSiteName - optional tagline").
    • Add a blockquote summary ("> A short description of your site").
    • Include structured sections ("## YourSection") with links to important content.
      # Rankscale.ai - Monitor Your Website or SaaS or Brand or Service or Product or Client in AI Search Engines

> Analyze and Optimize your AI Search Visibility with Results Tracking, Deep Insights and Actionable Recommendations - in Seconds.

Here's what you can do with Rankscale:

- Audit your website to see how AI understands it
- Identify Gaps and Content Optimization Potential
- Track your Presence in AI Search Results over Time
- Analyze AI Result Citations & Brand Sentiments
- Uncover, Survey, and Outpace Competitors
- Rankscale works accross: ChatGPT, AI Overviews, Perplexity, Claude, Gemini, Deepseek

[Rankscale Landing Page](https://rankscale.ai): Rankscale's landing page.

## Rankscale Features
- [Features](https://rankscale.ai/features.md): Learn about the features of Rankscale.

## Why use Rankscale?
- [Why Rankscale?](https://rankscale.ai/whyrankscale.md): Learn why Rankscale is the best tool for AI search compared to traditional SEO tools.

## Frequently Asked Questions
- [Frequently Asked Questions](https://rankscale.ai/faq.md): Answers to common questions about Rankscale, AI search, and Generative Engine Optimization (GEO).

## Blog
- [RankScale Blog](https://rankscale.ai/blog): AI SEO, visibility tracking, and brand monitoring insights.  Learn about AEO, GEO, and AI search optimization.

## Blog Posts
- [AI SEO: Mastering AEO & GEO](https://rankscale.ai/blog/explosive_facts_and_steps_to_master_ai_search_engine_optimization_in_2025): Master AI search engine optimization (AEO and GEO) in 2025: a guide.
- [DeepSeek R1: Revolutionizing AI](https://rankscale.ai/blog/deepseek_r1_revolutionizing_ai_with_transparent_reasoning_and_integrated_web_search): DeepSeek R1: Revolutionizing AI with Transparent Reasoning and Integrated Web Search
       

  3. Add markdown files to extend/route to in your sections

    • In "## YourSection", integrate a link (format:  "[title](URL):description") to the extended markdown content by using an additional "*.md" file --> See my "features.md" example for "- [Features](https://rankscale.ai/features.md): Learn about the features of Rankscale" below.
      # Features

## Website Audits

Perform In-Depth AI Audits of Your Website
Uncover how AI search engines perceive your content, pinpoint gaps, and discover optimal paths to greater visibility.

- Content Scoring: Get detailed insights on relevance, authority, engagement, and technical structure that affect AI search visibility.
- Gap Analysis & Recommendations: Pinpoint which areas need improvement — with content recommendations on critical sections.
- Comprehensive Crawls: Select between single-page or multi-page crawling to fully reveal your site's AI readiness.
- Customizable Bot Settings: Simulate AI bots accurately by toggling anti-bot measures for realistic audits

## Performance Tracking

Monitor and Understand Your AI Presence
See where and how often your brand appears in AI search results.

- Specify your own search terms: Design custom AI search prompts that fit to your brand and industry - or leverage AI to propose them for you.
- Real-Time Updates: Analyze trends and changes to your brand's AI search rankings accross time, topics and regions.
- Cross-Channel Insights: Get a unified view and compare results across different AI answer engines and industry topics.
- Sentiment Context: Understand how AI search engines perceive your brand and industry topics.

## Competitor Benchmarking

Find and Track your Competitors
Identify the key players in your AI search landscape, monitor their performance, and stay one step ahead.

- Competitor Discovery: Automatically uncover direct and indirect competitors for relevant AI search terms and industry topics.

- Side-by-Side Comparisons: Compare visibility, sentiment, rankings and other important metrics to gain a holistic view of your competitors.
- Competitor Citations: See how often your competitors are cited and which sources are responsible for their visibility.
- Benchmarking: Identify Competitor's strenghts and weaknesses accross industry topics and AI engines.
[...]

  4. Upload the llms.txt and  "*.md" file(s) to your site's root directory

    • If using cPanel or FTP, place it at /public_html/llms.txt or /public/llms.txt
    • If using Git, push it to your production build.
    • This makes them publicly accessible to AI crawlers.
    • Optionally you could create a path in your sitemap.xml as well (just a personal opinion)

  5.  Verify accessibility

    • Open https://yourwebsite.com/llms.txt in your browser.
    • Ensure it displays correctly and isn’t blocked by robots.txt.

  6. Optionally, add llms-full.txt for detailed AI indexing

    • Copy all structured markdown from your site into llms-full.txt.
    • This provides AI engines a full content snapshot for deeper indexing.

  7. Update as needed

    • Just like robots.txt, keep it fresh and accurate as your site evolves.
    • Use tools like Mintlify to automate the creation and any updates [4]

The Bottom Line

If your website relies on JavaScript, or you simply want to ensure AI models interpret your content correctly, adding llms.txt or even llms-full.txt is a quick, no-brainer improvement. I did it in minutes (ok, ok, let's say <60 minutes - I'm a slow thinker), and now Rankscale's content is properly structured for AI.

Want your site to rank higher in AI search results?

  1. Follow this guide
  2. Implement llms.txt
  3. Optimize and monitor your AI search visibility with Rankscale.ai [5]

Sources

Back to Blog