LLMs.txt Implementation

This repository implements the llms.txt specification to provide LLM-friendly access to documentation.

What is llms.txt?

The llms.txt specification provides a standardized way to make website documentation accessible to Large Language Models (LLMs). It includes:

1. /llms.txt file: A curated overview with links to key documentation
2. .html.md versions: Clean markdown versions of HTML pages at the same URL with .md appended

Implementation

1. LLMs.txt Generation

Script: docusaurus/scripts/generate-llms-txt.cjs

Automatically generates /llms.txt following the specification:

• Title: "Pacing Agency Tech Stack Documentation"
• Summary: Brief overview in blockquote format
• Sections:
  • Core Documentation (architecture, intro, tools overview)
  • Key Tools (GTM, Webflow, n8n, GA4, BigQuery, etc.)
  • Implementation Guides (marketing optimization, server-side tracking, self-hosting)
  • Docusaurus Development (contributor documentation)
  • Optional (less critical resources)

Build Integration:

{
  "scripts": {
    "prebuild": "node ./scripts/sync-tools.cjs && node ./scripts/validate-mdx.cjs && node ./scripts/generate-llms-txt.cjs",
    "generate-llms": "node ./scripts/generate-llms-txt.cjs"
  }
}

The llms.txt file is automatically regenerated on every build (local and Cloudflare Pages).

2. Markdown Export Plugin

Plugin: docusaurus/plugins/markdown-export-plugin.ts

Generates clean .html.md versions of all documentation pages during build:

• Input: Source .md/.mdx files in docs/
• Output: .html.md files in build/docs/ mirroring the URL structure
• Features:
  • Strips frontmatter (optional)
  • Adds custom header comment
  • Preserves markdown content

Configuration in docusaurus.config.ts:

plugins: [
  [
    './plugins/markdown-export-plugin.ts',
    {
      includeFrontmatter: false,
      header: '<!-- This is a plain markdown version of the documentation page for LLM consumption. Visit https://docs.pacing.agency for the full interactive version. -->',
    },
  ],
],

URL Patterns

llms.txt File

• URL: https://docs.pacing.agency/llms.txt
• Location: docusaurus/static/llms.txt
• Serves: Curated overview with links to key documentation

Markdown Versions

All documentation pages have corresponding .html.md versions:

HTML Page	Markdown Version
/docs/intro.html	/docs/intro.html.md
/docs/tools/gtm.html	/docs/tools/gtm.html.md
/docs/Guides/server-side-tracking-guide/	/docs/Guides/server-side-tracking-guide/index.html.md

Testing

Test file: docusaurus/scripts/__tests__/generate-llms-txt.test.cjs

Run tests:

# Run all tests including llms.txt
npm test

# Run only llms.txt tests
npm run test:llms

# Generate llms.txt manually
npm run generate-llms

Tests verify:

• Content generation
• Format compliance with llms.txt specification
• Required sections (title, summary, sections)
• URL format (.html.md extension)
• Key tool links
• File output

Benefits

For LLMs and AI Assistants

1. Quick Context: /llms.txt provides curated overview without parsing entire site
2. Clean Markdown: .html.md versions are stripped of HTML/JavaScript/ads
3. Standardized Format: Follows specification for consistent processing
4. Direct Access: No need to scrape or parse complex HTML

For Developers

1. Automatic: Regenerates on every build (local and production)
2. Comprehensive: Covers all documentation automatically
3. Maintainable: No manual updates needed - pulls from source files
4. Tested: Full test coverage ensures specification compliance

For Users

1. Better AI Assistance: AI coding assistants (Cursor, GitHub Copilot) can read documentation more effectively
2. Accurate Context: AI gets precise, up-to-date information
3. Faster Development: AI can find answers without guessing

Specification Compliance

This implementation fully complies with the llmstxt.org specification:

✅ Required:

• H1 title as first line
• Located at /llms.txt

✅ Recommended:

• Blockquote summary after title
• Detail paragraphs/lists before sections
• H2 sections with markdown lists of links
• "Optional" section for secondary information
• .html.md versions of documentation pages

✅ Format:

• Markdown syntax throughout
• [Link title](url): Optional description format
• No HTML or complex formatting

Maintenance

Adding New Tools

New tools added to tools/*.md automatically appear in:

1. The generated /llms.txt file (if marked as a key tool in the script)
2. The "Complete Tool List" link in the Optional section
3. The .html.md export after build

To add a tool to the "Key Tools" section in llms.txt:

1. Edit docusaurus/scripts/generate-llms-txt.cjs
2. Add the tool slug to the keyToolSlugs array:

const keyToolSlugs = [
  'gtm',
  'webflow',
  'n8n',
  'your-new-tool', // Add here
  // ...
];

Adding New Guides

New guides added to docusaurus/docs/Guides/ automatically appear in:

1. The "Implementation Guides" section of llms.txt
2. The .html.md export after build

No manual updates needed - the script scans the Guides directory.

Customizing Content

To customize the llms.txt content:

1. Edit the summary blockquote in generateLlmsTxt() function
2. Modify section descriptions
3. Add/remove sections as needed
4. Update the "Important Context" bullets

Directory Listings

The llms.txt file is automatically indexed by:

• llmstxt.site
• directory.llmstxt.cloud

These directories help LLMs discover documentation following the specification.

Integration with Other Tools

Cursor AI

Cursor AI automatically reads /llms.txt files to understand documentation structure. This improves:

• Context awareness when editing files
• More accurate code suggestions
• Better understanding of project architecture

GitHub Copilot

While GitHub Copilot doesn't natively support llms.txt (yet), the .html.md versions can be manually referenced in prompts or included in context.

Custom Tools

The clean markdown versions can be used for:

• Custom documentation search tools
• RAG (Retrieval-Augmented Generation) systems
• Documentation chatbots
• Training custom models

References

• llms.txt Specification
• llms.txt GitHub Repository
• Community Discord
• Example: FastHTML docs llms.txt