LLM-Index Explained
From XML Sitemaps to Intelligent Discovery Hubs
Discover how LLM-Index transforms traditional sitemaps into intelligent discovery hubs, delivering 93% token savings and20x faster navigation for AI agents.
🚀 Experience the Revolution
See how an intelligent discovery hub transforms agent navigation from crawling to precision routing
Navigation Guide
Understanding the Revolution
Implementation & Tools
What is LLM-Index?
An Intelligent Discovery Hub for AI Agents
LLM-Index transforms your traditional sitemap from a simple URL list into a smart navigation system designed specifically for AI agents and LLMs.
❌ Traditional Sitemap (XML)
- • Simple URL list
- • No context or intent
- • HTML-focused discovery
- • No audience targeting
- • No trust signals
- • Static structure
✅ LLM-Index (Intelligent)
- • Organized by audience & intent
- • Rich metadata & context
- • API and capability focused
- • Smart routing recommendations
- • Cryptographic trust levels
- • Dynamic with analytics
The Evolution Story
📄 Era 1: HTML Crawlers (1990s-2010s)
sitemap.xml was perfect for search engines crawling HTML pages. Simple, effective, but designed for human-browsable content.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://site.com/page</loc>
<lastmod>2025-01-01</lastmod>
<changefreq>monthly</changefreq>
<priority>0.8</priority>
</url>
<url>
<loc>https://site.com/about</loc>
<lastmod>2024-12-15</lastmod>
<changefreq>yearly</changefreq>
<priority>0.5</priority>
</url>
</urlset>🤖 Era 2: AI Agents (2020s+)
AI agents need more than URLs - they need intent, capabilities, and trust signals. Traditional sitemaps leave agents crawling blindly, wasting tokens and time.
{
"feed_type": "llm-index",
"metadata": {
"title": "Intelligent Discovery Hub",
"origin": "https://site.com",
"generated_at": "2025-06-22T10:30:00Z"
},
"smart_routing": {
"audience_based": {
"developer": {
"entry_point": "/.well-known/mcp.llmfeed.json",
"token_budget_allocation": {
// ... (content truncated for display - use copy button for full content)Quantified Performance Benefits
❌ Traditional Crawling
✅ LLM-Index Navigation
🎯 Economic Impact by Scale
How It Works
🎯 Smart Categorization
Instead of a flat list, feeds are organized by purpose and audience:
Core Infrastructure
Essential feeds for understanding the site
Developer Tools
Interactive tools and implementation guides
Documentation
Technical specs and reference materials
🧭 Intelligent Routing
Agents receive customized navigation paths based on their purpose:
🔐 Trust-Based Prioritization
Cryptographic signatures enable agents to make autonomous decisions:
Implementation Guide
🚀 Quick Start (5 Minutes)
Create a basic LLM-Index for your site:
{
"feed_type": "llm-index",
"metadata": {
"title": "{{My Site Discovery Hub}}",
"origin": "https://{{mysite.com}}",
"generated_at": "{{2025-06-15T00:00:00Z}}",
"description": "Intelligent navigation hub for AI agents"
},
"discovery_guidance": {
"recommended_entry_points": {
"new_visitors": "/.well-known/mcp.llmfeed.json",
"returning_agents": "/.well-known/llm-index.llmfeed.json"
},
"navigation_strategy": "audience_based"
},
// ... (content truncated for display - use copy button for full content)Save as /.well-known/llm-index.llmfeed.json
🎯 Progressive Enhancement
Phase 1: Basic Index
Simple feed listing with minimal metadata
Phase 2: Smart Routing
Add categories and audience-based navigation
Phase 3: Full Intelligence
Analytics, relationships, and automated updates
Tools & Automation
📋 Manual Creation
For simple sites, create your LLM-Index manually using our template.
🤖 Certified Prompt
Use our signed prompt to generate your LLM-Index with any LLM.
🔮 Coming Soon: Automated Tools
Ready to Transform Your Site Discovery?
Join the intelligent web revolution and give your site the navigation system AI agents deserve.