docs: structured llms.txt index + AI docs improvements by tippi-fifestarr · Pull Request #379 · aptos-labs/aptos-docs

tippi-fifestarr · 2026-02-24T19:46:54Z

Summary

Structured llms.txt index: Replaces the default plugin output (just 2 links to blob files) with a Decibel-style structured index listing every doc page with title, description, and per-page .md URL — grouped by section (##) with auto-derived sub-sections (###) from path depth
New integration (llms-txt-index.ts): Uses astro:route:setup hook to override the plugin's /llms.txt route while leaving llms-full.txt and llms-small.txt untouched
AI overview page: Added AskAptos chatbot section, updated llms.txt feed descriptions to mention per-page .md links
LLMs.txt usage page: Added Claude Code usage section (was missing), documented the per-page .md URL feature
Homepage: Fixed AskAptos chatbot link from (#) placeholder to /build/ai
Translations: All changes translated to ES and ZH

Builds on the foundation from #375. The key insight is that the default llms.txt index only links to blob files — agents can't tell which pages are relevant without ingesting everything. The structured index lets them pick specific pages via .md URLs.

Test plan

pnpm dev → verify http://localhost:4321/llms.txt shows structured index with #/##/### hierarchy
Verify /llms-full.txt and /llms-small.txt still work (plugin routes untouched)
Verify per-page .md URLs work (e.g. /build/ai.md)
Check homepage AskAptos link resolves to /build/ai
Spot-check ES/ZH translations render correctly

🤖 Generated with Claude Code

vercel · 2026-02-24T19:46:58Z

@tippi-fifestarr is attempting to deploy a commit to the AptosLabs Team on Vercel.

A member of the Team first needs to authorize it.

…nd .md URLs Replace the default starlight-llms-txt plugin index (which only linked to full/small blob files) with a structured index that lists every documentation page grouped by section and auto-derived sub-sections. Each entry includes the page title, description, and a direct .md URL for per-page AI ingestion. Also adds AskAptos chatbot section to AI overview, documents the per-page .md URL feature and Claude Code usage on the llms-txt page, fixes the homepage AskAptos link, and translates all changes to ES and ZH. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This pull request enhances the AI documentation tooling by replacing the default starlight-llms-txt plugin's index with a structured, hierarchical index that organizes documentation pages by section. The structured index provides page titles, descriptions, and per-page .md URLs to help AI agents efficiently select relevant documentation without ingesting all content.

Changes:

Added structured llms.txt index with hierarchical organization (## for sections, ### for sub-sections) and per-page .md URLs
Created custom integration (llms-txt-index.ts) to override the plugin's default /llms.txt route via astro:route:setup hook
Updated AI tools documentation pages to include AskAptos chatbot section and Claude Code usage guide (previously missing)
Fixed homepage AskAptos chatbot link from placeholder (#) to /build/ai
Applied all English documentation changes to Spanish and Chinese translations

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`src/pages/llms-index.ts`	New API route that generates structured llms.txt index with sections, sub-sections, and per-page `.md` links
`src/integrations/llms-txt-index.ts`	Astro integration that overrides plugin's `/llms.txt` route via `astro:route:setup` hook
`astro.config.mjs`	Imports new integration and adds plugin configuration (description, optional links)
`src/content/docs/llms-txt.mdx`	Adds "Per-Page Markdown Access" and "Claude Code" usage sections
`src/content/docs/build/ai.mdx`	Adds AskAptos chatbot section, updates llms.txt feed table descriptions
`src/content/docs/index.mdx`	Fixes AskAptos chatbot link from `#` to `/build/ai`
`src/content/docs/es/*.mdx`	Spanish translations of all English documentation changes
`src/content/docs/zh/*.mdx`	Chinese translations of all English documentation changes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/pages/llms-index.ts

Copilot · 2026-02-24T20:02:00Z

src/integrations/llms-txt-index.ts

+        if (
+          route.component.includes("starlight-llms-txt") &&
+          route.component.endsWith("/llms.txt.ts")
+        ) {


The route component override uses a type cast that bypasses TypeScript's type system. While this may be necessary given Astro's integration API, consider adding a comment explaining why the cast is needed and ensuring the route.component property is indeed writable at runtime. Additionally, verify that the component path is correct and relative to the project root.

Suggested change

) {

) {

// NOTE: Astro's RouteOptions type treats `component` as effectively read-only,

// but the `astro:route:setup` integration hook is explicitly designed to allow

// reassigning `route.component` at runtime in order to swap out the route handler.

// We assert a mutable `component` property here to satisfy TypeScript while relying

// on Astro's documented behavior that this mutation is supported.

// The replacement path is intended to be relative to the project root and must

// point to a valid page/component file that will handle the `/llms.txt` route.

Copilot · 2026-02-24T20:02:01Z

src/pages/llms-index.ts

+export const GET: APIRoute = async () => {
+  // eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-unsafe-call
+  const docs = await getCollection("docs");
+
+  // Filter to English-only, exclude 404, locale roots, and excluded pages
+  const englishDocs = (docs as Doc[])
+    .filter(
+      (doc) =>
+        !doc.id.startsWith("es/") &&
+        !doc.id.startsWith("zh/") &&
+        doc.id !== "es" &&
+        doc.id !== "zh" &&
+        !doc.id.includes("404") &&
+        !EXCLUDE_PAGES.has(doc.id),
+    )
+    .sort((a, b) => a.id.localeCompare(b.id));
+
+  // Group by section
+  const sections = new Map<string, Doc[]>();
+  const topLevel: Doc[] = [];
+
+  for (const doc of englishDocs) {
+    const sectionKey = getSectionKey(doc.id);
+    if (sectionKey === "other") {
+      topLevel.push(doc);
+    } else {
+      if (!sections.has(sectionKey)) sections.set(sectionKey, []);
+      const sectionList = sections.get(sectionKey);
+      if (sectionList) sectionList.push(doc);
+    }
+  }
+
+  // Build a lookup of slug -> doc for sub-section title resolution
+  const docsBySlug = new Map<string, Doc>();
+  for (const doc of englishDocs) {
+    docsBySlug.set(doc.id, doc);
+  }
+
+  const lines: string[] = [
+    "# Aptos Developer Documentation",
+    "",
+    "> Developer documentation for the Aptos blockchain — Move smart contracts, SDKs, APIs, indexer, node operations, and AI tools.",
+    "",
+    "This file lists all documentation pages with descriptions. Each page is also available as raw Markdown by appending `.md` to its URL (e.g. `https://aptos.dev/build/guides/first-transaction.md`).",
+    "",
+    "## Full Documentation",
+    "",
+    `- [Complete documentation](${SITE_URL}/llms-full.txt): All pages concatenated into a single file`,
+    `- [Abridged documentation](${SITE_URL}/llms-small.txt): Condensed version for smaller context windows`,
+    "",
+  ];
+
+  // Top-level pages
+  for (const doc of topLevel) {
+    lines.push(formatEntry(doc));
+  }
+  if (topLevel.length > 0) lines.push("");
+
+  // Grouped sections with auto-derived sub-sections
+  for (const [sectionKey, sectionLabel] of Object.entries(SECTION_CONFIG)) {
+    const sectionDocs = sections.get(sectionKey);
+    if (!sectionDocs || sectionDocs.length === 0) continue;
+
+    lines.push(`## ${sectionLabel}`);
+    lines.push("");
+
+    // Split into direct children and sub-sectioned pages
+    const directChildren: Doc[] = [];
+    const subSections = new Map<string, Doc[]>();
+    const subSectionOrder: string[] = [];
+
+    for (const doc of sectionDocs) {
+      const subKey = getSubSectionKey(doc.id, sectionKey);
+      if (subKey) {
+        if (!subSections.has(subKey)) {
+          subSections.set(subKey, []);
+          subSectionOrder.push(subKey);
+        }
+        const subList = subSections.get(subKey);
+        if (subList) subList.push(doc);
+      } else {
+        directChildren.push(doc);
+      }
+    }
+
+    // Print direct children first
+    for (const doc of directChildren) {
+      lines.push(formatEntry(doc));
+    }
+
+    // Print sub-sections
+    for (const subKey of subSectionOrder) {
+      const subDocs = subSections.get(subKey);
+      if (!subDocs || subDocs.length === 0) continue;
+
+      // Use the index page's title as the sub-section header, or derive from path
+      const indexDoc = docsBySlug.get(subKey);
+      const fallback = subKey.split("/").pop() ?? subKey;
+      const subLabel = indexDoc ? indexDoc.data.title : fallback;
+
+      lines.push("");
+      lines.push(`### ${subLabel}`);
+      lines.push("");
+
+      for (const doc of subDocs) {
+        lines.push(formatEntry(doc));
+      }
+    }
+
+    lines.push("");
+  }
+
+  return new Response(lines.join("\n"), {
+    status: 200,
+    headers: {
+      "Content-Type": "text/plain; charset=utf-8",
+      "Cache-Control": "public, max-age=3600",
+    },
+  });
+};


The GET handler lacks error handling. If getCollection fails or if there are issues during content processing (e.g., malformed doc data), the route will return a generic 500 error without useful diagnostics. Wrap the main logic in a try-catch block and return a meaningful error response if something goes wrong.

This endpoint is prerender = true so it runs at build time only. If getCollection fails the build itself fails, which is the behavior we want. Adding try-catch would mask build errors rather than surface them.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Tippi Fifestarr <62179036+tippi-fifestarr@users.noreply.github.com>

gregnazario force-pushed the ai/structured-llms-txt branch from 192e7fe to 8145d76 Compare February 24, 2026 19:53

gregnazario requested review from Copilot and gregnazario and removed request for Copilot February 24, 2026 19:54

Copilot started reviewing on behalf of gregnazario February 24, 2026 19:56 View session

Copilot AI reviewed Feb 24, 2026

View reviewed changes

Update src/pages/llms-index.ts

f3c643f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Tippi Fifestarr <62179036+tippi-fifestarr@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: structured llms.txt index + AI docs improvements#379

docs: structured llms.txt index + AI docs improvements#379
tippi-fifestarr wants to merge 2 commits intoaptos-labs:mainfrom
tippi-fifestarr:ai/structured-llms-txt

tippi-fifestarr commented Feb 24, 2026

Uh oh!

vercel bot commented Feb 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

Copilot AI Feb 24, 2026

Uh oh!

tippi-fifestarr Feb 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-        ) {
+        ) {
+          // NOTE: Astro's RouteOptions type treats `component` as effectively read-only,
+          // but the `astro:route:setup` integration hook is explicitly designed to allow
+          // reassigning `route.component` at runtime in order to swap out the route handler.
+          // We assert a mutable `component` property here to satisfy TypeScript while relying
+          // on Astro's documented behavior that this mutation is supported.
+          // The replacement path is intended to be relative to the project root and must
+          // point to a valid page/component file that will handle the `/llms.txt` route.

Conversation

tippi-fifestarr commented Feb 24, 2026

Summary

Test plan

Uh oh!

vercel bot commented Feb 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

tippi-fifestarr Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants