📚 Documentation Noob Test Report - 2026-04-02 #24114
Replies: 1 comment 1 reply
-
|
/plan single issue |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
🚀 Plan Command has started processing this discussion comment |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
/gh-aw/), Quick Start (/gh-aw/setup/quick-start/), CLI Commands (/gh-aw/setup/cli/)🔴 Critical Issues
1. Sitemap returns 404
/gh-aw/sitemap-index.xmlis linked in the<head>of every page but returns a 404. This breaks SEO crawlers and any user who tries to find an index of the docs.2. Broken canonical URLs in page metadata
Every page has
<link rel="canonical">pointing tohttps://github.github.com/gh-aw/...— butgithub.github.comis not the real production domain (it appears to be the GitHub Pages subdomain). If the production URL is different, these canonical links are wrong and could confuse both users and search engines.3. Search is disabled in dev mode
The site shows: "Search is only available in production builds." A new user who tries
Ctrl+Kto find something gets no results. This is expected in dev, but worth noting — any CI-based docs preview links shared with users would also be broken.🟡 Confusing Areas
1. "AI Account" prerequisite is vague
The Quick Start lists "AI Account - GitHub Copilot, Anthropic Claude or OpenAI Codex" as a prerequisite, but doesn't explain:
COPILOT_GITHUB_TOKEN? Is it the same as a regular GitHub PAT? How is it different fromGITHUB_TOKEN?COPILOT_GITHUB_TOKEN,ANTHROPIC_API_KEYorOPENAI_API_KEY" but doesn't tell you where to get those or how to add them to GitHub Actions.2. "What is frontmatter?" — undefined jargon
The Quick Start Step 4 says "If you have changed the frontmatter, regenerate the workflow YAML…" but never explains what frontmatter is. For a complete beginner, this term is opaque. A tooltip, link to the reference, or one-sentence explanation would help significantly.
3. "Peli's Agent Factory" nav item is confusing
The main navigation bar includes "Peli's Agent Factory" as a top-level item. A new user has no idea what this is. It appears to be an example/blog section, but having a person's name in the main nav of a developer tool is unexpected and can make the product feel less professional or harder to understand structurally.
4. "Examples" nav links to
#galleryanchor on homepageThe top nav "Examples" link goes to
/#gallery(an anchor on the homepage), not a dedicated examples page. However, there is also/gh-aw/examples/multi-repo/. This inconsistency could frustrate users expecting a dedicated examples index page.5. Step 2 in Quick Start — command output not shown
The guide says this starts an "interactive process" but doesn't show what that looks like. No screenshot, no sample output. A beginner won't know if the process went correctly.
6. The early-access warning buries the lede
There's an info box: "Note: GitHub Agentic Workflows is in early development and may change significantly. Use it with caution, and at your own risk." This is shown after the marketing text, but a security-conscious developer needs to see this before deciding to install. It should appear higher on the page or be more visually prominent.
7. CLI page has too much GHES content up-front
The
/gh-aw/setup/cli/page starts cleanly with a commands table, but then goes deep into GitHub Enterprise Server configuration very early. A beginner using GitHub.com doesn't need to read about GHES, and this makes the page feel overwhelming.🟢 What Worked Well
1. Quick Start is concise and well-structured
The 4-step structure (Install → Add workflow → Wait → Customize) is clear and achievable. Estimated time ("10 minutes") sets expectations well.
2. Prerequisites list is specific
Listing specific versions (
ghv2.0.0+) and links to install them is genuinely helpful.3. Security "Guardrails Built-In" section on the homepage is excellent
The five-layer security explanation (read-only tokens, no secrets in agent, network firewall, safe outputs, human review) is clear and builds confidence. This is the best content on the site.
4. CLI commands table is clean
The commands table in
/gh-aw/setup/cli/with descriptions is easy to scan and well-organized.5. "What's next?" section at the bottom of Quick Start
Good progressive disclosure — pointing to Creating Workflows, Architecture, and FAQ as next steps.
Recommendations
Quick wins (high impact, low effort)
COPILOT_GITHUB_TOKEN— link directly to the auth reference page (/gh-aw/reference/auth/)Longer-term improvements
add-wizardinteractive flow so users know what to expect#galleryReferences: §23902187144
Beta Was this translation helpful? Give feedback.
All reactions