📚 Documentation Noob Test Report - 2026-03-23

[github-actions[bot]]

Summary

• Date of test: 2026-03-23
• Pages visited: Homepage, Quick Start, CLI Commands, Creating Workflows, About Workflows, How They Work, FAQ, Engines Reference, Auth Reference, Examples (Issue/PR Events, Scheduled, Comment-Triggered, Manual), Troubleshooting (Common Issues), Glossary, Frontmatter Reference, GitHub Actions Primer
• Overall impression: The documentation is comprehensive and well-structured for someone with a GitHub/dev background, but contains a few typos and minor confusions that could trip up a true beginner on first read.

---

Critical Issues Found

🔴 No truly blocking issues were found — all key pages return HTTP 200 and navigation is intact.

---

Confusing Areas

🟡 Missing verb in overview.mdx (Grammar bug)

File: docs/src/content/docs/introduction/overview.mdx, line 44

Current text:

The `gh aw compile` command this markdown file into a hardened [GitHub Actions Workflow]...
```

**Issue**: The verb is missing. It should read "**compiles** this markdown file" or "**converts** this markdown file". As a newcomer, this sentence reads as broken grammar and undermines trust.

**Suggested fix**:
```
The `gh aw compile` command **compiles** this markdown file into a hardened GitHub Actions Workflow...
```

---

#### 🟡 Typo: "blacklog" instead of "backlog" in Quick Start

**File**: `docs/src/content/docs/setup/quick-start.mdx`, line ~101

**Current text**:
```
your issue blacklog, your CI setup...
```

**Suggested fix**:
```
your issue backlog, your CI setup...

---

🟡 Confusing model default comment in engines.md

File: docs/src/content/docs/reference/engines.md, line 51

Current text:

engine:
  id: copilot
  model: gpt-5                          # defaults to claude-sonnet-4
```

**Issue**: The comment says "defaults to `claude-sonnet-4`" in a **Copilot** engine block, while the example value shown is `gpt-5`. This is confusing because:
1. Why would a Copilot engine default to a Claude model?
2. The example model (`gpt-5`) contradicts the default comment (`claude-sonnet-4`)

This likely needs to be corrected to show the actual Copilot default model.

---

#### 🟡 Code block language `aw warp` should be `aw wrap` in how-they-work.mdx

**File**: `docs/src/content/docs/introduction/how-they-work.mdx`, line 14

**Current**:

```

The word warp appears to be a typo for wrap (which enables long-line wrapping in code blocks). Every other code block in the docs uses ```yaml wrap or ```bash wrap. Only this one instance uses warp. While it renders fine (since aw is the language and warp/wrap is a display hint), it's inconsistent and could cause confusion if copied as a template.

---

🟡 Navigation "Examples" link goes to index.html#gallery — non-standard URL

Location: Top navigation bar (CustomHeader.astro line 12)

Current: The "Examples" header nav link points to /gh-aw/index.html#gallery

Issue for beginners: The .html suffix in the URL is unusual for modern static sites and the #gallery anchor is not immediately visible on the homepage. A new user clicking "Examples" in the top nav lands on the home page scrolled to a gallery section, which may be unexpected since they'd expect a dedicated examples page. The sidebar correctly shows separate example sections (Issue/PR Events, Scheduled, etc.), creating a disconnect between top-nav and sidebar navigation.

---

🟡 Step 2 in Quick Start: add-wizard command format not explained

File: docs/src/content/docs/setup/quick-start.mdx

gh aw add-wizard githubnext/agentics/daily-repo-status

Issue: A beginner doesn't know what githubnext/agentics/daily-repo-status refers to. Is it <github-org>/<repo>/<workflow-file>? Is it a registry path? There's no explanation of this format before using it. This could leave users wondering why they can't use their own repo path format.

Suggested addition: Add a brief note explaining the format: <org>/<repo>/<workflow-name> — the path to a workflow markdown file in the Agentics collection.

---

🟡 "Frontmatter" concept introduced without definition on first reference in Quick Start

File: docs/src/content/docs/setup/quick-start.mdx, Step 4

"If you have changed the frontmatter, regenerate the workflow YAML..."

Issue: This is the first mention of "frontmatter" in the Quick Start, with no inline explanation or link to the glossary. A user who just installed the tool and is customizing their first workflow doesn't know what frontmatter means. There's a tooltip/link on the word in the overview.mdx page but not here.

Suggested fix: Add a brief parenthetical or link: "frontmatter (the YAML section between --- at the top of the .md file)"

---

🟡 Gemini missing from main navigation "Available Coding Agents" section header

File: docs/src/content/docs/reference/engines.md

The page title and sidebar say "Copilot, Claude, Codex, and Gemini" but the introductory text on the How They Work page says "Workflows support GitHub Copilot (default), Claude by Anthropic, and Codex" — with no mention of Gemini. A beginner reading How They Work first won't know Gemini is supported until they find the engines reference.

---

What Worked Well

✅ Clear prerequisites list in Quick Start

The Quick Start page has a well-organized prerequisites block with links to install each dependency. A beginner immediately knows what they need before starting.

✅ Excellent estimated time indicators

Both Quick Start (⏱️ 10 min) and Creating Workflows (⏱️ 5-15 min) have time estimates at the top, which sets appropriate expectations.

✅ Multiple AI engine options clearly documented

The engines reference table is clean and beginner-friendly with a clear mapping of engine → secret required.

✅ TIP callouts for common failures

The Quick Start includes helpful [!TIP] callouts for authentication issues and first-run secrets failures. These preempt common beginner mistakes.

✅ Good redirect infrastructure

There are many historical URL redirects configured (/start-here/quick-start/ → /setup/quick-start/, etc.), meaning bookmarks and old links continue to work.

✅ Comprehensive FAQ

The FAQ page covers a wide range of questions new users will have, including the determinism concern, cross-repo access, and editing without recompiling.

✅ Video demonstrations

The Quick Start and Creating Workflows pages include embedded video tutorials — very helpful for visual learners.

✅ GitHub Actions Primer page

The primer page is an excellent addition for developers unfamiliar with GitHub Actions, reducing the assumed knowledge gap.

---

Recommendations

Quick wins

1. Fix the missing verb in overview.mdx line 44: add "compiles" to "The \gh aw compile` command this markdown file into..."`
2. Fix "blacklog" → "backlog" in quick-start.mdx
3. Fix "aw warp" → "aw wrap" in how-they-work.mdx
4. Fix the model comment in engines.md line 51: the comment # defaults to claude-sonnet-4 in a Copilot engine block is misleading
5. Add frontmatter explanation in Quick Start Step 4 (inline note or link to glossary)

Medium-term improvements

6. Explain add-wizard path format in Quick Start Step 2 — briefly explain org/repo/workflow-name convention
7. Add Gemini to How They Work page — it mentions only Copilot, Claude, Codex but not Gemini
8. Consider a dedicated Examples landing page instead of the index.html#gallery anchor approach for the top nav "Examples" link

Longer-term

9. Add a "What is frontmatter?" callout or expandable section early in the Quick Start for readers with no YAML experience
10. Add a "first run checklist" that summarizes what to verify before running a workflow (secrets set, Actions enabled, correct branch)

---

References:

• §23427001846

AI generated by Documentation Noob Tester · history

• expires on Mar 24, 2026, 8:16 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-24T08:16:36.614Z.

Closed by Workflow