🔍 Claude Code User Documentation Review - 2026-04-01 #23897
Closed
Replies: 1 comment
-
|
This discussion has been marked as outdated by Claude Code User Documentation Review. A newer discussion is available at Discussion #24112. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Executive Summary
As a developer who uses Claude Code and actively avoids GitHub Copilot, I reviewed the gh-aw documentation to assess whether I could successfully understand and adopt this tool. The good news: Claude is a first-class citizen in gh-aw, with dedicated documentation in
engines.md, authentication instructions, version pinning, and custom endpoint support. The interactiveadd-wizardonboarding explicitly offers Claude as an engine choice. However, there are several friction points: the default engine is Copilot, architecture diagrams name "Copilot CLI" as the agent, no actual example workflow files useengine: claude, and some documentation sections inadvertently use Copilot-centric framing.Key Finding: Claude Code users can successfully use gh-aw, but will encounter confusion from Copilot-centric defaults and diagrams — particularly if they explore the codebase for examples or follow the architecture diagrams.
Persona Context
I reviewed this documentation as a developer who:
Question 1: Onboarding Experience
Can a Claude Code user understand and get started with gh-aw?
Short answer: Yes, with some friction. The prerequisites in the Quick Start explicitly list
Anthropic Claudeas an AI account option alongside Copilot. Thegh aw add-wizardcommand guides through engine selection interactively, which is excellent for first-time users. However, the README itself gives no hint about engine choices — it simply links to the Quick Start without any engine context.Specific Issues Found:
README.md— No engine overview in the summary: The README describes gh-aw as "Write agentic workflows in natural language markdown" without ever mentioning that you can use Claude, Copilot, or Codex. A user skimming the README before installing could assume Copilot is required, especially given the GitHub branding.how-they-work.mdx— Engines mentioned but Copilot-first: "Workflows support GitHub Copilot (default), Claude by Anthropic, and Codex." This is accurate but buries Claude behind the default. A non-Copilot user reading this might wonder if they're a second-class citizen.Quick Start Step 2 is well designed — the wizard handles engine selection, including Claude and Codex. No blocker here.
Recommended Fixes:
how-they-work.mdxto something like "multiple AI engines (GitHub Copilot, Claude, Codex, Gemini) — Copilot is the default if unspecified."Question 2: Inaccessible Features for Non-Copilot Users
What features or steps don't work without Copilot?
Features That Require Copilot:
engine.agent(custom agent files) — The.github/agents/custom agent mechanism is Copilot-only. Documented clearly inengines.mdwithengine.agent: ❌for Claude/Codex/Gemini. However, Claude Code users who hear about "custom agents" and expect a parallel mechanism will find none.max-continuations(autopilot mode — multiple consecutive runs) — Copilot-only. Claude offersmax-turnsinstead (limits per-run chat iterations), but there is no Claude equivalent of running the workflow autonomously multiple times in sequence.engine: copilotimplicit default — Workflows without an explicitengine:field silently default to Copilot. If a Claude user copies a community workflow that omitsengine:, it will attempt to use Copilot rather than Claude.Features That Work Without Copilot (engine-agnostic):
tools:—edit,github,bash,web-fetch,playwright,cache-memory,repo-memory,qmdcreate_issue,add_comment,create_pull_request)mcp-servers:)engine.api-targetandANTHROPIC_BASE_URLcustom endpoint routingengine.id: claude, version: "2.1.70")max-turns(Claude-specific feature, documented)Missing Documentation:
copilot-custom-agents.mdreference page. Claude has no equivalent guide beyondengines.md.max-turnsdoes differently vsmax-continuationsand how a Claude user should architect long-running workflows differently.Question 3: Documentation Gaps and Assumptions
Where does the documentation assume Copilot usage?
Copilot-Centric Language Found In:
File:
docs/src/content/docs/introduction/architecture.mdx— The Agent Workflow Firewall (AWF) diagram has a node labeledCOPILOT["Copilot CLI"]as the example agent, alongside "WebFetch Tool" and "WebSearch Tool". This visually implies the system is built around Copilot. A Claude user could incorrectly conclude AWF only works with Copilot.File:
docs/src/content/docs/introduction/architecture.mdx— The MCP Gateway diagram showsAGENT["Agent container\nCopilot CLI + MCP client\n172.30.0.20"]. Again, hardcodes Copilot as the agent in a diagram intended to explain generic architecture.File:
docs/src/content/docs/reference/engines.md— The Extended Configuration example block usesengine: copilot/id: copilotas the primary example. While this is just a YAML example, all 6 of theargs,command,env, andapi-targetexamples default to showing the Copilot engine, with Claude/Codex variants only appearing in dedicated subsections..github/workflows/ci.yml— The project's own CI workflow usesengine: copilot, meaning the repo "eats its own dogfood" but only with Copilot.Missing Alternative Instructions:
engine: copilot→engine: claude, swapCOPILOT_GITHUB_TOKEN→ANTHROPIC_API_KEY, notemax-continuationsis not available).Severity-Categorized Findings
🚫 Critical Blockers (Score: 1/10)
No genuine critical blockers found
Claude users can get started without Copilot. The
add-wizardcommand explicitly asks for engine choice. Authentication for Claude (ANTHROPIC_API_KEY) is documented inengines.md,tokens.md,faq.md, andcost-management.md. There are no steps in the Quick Start that fail without Copilot.Score rationale: 1/10 means "essentially no blockers" — the documentation gives Claude users a viable path.
Obstacle 1: Architecture Diagrams Hard-Code "Copilot CLI"
Impact: Confusion about whether AWF and MCP Gateway work with Claude
Current State:
architecture.mdxcontains Mermaid diagrams where the agent node is literally labeledCOPILOT["Copilot CLI"]. One diagram showsAGENT["Agent container\nCopilot CLI + MCP client\n172.30.0.20"].Why It's Problematic: A Claude Code user reading the security architecture section — which is important for understanding the security model — will see "Copilot CLI" as the agent and may conclude that the security architecture only applies to Copilot. This is not true; Claude runs in the same AWF container.
Suggested Fix: Change diagram nodes to use engine-neutral labels:
AGENT["AI Agent\n(Claude/Copilot/Codex)\n172.30.0.20"]andENGINE["AI Engine\n(Copilot, Claude, Codex)"].Affected Files:
docs/src/content/docs/introduction/architecture.mdxObstacle 2: Zero Example Workflows Using Claude Engine
Impact: No working examples for Claude users to copy-paste or learn from
Current State: Searching
.github/workflows/forengine: claudereturns zero results. The CI usesengine: copilot. There are no.mdor.ymlworkflow files in the repo demonstrating a Claude-engine workflow.Why It's Problematic: When developers learn by example, they look at the repository's own workflows. Finding zero Claude examples — while the project's own CI uses Copilot — sends a subtle signal that Copilot is the "real" engine and Claude is an afterthought.
Suggested Fix: Add at least one example workflow in the docs or
.github/workflows/usingengine: claudeto demonstrate the pattern. The documentationexamples/directory could include adaily-repo-status-claude.mdvariant.Affected Files:
.github/workflows/(no Claude examples exist)Obstacle 3: Copilot Default Without Prominent Warning
Impact: Silent engine mismatch when copying community workflows
Current State:
engines.mdstates "Copilot CLI is the default —engine:can be omitted when using Copilot." The feature comparison table shows Copilot as the first column with most features marked supported.Why It's Problematic: If a Claude user copies a community workflow that omits the
engine:field, it will silently attempt to use Copilot and fail (missingCOPILOT_GITHUB_TOKEN). The error message they'll get won't intuitively guide them to addengine: claude.Suggested Fix: Add a callout in⚠️ If you don't use Copilot, always specify
engines.mdand in the frontmatter reference: "engine:in your workflow frontmatter — omitting it defaults to Copilot CLI."Affected Files:
docs/src/content/docs/reference/engines.md,docs/src/content/docs/reference/frontmatter.mdObstacle 4: No Claude-Specific Getting-Started Guide
Impact: Asymmetric documentation depth — Copilot has a dedicated custom agents page, Claude does not
Current State:
copilot-custom-agents.mdis a dedicated reference page for the Copilotengine.agentfeature. Claude has no equivalent landing page. Claude-specific features (max-turns,ANTHROPIC_BASE_URL) are scattered inengines.md.Why It's Problematic: A Claude Code user who wants to understand all their options has to search across multiple reference pages. There's no "Using Claude with gh-aw" guide they can follow.
Suggested Fix: Create a
docs/src/content/docs/guides/claude.mdpage covering: settingengine: claude, obtainingANTHROPIC_API_KEY, usingmax-turns, setting a customANTHROPIC_BASE_URL, and noting the feature differences from Copilot.Affected Files:
docs/src/content/docs/guides/(file doesn't exist)💡 Minor Confusion Points (Score: 5/10)
docs/src/content/docs/reference/engines.mdmax-turnsdescription is Claude-only but not celebrated: The feature table showsmax-turns: ✅for Claude as a differentiator. This is a Claude advantage, but there's no callout highlighting it as a reason to choose Claude. File:engines.mdinstall-and-add-workflow-in-cli.mp4is probably a Copilot demo. No text description indicates which engine is being shown. File:docs/src/content/docs/setup/quick-start.mdxdocs/src/content/docs/reference/faq.mdEngine Comparison Analysis
Available Engines
Based on my review, gh-aw supports four engines with dedicated documentation:
engine: copilot— Default. Extensively documented, custom agents,max-continuations, most examples.engine: claude— Well documented inengines.md,tokens.md,faq.md.max-turnsunique advantage.engine: codex— Documented.web-searchopt-in is the key difference.engine: gemini— Listed in the comparison table. Least documentation coverage.Documentation Quality by Engine
Rating Scale: ⭐⭐⭐⭐⭐ (Excellent) to ⭐ (Poor/Missing)
Tool Availability Analysis
Tools Review
Engine-Agnostic Tools (all engines):
tools.edit— File editingtools.github— GitHub API operations (toolsets, remote/local)tools.bash— Shell command executiontools.web-fetch— Web content fetchingtools.playwright— Browser automationtools.cache-memory— Persistent memory across runstools.repo-memory— Repository-specific memorytools.qmd— Vector search over documentationtools.agentic-workflows— Workflow introspectionmcp-servers:— All custom MCP server integrationsEngine-Specific Tools:
tools.web-search— Available for all engines but: Codex requires explicit opt-in; other engines use MCP serverengine.agent— Copilot-only (custom agent files in.github/agents/)max-continuations— Copilot-onlymax-turns— Claude-onlyUnclear/Undocumented:
web-searchwork identically across all enginesagentic-workflows:introspection tool behaves consistently across enginesAuthentication Requirements
Current Documentation
Quick Start guide covers authentication for:
COPILOT_GITHUB_TOKENwith fine-grained PAT requirements)engines.md,tokens.md:ANTHROPIC_API_KEY)OPENAI_API_KEY)GEMINI_API_KEY)Missing for Claude Users
ANTHROPIC_API_KEY: The docs link toengines.mdand the auth reference but don't include a direct link toconsole.anthropic.comfor key generation. Copilot setup instructions typically include a direct link to the settings page.cost-management.mdcovers Claude billing but it's not prominently linked from the Quick Start.faq.mdcorrectly states OAuth is not supported for Claude (onlyANTHROPIC_API_KEY) — but this is buried in the FAQ rather than in a prominent note in the Quick Start.Secret Names
COPILOT_GITHUB_TOKEN(documented extensively, 10+ files)ANTHROPIC_API_KEY(documented in 8 files including primary references)OPENAI_API_KEY(documented)GEMINI_API_KEY(documented)Example Workflow Analysis
Workflow Count by Engine
Quality of Examples
Copilot Examples:
The project's own CI (
ci.yml) usesengine: copilot. This is the only concrete workflow example in the repo itself, and it reinforces Copilot as the "official" engine.Claude Examples:
None exist in
.github/workflows/. Theengines.mdreference page has YAML snippets showingengine.id: claudefor version pinning andANTHROPIC_BASE_URLrouting, but these are configuration fragments, not complete runnable workflows. A developer wanting to validate that Claude works end-to-end has no reference implementation.Recommended Actions
Priority 1: Critical Documentation Fixes
docs/src/content/docs/introduction/architecture.mdxengine:default warning in frontmatter and engines docs — Warn Claude/Codex users that omittingengine:defaults to Copilot. Files:docs/src/content/docs/reference/engines.md,docs/src/content/docs/reference/frontmatter.md.github/workflows/ordocs/src/content/docs/examples/. This is the highest-leverage fix for Claude user confidence.Priority 2: Major Improvements
ANTHROPIC_API_KEY,max-turns,ANTHROPIC_BASE_URL, and a working example. Target file:docs/src/content/docs/guides/claude.mddocs/src/content/docs/reference/tokens.mdREADME.mdPriority 3: Nice-to-Have Enhancements
engines.mdshowing what to change when switching from Copilot to Claude.max-turnsas a differentiating feature in the Quick Start or overview docs, rather than only noting it in the feature comparison table footnote.Positive Findings
What Works Well
engines.mdis comprehensive and accurate — All four engines documented with feature comparison table, version pinning, custom endpoints, and env vars. This is excellent reference material.add-wizardhandles multi-engine onboarding — The interactive wizard explicitly asks which engine to use and configures the correct secret. Claude Code users who use this path will have a smooth experience.ANTHROPIC_API_KEYis documented in 8 files — Authentication for Claude is not hidden or undocumented.max-turnsis documented as Claude-specific — The feature comparison table correctly identifies Claude differentiators.ANTHROPIC_BASE_URLandapi-targetfor Claude proxies is documented, which is important for enterprise users.engine: id: claude, version: "2.1.70"example is present.Conclusion
Can Claude Code Users Successfully Adopt gh-aw?
Answer: Yes, With Minor Effort
Reasoning: The core functionality — installing gh-aw, adding a workflow, selecting Claude as the engine, setting
ANTHROPIC_API_KEY, and running the workflow — is fully documented and accessible. Theadd-wizardinteractive onboarding is the smoothest path and works well for Claude users. Theengines.mdreference page is thorough. No step in the documented process is genuinely blocked for Claude users.The friction comes from implicit Copilot-centricity: architecture diagrams that name "Copilot CLI" as the agent, zero example workflows using Claude, a CI pipeline that uses Copilot, and a README that doesn't mention multi-engine support. These create doubt and confusion for a Claude Code user exploring the repository, even though the actual feature support is solid.
Overall Assessment Score: 7/10
Breakdown:
engines.mdis solid; missing a dedicated guide)max-continuationsand custom agents are notable gaps)Next Steps
The highest-value improvement is adding one working Claude engine example workflow. This single change would validate Claude support in a way that documentation alone cannot. Following that, fixing the architecture diagrams to be engine-neutral and adding a "Using Claude" guide would complete the documentation story for Claude Code users.
Appendix: Files Reviewed
Complete List of Documentation Files Analyzed
README.mddocs/src/content/docs/setup/quick-start.mddocs/src/content/docs/introduction/how-they-work.mdxdocs/src/content/docs/introduction/architecture.mdxdocs/src/content/docs/reference/tools.mddocs/src/content/docs/setup/cli.mddocs/src/content/docs/reference/engines.md(additional).github/workflows/ci.yml(engine usage grep)tokens.md,faq.md,cost-management.md(via search results)References:
Beta Was this translation helpful? Give feedback.
All reactions