UX Analysis Report – 2026-03-22

[github-actions[bot]]

Today's targeted analysis focused on:

• 2 documentation files
• 1 CLI command
• 2 workflow message configurations
• 1 validation file

Overall Quality: Professional, with two actionable gaps worth addressing.

Key Finding: The health command's --verbose flag is fully implemented in logic but never registered on the Cobra command — making it inaccessible to users.

---

Quality Highlights ✅

Example 1: Quick-Start Guide

• File: docs/src/content/docs/setup/quick-start.mdx
• What works well: Clear prerequisites checklist, estimated time, embedded video, numbered steps with interactive wizard guidance, and helpful callouts (TIP, NOTE).
• Reference: "⏱️ Estimated time: 10 minutes … Step 2 - Add the sample workflow and trigger a run"

Example 2: Cache Validation Error Messages

• File: pkg/workflow/cache_validation.go
• What works well: Error messages follow the ideal enterprise pattern — problem statement + cause + inline YAML code example showing the fix.
• Reference: "duplicate cache-memory ID found - each cache must have a unique ID … Change the cache ID to a unique value. Example: ..."

---

Improvement Opportunities 💡

High Priority

Opportunity 1: Missing --verbose Flag Registration — pkg/cli/health_command.go

• File: pkg/cli/health_command.go
• Current State: HealthConfig.Verbose is defined (line 26), cmd.Flags().GetBool("verbose") is called in RunE (line 60), and verbose logic is used in RunHealth (lines 131, 136). However, cmd.Flags().Bool("verbose", ...) is never called — the flag is not registered on the command.
• Issue: Users cannot pass --verbose — Cobra will return unknown flag: --verbose. The feature is silently broken.
• User Impact: Enterprise users troubleshooting workflow health have no way to enable verbose diagnostic output, reducing their ability to debug issues independently.
• Suggested Change: Add cmd.Flags().Bool("verbose", false, "Show verbose diagnostic output including raw API responses") to the flag registration block in NewHealthCommand().
• Design Principle: Trust and Reliability — Predictable behavior; if a config field exists and logic uses it, the flag must be exposed.

High Priority

Opportunity 2: Ambiguous Model Example in Engine Configuration — docs/src/content/docs/reference/engines.md

• File: docs/src/content/docs/reference/engines.md
• Current State (line 31):

  engine:
    id: copilot
    version: latest                       # defaults to latest
    model: gpt-5                          # defaults to claude-sonnet-4

• Issue: The comment # defaults to claude-sonnet-4 is placed next to model: gpt-5. An enterprise user configuring Copilot reads: "the default is claude-sonnet-4, but this example sets it to gpt-5" — and wonders why a Copilot engine would use a GPT-5 model name, and whether that's even valid.
• User Impact: Creates ambiguity about which model identifiers are valid per engine, potentially leading to misconfigured workflows and debugging time.
• Suggested Change: Replace the multi-field "kitchen sink" example with two separate, focused examples: one showing the default (with claude-sonnet-4 as the value), and one showing an override (with gpt-5 for the codex engine, or a more realistic copilot model name).
• Design Principle: Clarity and Precision — Examples should reflect realistic, non-misleading values.

---

Files Reviewed

Documentation

• docs/src/content/docs/setup/quick-start.mdx — Rating: ✅ Professional
• docs/src/content/docs/reference/engines.md — Rating: ⚠️ Needs Minor Work

CLI Commands

• pkg/cli/health_command.go — Rating: ❌ Needs Significant Work (broken flag)

Workflow Messages

• .github/workflows/archie.md — Rating: ✅ Professional (consistent persona, informative)
• .github/workflows/ci-doctor.md — Rating: ⚠️ Needs Minor Work (failure message vague: "Doctor needs assistance" gives no actionable direction)

Validation Code

• pkg/workflow/cache_validation.go — Rating: ✅ Professional

---

Metrics

• Files Analyzed: 6
• Quality Distribution:
  • ✅ Professional: 3
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 1

---

🎯 Actionable Tasks

Here are 2 targeted improvement tasks, each affecting a single file:

---

Task 1: Register Missing --verbose Flag — pkg/cli/health_command.go

File to Modify: pkg/cli/health_command.go

Current Experience

Running gh aw health --verbose returns unknown flag: --verbose. The verbose flag is read internally (line 60) and drives log output (lines 131–136), but cmd.Flags().Bool(...) is never called to register it. The feature is silently inaccessible.

Quality Issue

Design Principle: Trust and Reliability — users expect documented-looking config fields to correspond to actual CLI flags.

The HealthConfig struct advertises a Verbose bool field, and the verbose output code path exists and is non-trivial. However the entry point (the CLI flag) is missing, so enterprise users cannot access verbose diagnostics to troubleshoot failing workflows.

Proposed Improvement

Add the missing flag registration to NewHealthCommand() in pkg/cli/health_command.go:

Before (after line 84, the last cmd.Flags() call):

cmd.Flags().Int("days", 7, "Number of days to analyze (7, 30, or 90)")
cmd.Flags().Float64("threshold", 80.0, "Success rate threshold for warnings (percentage)")
addRepoFlag(cmd)
addJSONFlag(cmd)

After:

cmd.Flags().Int("days", 7, "Number of days to analyze (7, 30, or 90)")
cmd.Flags().Float64("threshold", 80.0, "Success rate threshold for warnings (percentage)")
cmd.Flags().Bool("verbose", false, "Show verbose diagnostic output including raw API responses")
addRepoFlag(cmd)
addJSONFlag(cmd)

Why This Matters

• User Impact: Enterprise users troubleshooting slow or failing health checks gain access to verbose diagnostic output already written and waiting
• Quality Factor: Completes an existing feature — no new behavior, just exposes what's already implemented
• Frequency: Any user running gh aw health for debugging

Success Criteria

• Changes made to pkg/cli/health_command.go only
• gh aw health --verbose no longer returns "unknown flag" error
• Verbose output is shown when flag is passed
• Quality rating improves from ❌ to ✅

Scope Constraint

• Single file only: pkg/cli/health_command.go
• No changes to other files required

---

Task 2: Clarify Engine Model Example — docs/src/content/docs/reference/engines.md

File to Modify: docs/src/content/docs/reference/engines.md

Current Experience

The "Extended Coding Agent Configuration" code block (around line 28–35) shows all fields together including model: gpt-5 # defaults to claude-sonnet-4. This juxtaposes a GPT-5 model name (typically associated with OpenAI/Codex) with a Copilot engine (id: copilot), while the comment says the default is claude-sonnet-4 (a different model family entirely). Enterprise users are left confused about which model names are valid for which engines.

Quality Issue

Design Principle: Clarity and Precision — examples must be realistic and non-misleading.

The single "kitchen sink" example tries to show all fields at once, but the model override (gpt-5 on a copilot engine) doesn't represent a realistic or recommended configuration. It creates unnecessary questions: "Can I use GPT-5 with Copilot? What models does Copilot support?"

Proposed Improvement

Split the single all-fields example into two focused examples. Replace the current block:

Before (around lines 26–35):

```yaml wrap
engine:
  id: copilot
  version: latest                       # defaults to latest
  model: gpt-5                          # defaults to claude-sonnet-4
  command: /usr/local/bin/copilot       # custom executable path
  args: ["--add-dir", "/workspace"]     # custom CLI arguments
  agent: agent-id                       # custom agent file identifier
  api-target: api.acme.ghe.com          # custom API endpoint hostname (GHEC/GHES)
```

After:

```yaml wrap
# All available extended configuration fields (Copilot engine)
engine:
  id: copilot
  version: latest           # defaults to latest
  model: claude-sonnet-4    # defaults to claude-sonnet-4
  command: /usr/local/bin/copilot  # custom executable path (skips installation)
  args: ["--add-dir", "/workspace"]
  agent: agent-id           # references .github/agents/agent-id.agent.md
  api-target: api.acme.ghe.com    # GHEC/GHES custom endpoint hostname
```

```yaml wrap
# Override the model for a Codex workflow
engine:
  id: codex
  model: gpt-5
```

Why This Matters

• User Impact: Eliminates the "which model do I use with which engine?" question that currently requires external research
• Quality Factor: Accurate examples reduce time-to-first-success for new enterprise adopters
• Frequency: This page is a primary reference for anyone configuring an engine

Success Criteria

• Changes made to docs/src/content/docs/reference/engines.md only
• Copilot engine example shows a valid Copilot model name as the default
• Model override example uses a model/engine pairing that makes sense
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: docs/src/content/docs/reference/engines.md
• No changes to other files required

---

References:

• Workflow Run §23402076859

AI generated by Delight · history

• expires on Mar 29, 2026, 11:35 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #22414.