Skip to content

Refactor: remove time-based criteria from usability-manager that cannot be tested #384

@FL4TLiN3

Description

@FL4TLiN3

Description

The usability-manager instruction includes time-based criteria that cannot be verified in automated testing.

Current State

## Usability Properties

- **Demo works zero-config**: Demo expert succeeds without any setup
- **Setup efficiency**: Setup completes in under 2 minutes (if applicable)
- **Error guidance**: All errors include "To fix:" steps
- **Doctor diagnostics**: Doctor correctly identifies issues (if applicable)
- **Fresh user success**: New users succeed within 5 minutes

Problematic criteria:

  1. "Setup completes in under 2 minutes" - Clock time is variable and environment-dependent
  2. "New users succeed within 5 minutes" - Cannot simulate "new users" or measure their time

Target State

Replace time-based criteria with step/interaction counts:

## Usability Properties

- **Demo works zero-config**: Demo expert succeeds without any setup or API keys
- **Setup efficiency**: Setup completes in ≤3 interactions (check env, prompt for missing, verify)
- **Error guidance**: All error messages include "To fix:" with actionable steps
- **Doctor diagnostics**: Doctor correctly identifies intentionally broken configuration
- **First-run success**: Default configuration works without customization

Or remove time-based criteria entirely if they can't be meaningfully tested:

## Usability Properties

- **Demo works zero-config**: Demo expert succeeds without any setup
- **Error guidance**: All errors include "To fix:" steps
- **Doctor diagnostics**: Doctor correctly identifies issues (if applicable)
- **First-run clarity**: Instructions are clear enough for copy-paste execution

Affected Areas

  • apps/create-expert/src/lib/create-expert-toml.ts (USABILITY_MANAGER_INSTRUCTION)

Acceptance Criteria

  • No time-based criteria that can't be tested
  • All criteria are verifiable in automated testing
  • Usability still meaningfully assessed

Metadata

Metadata

Assignees

No one assigned

    Labels

    create-expertcreate-expert CLI packagerefactorCode improvement without behavior change

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions