Skip to content

Refactor: expert-tester uses exec instead of delegation for testing #357

@FL4TLiN3

Description

@FL4TLiN3

Description

The expert-tester uses shell execution via exec to run experts for testing rather than using the delegation pattern. This bypasses the delegation model that other experts in the system follow.

Current Behavior

EXPERT_TESTER_INSTRUCTION (lines 482-485) instructs:

### 1. Execute Tests
For each test case, run:
\`\`\`bash
npx -y perstack run expert-name "test query" --workspace . --filter completeRun
\`\`\`

Design Inconsistency

The create-expert system uses delegation extensively:

  • create-expert → delegates to property-extractor, ecosystem-builder, etc.
  • integration-manager → delegates to functional-manager, usability-manager
  • functional-manager → delegates to expert-tester

But expert-tester breaks this pattern by using shell execution.

Considerations

Why exec might be intentional:

  1. Testing requires running the expert as an independent process
  2. The tested expert may not be part of the same delegation hierarchy
  3. Testing requires observing external behavior (stdout/stderr)

Why delegation would be better:

  1. Consistency with the rest of the system
  2. Better error handling (delegation failure handling is built-in)
  3. Avoids shell escape issues

Recommendation

Document the design decision explicitly. If exec is intentional, explain why in the instruction. If delegation is preferred, the tested experts should be added as delegates.

Affected Areas

  • apps/create-expert/src/lib/create-expert-toml.ts:466-527 (EXPERT_TESTER_INSTRUCTION)

Acceptance Criteria

  • Either add a comment explaining why exec is used instead of delegation
  • OR refactor to use delegation if appropriate

Metadata

Metadata

Assignees

No one assigned

    Labels

    create-expertcreate-expert CLI packagerefactorCode improvement without behavior change

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions