Feature
After running a focus group test, generate an improved version of the skill document that incorporates model feedback.
Current Behavior
Models report:
- Confusions
- Potential failures
- Suggested improvements
User manually updates their skill doc based on this feedback.
Proposed Behavior
After collecting model feedback, synthesize it into a rewritten skill document that:
- Fixes identified confusions - Clarify ambiguous sections that multiple models flagged
- Optimizes progressive disclosure - Restructure so agents get essential info first, details later
- Improves examples - Add/improve examples where models indicated confusion
- Better references - Add cross-references and context where models needed more info
- Clearer structure - Reorganize based on how models actually parse the document
Output
focus-group test ./SKILL.md "task" --rewrite
# Outputs:
# 1. Normal model feedback (current behavior)
# 2. NEW: ./SKILL.improved.md - rewritten skill doc
Implementation Notes
- Use a capable model (e.g., claude-3-5-sonnet) for synthesis
- Feed it: original doc + all model feedback + rewrite instructions
- May need structured output from test models to make synthesis reliable
- Consider iterative refinement (run focus group on improved doc)
Success Criteria
- Improved doc scores better on subsequent focus group runs
- Reduced "confusions" count across models
- Maintains original intent while improving clarity
Feature
After running a focus group test, generate an improved version of the skill document that incorporates model feedback.
Current Behavior
Models report:
User manually updates their skill doc based on this feedback.
Proposed Behavior
After collecting model feedback, synthesize it into a rewritten skill document that:
Output
Implementation Notes
Success Criteria