diff --git a/.changeset/document-exec-usage.md b/.changeset/document-exec-usage.md new file mode 100644 index 00000000..c9101bdc --- /dev/null +++ b/.changeset/document-exec-usage.md @@ -0,0 +1,9 @@ +--- +"create-expert": patch +--- + +Document why exec is used in expert-tester + +Added explanation that exec is used instead of delegation because: +- Tests need to run Experts as black-box (same as end-users) +- CLI execution ensures realistic test conditions diff --git a/apps/create-expert/src/lib/create-expert-toml.ts b/apps/create-expert/src/lib/create-expert-toml.ts index db4e42f9..a5fb3190 100644 --- a/apps/create-expert/src/lib/create-expert-toml.ts +++ b/apps/create-expert/src/lib/create-expert-toml.ts @@ -375,6 +375,10 @@ From the stage manager: ## Testing Process ### 1. Execute Tests + +NOTE: We use \`exec\` instead of delegation because we need to test the Expert as a black-box, +exactly as end-users would run it via the CLI. This ensures realistic test conditions. + For each test case, run: \`\`\`bash npx -y perstack run expert-name "test query" --workspace . --filter completeRun