You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Make the review pipeline config-driven with ordered built-in passes and Review Artifact v3.\n\nAlso harden acceptance execution with nested result resume handling, per-criterion timeout support, generic pass expectations, refreshed docs, and expanded smoke coverage.
Review rounds accumulate — each `trellis review` appends a numbered section with a fixed metadata block. Prior rounds provide context for subsequent reviewers and make review provenance visible.
118
+
Review rounds accumulate — each `trellis review` appends a numbered Review Artifact v3 section with per-pass `pass_results`. The default five-layer pipeline is `spec_compliance`, `scope_drift`, `regression_hunt`, `convention_check`, and `dark_patterns`, ordered by explicit `order` fields in `.ai/config.yaml`. Prior rounds provide context for subsequent reviewers and make review provenance visible.
Copy file name to clipboardExpand all lines: .ai/README.md
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,11 +11,13 @@ Trellis is a spec-driven framework for AI agent task planning and execution. Eve
11
11
1.**Plan:** AI generates a task spec in `.ai/specs/drafts/` via conversational ReAct loop
12
12
2.**Approve:** Developer reviews and moves spec to `.ai/specs/approved/`
13
13
3.**Execute:** AI picks up the approved spec, executes phases, validates at each checkpoint
14
-
4.**Review:** Adversarial review finds what execution missed — `trellis review` runs automated passes, scaffolds a machine-validated review artifact, and records review provenance in the latest round
14
+
4.**Review:** Adversarial review finds what execution missed — `trellis review` runs the configured `spec_compliance` and `scope_drift` checks, scaffolds Review Artifact v3, and prepares the adversarial `regression_hunt`, `convention_check`, and `dark_patterns` passes in the latest round
15
15
5.**Archive:** Completed specs move to `.ai/specs/archive/YYYY-MM/` with truthful review results recorded, or a human-reviewed override audited explicitly when the gate is blocked
16
16
17
17
The approval gate is the human oversight boundary. The review gate is the quality boundary. During execution, the agent operates autonomously through all phases, pausing only when blocked or deviating from the spec. A normal completion path still stays agent-driven; the human-reviewed override is an exceptional audited escape hatch, not the default workflow.
18
18
19
+
The default review topology lives in `config.yaml` and uses five ordered built-in passes: `spec_compliance`, `scope_drift`, `regression_hunt`, `convention_check`, and `dark_patterns`. Review Artifact v3 stores per-pass `pass_results`, reviewer provenance, and round status for that configured topology.
Copy file name to clipboardExpand all lines: .ai/prompts/exec.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -167,6 +167,8 @@ After all phases complete and before `trellis complete`:
167
167
5. Fix any blocking findings if needed
168
168
6. Run `trellis complete <task-id>` — reads the review, records verdict, archives
169
169
170
+
The default Review Artifact v3 pipeline is `spec_compliance`, `scope_drift`, `regression_hunt`, `convention_check`, and `dark_patterns`. `trellis review` scaffolds the adversarial sections in configured order and expects the reviewer to update `round_status` plus per-pass `pass_results` before completion.
171
+
170
172
`trellis complete` will **refuse to archive** if the latest review round is missing, malformed, incomplete, or failed. The only bypass is the exceptional human path: `trellis complete <task-id> --human-reviewed --reason "<why>"`, which requires interactive confirmation and records an audited override.
Copy file name to clipboardExpand all lines: .ai/prompts/review.md
+47-16Lines changed: 47 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@
8
8
9
9
## Mission
10
10
11
-
Find what's wrong. Not what's right — what's wrong.
11
+
Find what is wrong. Not what is right.
12
12
13
13
You are reviewing changes made during spec execution. A separate agent built this, or you did in a prior session. Either way, your job is to attack it.
14
14
@@ -31,14 +31,31 @@ A review that finds zero issues is suspicious. Look harder.
31
31
2. Read the git diff of all changes
32
32
3. Read `CONVENTIONS.md` and `AGENTS.md`
33
33
4. Read `.ai/reviews/{task-id}.md` — if prior review rounds exist, read what was found before. Don't re-report fixed issues. Note if a prior finding persists.
34
-
5. Attack the diff through the three vectors below — **all three are required**
35
-
6. Write findings into the latest review section in `.ai/reviews/{task-id}.md` — each adversarial section must have content or `trellis complete` will reject. Update the review provenance metadata for the reviewer who actually performed the review.
34
+
5. Attack the diff through the configured adversarial passes — by default: `regression_hunt`, `convention_check`, and `dark_patterns`
35
+
6. Write findings into the latest review section in `.ai/reviews/{task-id}.md`
36
+
7. Update the Review Artifact v3 metadata so the latest round is truthful and complete
37
+
38
+
---
39
+
40
+
## Default Review Pipeline
41
+
42
+
The default built-in five-pass pipeline in `.ai/config.yaml` is:
43
+
44
+
-`spec_compliance`
45
+
-`scope_drift`
46
+
-`regression_hunt`
47
+
-`convention_check`
48
+
-`dark_patterns`
49
+
50
+
`trellis review` already runs `spec_compliance` and `scope_drift` and scaffolds the adversarial sections in configured order. Your job is to complete the adversarial passes and finalize the metadata for Review Artifact v3.
51
+
52
+
If the project has changed pass titles in `.ai/config.yaml`, follow the headings already scaffolded by `trellis review`. The built-in pass ids stay the same even if the visible section title changes.
36
53
37
54
---
38
55
39
56
## Attack Vectors
40
57
41
-
### 1. Regression Hunt
58
+
### 1. Regression Hunt (`regression_hunt`)
42
59
43
60
For each modified file, find every caller, importer, and downstream consumer. What assumptions do they make that this change violates?
44
61
@@ -48,15 +65,15 @@ For each modified file, find every caller, importer, and downstream consumer. Wh
48
65
- Verify event listeners and subscribers still match event shapes
49
66
- Check if removed or renamed exports are still referenced elsewhere
50
67
51
-
### 2. Convention Violations
68
+
### 2. Convention Check (`convention_check`)
52
69
53
70
Read `CONVENTIONS.md` and `AGENTS.md`. For each changed file, check whether the new code violates a documented rule.
54
71
55
72
- Cite the specific convention and the specific violating line
56
73
- Don't flag style preferences — only documented, stated conventions
57
74
- Check naming patterns, layer boundaries, import rules, test patterns
58
75
59
-
### 3. Defect Scan
76
+
### 3. Dark Patterns (`dark_patterns`)
60
77
61
78
For each change, actively hunt for:
62
79
@@ -81,30 +98,36 @@ For each change, actively hunt for:
81
98
82
99
## Output
83
100
84
-
`trellis review` scaffolds the review file at `.ai/reviews/{task-id}.md` with numbered review sections. Fill in the latest section using the fixed Review Artifact v2 contract:
101
+
`trellis review` scaffolds the review file at `.ai/reviews/{task-id}.md` with numbered review sections. Fill in the latest section using the Review Artifact v3 contract:
85
102
86
103
````markdown
87
104
## Review N — {timestamp}
88
105
89
106
### Metadata
90
107
```json
91
108
{
92
-
"schema_version": 2,
109
+
"schema_version": 3,
93
110
"round_status": "completed",
94
111
"reviewer_mode": "fresh_agent",
95
112
"reviewer_session": "session-id-or-empty-string",
96
113
"reviewed_at": "{timestamp}",
97
114
"override_reason": null,
98
-
"automated_passes": {
115
+
"pass_results": {
99
116
"spec_compliance": "pass",
100
-
"scope_drift": "pass"
117
+
"scope_drift": "pass",
118
+
"regression_hunt": "pass",
119
+
"convention_check": "pass",
120
+
"dark_patterns": "pass"
101
121
}
102
122
}
103
123
```
104
124
105
-
### Automated Passes
125
+
### Pass Results
106
126
- spec_compliance: PASS
107
127
- scope_drift: PASS
128
+
- regression_hunt: PASS
129
+
- convention_check: PASS
130
+
- dark_patterns: PASS
108
131
109
132
### Regression Hunt
110
133
{For each modified file, trace callers/importers. What assumptions break?
@@ -114,9 +137,9 @@ List findings or "No issues found — checked [what you checked]".}
114
137
{Read CONVENTIONS.md and AGENTS.md. Does new code violate any documented rule?
115
138
List findings or "No issues found — checked [what you checked]".}
116
139
117
-
### Defect Scan
118
-
{Hunt for hardcoded values, off-by-one, missing null checks, race conditions,
copy-paste errors, unhandled error paths, and security issues.
120
143
List findings or "No issues found — checked [what you checked]".}
121
144
122
145
### Blocking
@@ -129,9 +152,17 @@ List findings or "No issues found — checked [what you checked]".}
129
152
{pass | fail | pass_with_issues}
130
153
````
131
154
132
-
Set `reviewer_mode` to `fresh_agent`, `auto`, or `executor` to match the real reviewer. Leave `override_reason` as `null` for normal reviews. Prior review rounds remain in the file as context. Don't modify them — only fill in the latest section.
155
+
Update these metadata fields explicitly:
156
+
157
+
- Set `round_status` to `completed` when the review is actually done
158
+
- Set `reviewer_mode` to `fresh_agent`, `auto`, or `executor` to match the real reviewer
159
+
- Set `reviewer_session` to the real session identifier or `""`
160
+
- Keep the automated pass results for `spec_compliance` and `scope_drift`
161
+
- Set adversarial `pass_results` for `regression_hunt`, `convention_check`, and `dark_patterns` to `pass`, `pass_with_issues`, or `fail`
162
+
163
+
Prior review rounds remain in the file as context. Do not rewrite them.
133
164
134
-
**All three adversarial sections (Regression Hunt, Convention Check, Defect Scan) must contain content.** Each must have at least one finding or an explicit "No issues found" with a brief note of what was checked. `trellis complete` will reject reviews with empty adversarial sections.
165
+
**All configured adversarial sections must contain content.** Each must have at least one finding or an explicit "No issues found" with a brief note of what was checked. `trellis complete` will reject reviews with empty configured sections or with `round_status` left at `in_progress`.
135
166
136
167
**Verdict rules:** Any blocking finding → `fail`. Non-blocking only → `pass_with_issues`. Clean → `pass`.
Copy file name to clipboardExpand all lines: .ai/schemas/spec.json
+37-4Lines changed: 37 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -223,7 +223,12 @@
223
223
"description": {"type": "string"},
224
224
"command": {"type": "string"},
225
225
"expected": {"type": "string"},
226
-
"cwd": {"type": "string", "description": "Working directory relative to workspace root"}
226
+
"cwd": {"type": "string", "description": "Working directory relative to workspace root"},
227
+
"timeout_seconds": {
228
+
"type": "integer",
229
+
"minimum": 1,
230
+
"description": "Command timeout in seconds. Defaults to 600 when omitted."
231
+
}
227
232
}
228
233
}
229
234
}
@@ -327,10 +332,38 @@
327
332
"type": "string",
328
333
"description": "Working directory for the command, relative to workspace root. Useful in monorepo/workspace setups where different criteria target different submodules."
329
334
},
335
+
"timeout_seconds": {
336
+
"type": "integer",
337
+
"minimum": 1,
338
+
"description": "Command timeout in seconds. Defaults to 600 when omitted."
339
+
},
330
340
"result": {
331
-
"type": "string",
332
-
"enum": ["pass", "fail"],
333
-
"description": "Result recorded by trellis exec"
341
+
"oneOf": [
342
+
{
343
+
"type": "string",
344
+
"enum": ["pass", "fail"],
345
+
"description": "Flat result recorded by trellis exec"
346
+
},
347
+
{
348
+
"type": "object",
349
+
"required": ["status"],
350
+
"properties": {
351
+
"status": {
352
+
"type": "string",
353
+
"enum": ["pass", "fail"]
354
+
},
355
+
"timestamp": {
356
+
"type": "string",
357
+
"format": "date-time"
358
+
},
359
+
"output": {
360
+
"type": "string"
361
+
}
362
+
},
363
+
"additionalProperties": false,
364
+
"description": "Nested result block supported for execution records"
0 commit comments