Skip to content

feat: setup steps, optional test name, changelog (v0.4.0)#38

Merged
codesoda merged 3 commits intomainfrom
feat/setup-steps
Apr 8, 2026
Merged

feat: setup steps, optional test name, changelog (v0.4.0)#38
codesoda merged 3 commits intomainfrom
feat/setup-steps

Conversation

@codesoda
Copy link
Copy Markdown
Owner

@codesoda codesoda commented Apr 8, 2026

Summary

  • Setup steps (setup = true): always run even when checkpoint-skipped, excluded from pass/fail tallies, tolerate missing RESULT markers
  • Optional test name: name field in *.test.toml now defaults to the file stem when omitted
  • CHANGELOG.md: added based on Keep a Changelog format
  • Release workflow: now extracts notes from CHANGELOG.md instead of auto-generating
  • Version bump to 0.4.0

Test plan

  • cargo test — all 189 tests pass
  • cargo build --release — compiles cleanly
  • Verify release workflow picks up changelog entry when tag is pushed

- Setup steps (`setup = true`) always run even when checkpoint-skipped
  and are excluded from pass/fail tallies
- Test file `name` defaults to the file stem when omitted
- Add CHANGELOG.md based on Keep a Changelog
- Release workflow extracts notes from CHANGELOG.md
Copy link
Copy Markdown

@cadence cadence bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cadence PR Review

Goal
Add three user-facing changes for v0.4.0: setup-step semantics in execution/reporting, default test names when name is omitted in *.test.toml, and changelog-driven release notes. The relevant session evidence shows the user explicitly asked for filename-based default names, and the PR description scopes the rest of the release work and validation.

Signal Details
Score A- strong requirement coverage, lighter workflow verification
Models claude-opus-4-6
Phases 65% implementation, 25% verification, 10% release prep

Key Decisions

  • The user chose to ship the parser fix together with setup-step behavior, release-note workflow changes, and the v0.4.0 version bump in one PR.
  • The model implemented optional test names by introducing a separate raw deserialization struct and deriving the default from the source filename during parse.
  • The user and model verified the broader behavior changes with local cargo test and cargo build --release, while leaving the tag-triggered release workflow unverified in the PR test plan.

Outcome

The model largely achieved the stated PR goal and stayed aligned with the requested behavior changes, especially on default test naming and the setup-step execution/reporting semantics. The main gap is verification discipline: the release workflow change is plausible but not exercised here, so a meaningful part of the release-focused scope still relies on manual follow-through.

Recommendations

Verification  Prompt for workflow dry-run evidence

This PR shows the model can implement workflow edits cleanly, but automation changes are harder to trust from diff inspection alone. Adding an explicit local simulation requirement would improve confidence on similar release-focused PRs.

Before

Make the release workflow use CHANGELOG.md.

Reframe

When a PR includes GitHub Actions or release automation, include a prompt requirement to simulate the changed script locally and summarize the exact expected artifact or release-note output.

If this review was useful, please react with 👍 below. Otherwise, react with 👎.

@codesoda codesoda merged commit 2ec71fa into main Apr 8, 2026
1 check passed
@codesoda codesoda deleted the feat/setup-steps branch April 8, 2026 05:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant