Skip to content

feat(gsd): add enhanced verification checks for auto-mode#3468

Open
OfficialDelta wants to merge 5 commits intogsd-build:mainfrom
OfficialDelta:feat/enhanced-verification
Open

feat(gsd): add enhanced verification checks for auto-mode#3468
OfficialDelta wants to merge 5 commits intogsd-build:mainfrom
OfficialDelta:feat/enhanced-verification

Conversation

@OfficialDelta
Copy link
Copy Markdown
Contributor

TL;DR

What: Adds 7 mechanical verification checks that run before and after each auto-mode task.
Why: Catches common agent mistakes (reading files before creating them, installing non-existent packages, broken imports) earlier in the pipeline.
How: Pre-execution checks validate the task plan; post-execution checks validate the result. Blocking failures pause auto-mode for review.

What

Pre-execution checks (before task starts)

Check What it does Blocking?
File ops review Surfaces create/edit/delete intent for manual review No (info only)
Read-before-create Fails when plan reads a file before creating it Yes
Package existence Verifies npm packages exist before install Yes
Interface contract Warns on mismatched function signatures No (warn only)

Post-execution checks (after task completes)

Check What it does Blocking?
Import resolution Verifies relative imports resolve to files No (warn only)
Export verification Confirms exported symbols are defined No (warn only)
Type consistency Validates return types match declarations No (warn only)

Why

Auto-mode can generate code that compiles but has subtle issues:

  • Reading from files that don't exist yet (plan ordering bug)
  • Installing packages that were hallucinated (npm install foo where foo 404s)
  • Importing from modules that don't exist
  • Function signatures that don't match their declarations

These checks catch issues before they cascade into harder-to-debug failures downstream.

How

Integration

  • Checks run via runEnhancedPreChecks() and runEnhancedPostChecks() in auto-verification.ts
  • Results are recorded in the verification evidence structure
  • Blocking failures trigger pauseNeeded = true in auto-post-unit.ts

Preferences

enhanced_verification: true          # master toggle (default: true)
enhanced_verification_pre: true      # pre-execution checks
enhanced_verification_post: true     # post-execution checks  
enhanced_verification_strict: false  # treat warnings as blocking (default: false)

Design decisions

  • Permissive by default (R012): Most checks warn rather than block. Only provably-impossible conditions (package 404, read-before-create) are blocking.
  • Parallel execution: Package checks use Promise.all() for concurrent npm lookups.
  • TypeScript ESM awareness: Import resolution handles .js.ts convention.

Testing

  • Unit tests: pre-execution-checks.test.ts, post-execution-checks.test.ts
  • Integration tests: enhanced-verification-integration.test.ts, pre-execution-pause-wiring.test.ts
  • All tests pass in CI

Files changed

File Purpose
pre-execution-checks.ts 4 pre-task validation checks
post-execution-checks.ts 3 post-task consistency checks
auto-verification.ts Wiring for enhanced checks
auto-post-unit.ts Pause control flow on blocking failures
preferences-types.ts Preference type definitions
preferences-validation.ts Preference validation
preferences.ts Add to mergePreferences
verification-evidence.ts Result type definitions

AI-assisted: This PR was developed using GSD-2's own auto-mode pipeline. All code has been reviewed and is understood by the author.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

🔴 PR Risk Report — CRITICAL

Files changed 13
Systems affected 1
Overall risk 🔴 CRITICAL

Affected Systems

Risk System
🔴 critical Auto Engine
File Breakdown
Risk File Systems
🔴 src/resources/extensions/gsd/auto-post-unit.ts Auto Engine
🔴 src/resources/extensions/gsd/auto-verification.ts Auto Engine
package-lock.json (unclassified)
src/resources/extensions/gsd/post-execution-checks.ts (unclassified)
src/resources/extensions/gsd/pre-execution-checks.ts (unclassified)
src/resources/extensions/gsd/preferences-types.ts (unclassified)
src/resources/extensions/gsd/preferences-validation.ts (unclassified)
src/resources/extensions/gsd/preferences.ts (unclassified)
src/resources/extensions/gsd/tests/enhanced-verification-integration.test.ts (unclassified)
src/resources/extensions/gsd/tests/post-execution-checks.test.ts (unclassified)
src/resources/extensions/gsd/tests/pre-execution-checks.test.ts (unclassified)
src/resources/extensions/gsd/tests/pre-execution-pause-wiring.test.ts (unclassified)
src/resources/extensions/gsd/verification-evidence.ts (unclassified)

⚠️ Critical risk — please verify: state persistence, auth token lifecycle, agent loop race conditions, RPC protocol compatibility.

@github-actions github-actions bot added enhancement New feature or request Medium Priority labels Apr 3, 2026
Alan Alwakeel added 5 commits April 4, 2026 23:43
Adds 4 pre-execution checks that run before each task:
- File ops review: surfaces create/edit/delete intent for manual review
- Read-before-create guard: fails when plan reads a file before creating it
- Package existence: verifies npm packages exist before install attempts
- Interface contract: warns on mismatched function signatures

Includes preference types and validation for enhanced_verification settings.
Adds 3 post-execution checks that run after task completion:
- Import resolution: verifies relative imports resolve to existing files
- Export verification: confirms exported symbols are defined
- Type consistency: validates function return types match declarations

All checks follow the permissive-by-default pattern (R012) - warnings don't block.
…cation

Integrates pre/post-execution checks into auto-mode:
- auto-verification.ts: runEnhancedPreChecks/runEnhancedPostChecks integration
- auto-post-unit.ts: pause control flow when blocking checks fail
- Respects enhanced_verification_strict preference for blocking vs warning

Control flow: blocking failures trigger auto-mode pause for user review.
The enhanced_verification_* preferences were validated and typed but not
included in mergePreferences(), causing project-level overrides to be
silently ignored. This fix ensures project preferences properly merge
with user-level defaults.
…cation

- pre-execution-pause-wiring.test.ts: Tests blocking check → pause control flow
- enhanced-verification-integration.test.ts: End-to-end integration coverage

Verifies that blocking pre-execution failures trigger auto-mode pause and
that the enhanced verification pipeline integrates correctly with existing
verification infrastructure.
@trek-e trek-e force-pushed the feat/enhanced-verification branch from 0b8a475 to c4091e0 Compare April 5, 2026 03:43
@jeremymcs
Copy link
Copy Markdown
Collaborator

This PR has merge conflicts with the base branch. Please rebase or merge main to resolve before review can proceed.

🤖 Automated PR audit — 2026-04-04

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Medium Priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants