Skip to content

Audit: migrate advisory prompt rules to deterministic enforcement #519

@dollspace-gay

Description

@dollspace-gay

Summary

Comprehensive audit of all rules currently enforced via prompt injection (system reminder text) that could instead be enforced deterministically via hooks, CLI validation, linters, or CI gates. Moving rules from advisory to deterministic enforcement eliminates the compliance gap where agents ignore prompt-injected rules under context pressure.

Current state: 4 rules are hard-enforced via hooks. ~20 additional rules are injected as prompt text labeled "MANDATORY" / "ABSOLUTE REQUIREMENT" but have zero enforcement. Per GH #501, agents consistently comply with hard-gated rules and consistently ignore advisory ones.

Tier 1: Hook enforcement (PreToolUse/PostToolUse, minimal work)

These can be added to the existing work-check.py and post-edit-check.py hooks using the same patterns already in place.

1. Block stubs in agent-written files

Rule: "Never write TODO, FIXME, pass, ..., unimplemented!(), or empty function bodies" (global.md:111)
Current: post-edit-check.py detects stubs but exits 0 (advisory)
Fix: Change to exit 2 when stubs are detected in files the agent just wrote. The detection regex already exists (lines 41-61 of post-edit-check.py). Gate on agent context only — don't block the driver.
Mechanism: PostToolUse hook, exit 2

2. Require --kind on every crosslink issue comment

Rule: "Every comment MUST use the --kind flag. A comment without --kind is an incomplete comment." (tracking-strict.md:129)
Current: Zero enforcement
Fix: In work-check.py, when the Bash command matches crosslink issue comment or crosslink comment, check if --kind is present in the command string. Block if missing.
Mechanism: PreToolUse hook on Bash, regex match + exit 2

3. Require at least one label before crosslink issue close

Rule: Labels control CHANGELOG.md categories. Unlabeled issues default to "Changed" which is often wrong. (tracking-strict.md:33-41)
Current: Zero enforcement
Fix: In work-check.py, when the command matches crosslink issue close, query SQLite for label count on that issue. Block if zero labels. Skip check when --no-changelog is present (internal/refactor work doesn't need labels).
Mechanism: PreToolUse hook on Bash, SQLite query + exit 2

4. Block raw WebFetch tool usage

Rule: "Use mcp__crosslink-safe-fetch__safe_fetch for all web requests. Never use raw WebFetch." (global.md:80)
Current: pre-web-check.py injects RFIP reminder but exits 0
Fix: Add WebFetch to the PreToolUse matcher in work-check.py and exit 2 with a message directing to mcp__crosslink-safe-fetch__safe_fetch. Leave WebSearch as advisory (no safe alternative exists).
Mechanism: PreToolUse hook on WebFetch, exit 2

5. Auto-inject intervention logging after hook blocks

Rule: "You MUST log an intervention when a hook or policy blocks your tool call" (global.md:153-156)
Current: The block message includes intervention logging instructions, but agents skip it ~90% of the time
Fix: When work-check.py blocks a tool call (exit 2), write a breadcrumb file (.crosslink/.cache/pending-intervention). On the next successful tool call, prompt-guard.py checks for the breadcrumb and injects a hard reminder. Alternatively, gate the next Bash/Write/Edit on the intervention having been logged (check for a recent crosslink intervene command in shell history or a cleared breadcrumb).
Mechanism: PreToolUse hook + state file, exit 2 until intervention logged

6. Enforce session work <id> (not just any active session)

Rule: "ALWAYS use crosslink session work to mark focus" (tracking-strict.md:112)
Current: work-check.py checks for any active issue via crosslink session status, but doesn't verify session work was called — the issue could have been set by crosslink quick (which is fine) or inherited from a prior session (which might be stale)
Fix: In the session status JSON check, verify working_on.id is present and matches a currently-open issue. If the working_on issue is closed, block with "Your active work item is closed. Pick a new one with crosslink session work <id>."
Mechanism: PreToolUse hook, JSON parse + exit 2

Tier 2: CLI validation (enforce in the Rust binary itself)

These are validation rules that belong in the crosslink CLI, not in hooks. They fire at command execution time.

7. Issue title format validation

Rule: Titles must start with a verb (Add, Fix, Update, etc.), be changelog-ready, not be vague (tracking-strict.md:16-31)
Current: Zero enforcement
Fix: In crosslink issue create and crosslink quick, validate:

  • Title length >= 10 characters (reject "Fix bug", "Update code")
  • Title starts with a known verb from an allowlist (Add, Fix, Update, Remove, Improve, Implement, Refactor, etc.)
  • Warn (not block) if title doesn't match — print the good/bad examples from the rules
    Mechanism: CLI validation in commands/create.rs, warning or error

8. Comment without --kind rejected at CLI level

Rule: Same as #2 above
Fix: In addition to the hook check, crosslink issue comment itself could require --kind when comment_discipline is "required" in hook-config.json. This catches direct CLI invocations too, not just hook-gated ones.
Mechanism: CLI validation in commands/comment.rs

9. Subissue scheduling rejection

Rule: "--scheduled or --due with --parent must be rejected" (GH #361, REQ-12)
Fix: Already planned for the scheduling feature — validate at CLI parse time via clap conflicts_with or manual check in commands/create.rs.
Mechanism: CLI validation, clap conflicts

Tier 3: CI gates (enforce in GitHub Actions)

These rules can't be checked per-tool-call but can be verified before merge.

10. Every closed issue has typed comments

Rule: "If you close an issue that has zero typed comments, you have violated this rule." (tracking-strict.md:147)
Current: Partially enforced by comment discipline hooks (plan before commit, result before close). But issues can still be closed with only those two minimal comments.
Fix: CI gate that runs crosslink issue list -s closed --json and checks that each recently-closed issue has at least: 1 plan comment, 1 result comment. Fail the CI check if any are missing.
Mechanism: CI workflow step, crosslink issue show <id> --json | jq '.comments[] | select(.kind == "plan")' | wc -l

11. Test-before-commit gate

Rule: "Run the project's test suite after making code changes" (global.md:116)
Current: post-edit-check.py reminds but doesn't block. Agents routinely skip tests.
Fix: Gate git commit on .crosslink/last_test_run marker timestamp being newer than the most recent file modification. The marker is already written by crosslink issue tested <id>. If tests haven't run since the last edit, block the commit with "Run tests first: cargo test / npm test".
Mechanism: PreToolUse hook on git commit, file timestamp comparison + exit 2

12. No hardcoded secrets in committed files

Rule: "Never hardcode credentials, API keys, or tokens" (global.md:82)
Current: Zero enforcement
Fix: PostToolUse on Write/Edit, grep the file content for known secret patterns (sk-, ghp_, AKIA, password\s*=\s*"[^"]+", Bearer [A-Za-z0-9._-]{20,}). Warn (not block) since false positives are likely. Also add to CI as a trufflehog or gitleaks step.
Mechanism: PostToolUse advisory + CI gate

13. Read-before-write enforcement

Rule: "Always read a file before editing it. Never guess at contents." (global.md:112)
Current: Zero enforcement (Claude Code itself has this built in for Edit, but not for Write)
Fix: PreToolUse on Write — check if a Read for the same file_path occurred in the current session. Requires tracking Read calls in a state file (.crosslink/.cache/reads-this-session.txt). When Write is attempted on a file not in the reads list, warn "You haven't read this file yet."
Mechanism: PreToolUse hook with state tracking, advisory (exit 0 with warning)

14. crosslink workflow diff --check in CI

Rule: Deployed hook/rule files should match embedded defaults or be explicitly marked custom (workflow.rs:225-247)
Current: crosslink workflow diff --check exists but isn't in CI
Fix: Add to CI workflow: crosslink workflow diff --check. Fails if policy files have drifted without # crosslink:custom marker.
Mechanism: CI workflow step

Tier 4: Linter rules (static analysis)

15. No unwrap() / expect() in production code

Rule: Enforced by cargo clippy -- -W clippy::unwrap_used -W clippy::expect_used in CI
Current: Already in CI. Not in local hooks.
Fix: Add clippy with these flags to post-edit-check.py for Rust files (already runs clippy but without these flags locally).
Mechanism: PostToolUse linter, advisory

16. No SQL string interpolation

Rule: "Parameterized queries only. Never interpolate user input into SQL." (global.md:81)
Fix: Grep for patterns like format!("SELECT.*{} or &format!("INSERT.*{} in Rust files. Add as a CI lint step.
Mechanism: CI grep/lint

Enforcement coverage summary

Category Current After Tier 1 After All
Hard-gated (hook exit 2) 4 10 13
CLI validation 0 0 3
CI gates 1 (clippy) 1 5
Advisory (hook exit 0) 4 4 4
Prompt-only (no enforcement) ~20 ~9 ~4

Implementing Tier 1 alone (6 items) would move enforcement from 4 hard gates to 10 — a 2.5x increase with no new infrastructure.

Implementation priority

  1. Shared issue database on git coordination branch #2 (--kind on comments) + Add crosslink review subcommand for guided policy review with Claude #3 (labels before close) — highest value, simplest to implement, already have the SQLite query pattern from Audit: config keys with no hook enforcement rely solely on prompt injection #501
  2. Support hook-config.local.json for machine-local configuration #1 (block stubs) — flip exit 0 to exit 2 in existing detection code
  3. Rebrand chainlink → crosslink for forecast-bio #4 (block WebFetch) — one-line addition to work-check.py
  4. Add crosslink integrity subcommand with check and repair modes #11 (test-before-commit) — medium effort, high impact on code quality
  5. feat: crosslink review diff + /review slash command #7 (title validation) — CLI-side, requires Rust changes
  6. feat: support hook-config.local.json for machine-local overrides #5 (intervention logging) — needs state file plumbing, medium complexity

Related

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestquestionFurther information is requested

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions