[Schema Consistency] Schema Consistency Audit - 2025-11-23: Field Relationship Dependencies

[github-actions[bot]]

This automated analysis identifies inconsistencies between the JSON schema, parser/compiler implementation, documentation, and real workflows. Today's analysis used a brand new strategy focused on discovering implicit field relationships and dependency chains.

Analysis Overview

Strategy Used: Schema Relationship Graph & Dependency Chain Analysis (Strategy-018) - NEW APPROACH

This analysis examined implicit relationships between schema fields - dependencies, conflicts, cascading defaults, and conditional behaviors that exist in the implementation but aren't explicitly documented in the schema. Unlike previous strategies that analyze individual fields, this approach maps the dependency graph between fields.

Scope Analyzed:

• 33 top-level schema fields + nested properties
• Inter-field conflict rules in compiler
• Cascading auto-enablement logic
• Conditional format-dependent behaviors
• Field precedence chains
• 77 workflows with safe-outputs configuration
• 3 workflows using command triggers

Inconsistencies Found: 8 relationship patterns

• Critical Issues: 2 (hierarchical dependencies, cascading defaults)
• Moderate Issues: 2 (conditional formats, default values)
• Info/Documentation: 4 (fallback chains, precedence rules)

Critical Findings

Critical Issue #1: command.events Enables Event Combination

Type: Hierarchical Dependency / Design Pattern Gap

Discovery:
The command trigger has format-dependent conflict behavior:

Simple Format (conflicts exist):

on:
  command: mybot
  issues: {}           # ERROR: "cannot use 'command' with 'issues'"
  pull_request: {}     # ERROR: "cannot use 'command' with 'pull_request'"

Object Format with events (conflicts removed):

on:
  command:
    name: mybot
    events: [issues, issue_comment, pull_request]  # ✅ OK - explicitly controlled

Code Evidence (pkg/workflow/compiler.go:1280):

// Check for conflicting events (only at top level)
conflictingEvents := []string{"issues", "issue_comment", "pull_request", "pull_request_review_comment"}
for _, eventName := range conflictingEvents {
    if _, hasConflict := onMap[eventName]; hasConflict {
        return fmt.Errorf("cannot use 'command' with '%s' in the same workflow", eventName)
    }
}

This validation only checks sibling relationships in the on: map. When using command.events, the events are nested inside the command object, not siblings.

Real World Usage:

# .github/workflows/archie.md (compiles successfully)
on:
  command:
    name: archie
    events: [issues, issue_comment, pull_request, pull_request_comment]

Schema Gap:
The schema shows command in the same oneOf pattern as all other events, but doesn't document:

• Simple format conflicts with specific events
• Object format with events field removes conflicts
• events field acts as a relationship modifier

Impact:

• IDE/editor validation cannot detect format-specific conflicts
• Users may not understand when conflicts apply vs. when they don't
• Design pattern (hierarchical dependency) undocumented

Recommendation:
Add schema description noting format-dependent behavior

---

Critical Issue #2: Cascading Auto-Enablement in safe-outputs

Type: Implicit Dependencies / Hidden Defaults

Discovery:
When safe-outputs is configured, three child fields are automatically enabled even if not specified:

safe-outputs: (configured) →
  ├─ missing-tool: auto-enabled (can disable: missing-tool: false)
  ├─ noop: auto-enabled with max: 1 (can disable: noop: false)
  └─ threat-detection: auto-enabled if any output type exists

Usage Statistics:

• 77 workflows have safe-outputs configured
• 3 workflows explicitly configure threat-detection (4%)
• 3 workflows explicitly configure missing-tool (4%)
• 0 workflows explicitly configure noop (0%)
• 74 workflows get automatic threat-detection (96%)
• 74 workflows get automatic missing-tool (96%)
• 77 workflows get automatic noop with max: 1 (100%)

Schema Gap:
The schema shows these as independent optional fields but doesn't document:

• Auto-enablement when parent exists
• Default max value for noop (1)
• Dependency on HasSafeOutputsEnabled() check
• How to explicitly disable (set to false)

Impact:

• High - Users unaware of automatic security features
• 96% of workflows rely on implicit threat detection
• Default noop max (1) not documented in schema
• Cannot predict behavior from schema alone

Recommendation:
Add schema metadata documenting the cascade

Moderate Findings

Moderate Issue #1: max-patch-size Default and Type Coercion

Type: Conditional Defaults / Type Flexibility

Discovery:
The max-patch-size field has cascading default logic with type coercion:

user specifies max-patch-size →
  ├─ int/int64/uint64 >= 1: use value
  ├─ float64: truncate to int (with warning), use if >= 1
  ├─ value < 1: ignore, apply default (1024 KB)
  └─ not specified: apply default (1024 KB)

Schema Gaps:

• Default value (1024 KB) not in schema
• Float truncation behavior not documented
• Runtime accepts floats, schema says integer only

Impact: Medium - Users may not know default, float behavior surprising

---

Moderate Issue #2: timeout-minutes → timeout_minutes Fallback

Type: Deprecation Precedence Chain

Discovery:
Preference order with deprecation warning:

timeout-minutes (exists?) →
  YES: use timeout-minutes
  NO: timeout_minutes (exists?) →
    YES: use timeout_minutes + emit deprecation warning
    NO: no timeout configured

Impact: Low - Works correctly, just needs documentation

Info / Documentation Gaps

Info #1: GitHub Token Precedence Chain

Discovery:
Token resolution follows 4-level precedence:

safe-outputs.{output-type}.github-token (most specific)
  ↓ (if not set)
safe-outputs.github-token (output-level)
  ↓ (if not set)
frontmatter.github-token (top-level)
  ↓ (if not set)
${{ github.token }} (GitHub Actions default)

Schema Gap: Three github-token fields shown independently, precedence not documented

---

Info #2: Network Field Name Collision

Discovery:
Two different fields both named network:

1. Top-level: frontmatter.network (engine network permissions)
2. MCP-specific: mcp-servers.*.network (MCP isolation config)

Different contexts, same name.

---

Info #3: Required Field Not Enforced

Discovery:
Schema marks on as required but compiler doesn't enforce - generates defaults if missing.

Cross-Reference: Confirmed finding from Strategy-003 (2025-11-20)

---

Info #4: HasSafeOutputsEnabled() Dependency

Discovery:
Many safe-outputs behaviors depend on HasSafeOutputsEnabled() check, which is not a field but a computed property.

Positive Findings

✅ command.events Pattern Works Correctly: Real workflow (archie.md) successfully uses command.events: [issues, pull_request] format

✅ Automatic Security Features: 96% of safe-outputs workflows benefit from automatic threat-detection enablement

✅ Type Coercion Resilience: max-patch-size accepts multiple numeric types with graceful float truncation

✅ Backward Compatibility: timeout_minutes → timeout-minutes fallback preserves old workflows

✅ Token Precedence Chain: GitHub token resolution correctly implements 4-level precedence

✅ No Broken Relationships: All discovered relationships work correctly in implementation - gaps are documentation only

Key Insights

1. Hierarchical Dependencies: Field behavior can change based on format (command string vs. object) - schema doesn't express this

2. Cascading Defaults: Parent field existence triggers auto-enablement of children (safe-outputs → threat-detection) - invisible to schema

3. Relationship Modifiers: Sub-fields can change parent field's relationships (command.events removes conflicts)

4. 96% Implicit Enablement: Nearly all safe-outputs workflows rely on automatic feature enablement

5. Inter-Field Relationships: More important than individual field validation for understanding system behavior

Recommendations

Priority 1: Document Hierarchical Dependencies (CRITICAL)

Add to schema description that format determines conflict behavior for command trigger.

Where: pkg/parser/schemas/main_workflow_schema.json - command field description

---

Priority 2: Document Cascading Auto-Enablement (HIGH)

Add schema metadata documenting that missing-tool, noop, and threat-detection are automatically enabled when safe-outputs exists.

Where: Each auto-enabled field's schema definition

---

Priority 3: Add Default Value Metadata (MEDIUM)

Formalize defaults:

• noop.max → 1
• max-patch-size → 1024 KB

Where: Field definitions in main_workflow_schema.json

---

Priority 4: Document Precedence Chains (MEDIUM)

Add note explaining github-token and network field precedence rules.

---

Priority 5: Consider Schema Extensions

• Use $comment fields for implementation notes
• Add x-dependencies custom field for inter-field relationships
• Consider separate relationship documentation file

Strategy Performance

Strategy: Schema Relationship Graph & Dependency Chain Analysis (Strategy-018)

Status: ⭐ NEW STRATEGY - First use

Effectiveness: Very High

Unique Value:

• First strategy to analyze inter-field relationships instead of individual fields
• Discovered hierarchical dependencies invisible to field-level validation
• Revealed design patterns (command.events enables, safe-outputs cascades)
• Found 96% implicit reliance on auto-enablement

Should Reuse: ✅ YES - Added to strategy cache for future use

Methodology Summary

1. Field Extraction: Listed all 33 top-level schema fields + nested properties
2. Dependency Mining: Searched for conditional field checks in compiler code
3. Conflict Detection: Found hardcoded conflicting events for command trigger
4. Cascade Analysis: Traced auto-enablement logic for safe-outputs children
5. Real Workflow Validation: Tested archie.md with command.events pattern
6. Relationship Mapping: Built dependency graph from code patterns
7. Usage Analysis: Counted workflows relying on implicit behaviors (96%)
8. Pattern Documentation: Identified 8 relationship patterns

Total Analysis Time: ~30 minutes
Files Analyzed: 15+ (schema, compiler, parser, workflows)
Code Patterns Found: 8 relationship patterns
Workflows Validated: 3 (archie, brave, cloclo with command triggers)

---

Strategy Cache Updated: ✅ Strategy-018 added to /tmp/gh-aw/cache-memory/strategies.json

Detailed Findings: See /tmp/gh-aw/cache-memory/strategy-018-findings-2025-11-23.md

AI generated by Schema Consistency Checker

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 1 week ago.