[Schema Consistency] Schema Consistency Check: Conditional Logic Validation Gaps (2025-11-26)

[github-actions[bot]]

🔍 Schema Consistency Check - 2025-11-26

Overview

This analysis focused on conditional logic and feature flag interactions across the schema, parser, compiler, documentation, and real workflows. The investigation revealed significant gaps where runtime validation enforces rules not represented in the JSON Schema, making external validation tools unable to catch configuration errors.

Key Discovery: The compiler implements sophisticated conditional validation (like the label-only event exception for command triggers) that cannot be expressed in JSON Schema, creating a validation gap for IDE tooling and external validators.

Full Report Details

Summary

• Inconsistencies Found: 8
• Categories Analyzed: Schema conditionals, compiler validation, documentation, workflows
• Strategy Used: Strategy-005 - Conditional Logic & Feature Flag Analysis
• Effectiveness: Very High

Analysis Methodology

1. Schema Analysis: Examined 93 oneOf patterns and 8 if/then/else conditionals
2. Compiler Tracing: Traced conditional validation logic in pkg/workflow/compiler.go
3. Documentation Review: Cross-referenced with docs/troubleshooting/errors.md and reference docs
4. Edge Case Integration: Incorporated findings from recent edge case testing (tests 19, 20, 24)
5. Workflow Validation: Sampled real workflows for conditional pattern usage

Critical Issues

1. Command Trigger Conflicts Not Schema-Validated

Location: pkg/workflow/compiler.go:1334-1343

Issue: The compiler enforces mutual exclusivity between command triggers and comment-related events (issues, issue_comment, pull_request, pull_request_review_comment), but this rule is not encoded in the JSON Schema.

Code Evidence:

conflictingEvents := []string{"issues", "issue_comment", "pull_request", "pull_request_review_comment"}
for _, eventName := range conflictingEvents {
    if eventValue, hasConflict := onMap[eventName]; hasConflict {
        // Special case: allow issues/pull_request if they only have labeled/unlabeled types
        if (eventName == "issues" || eventName == "pull_request") && parser.IsLabelOnlyEvent(eventValue) {
            continue // Allow this - it doesn't conflict with command triggers
        }
        return fmt.Errorf("cannot use 'command' with '%s' in the same workflow", eventName)
    }
}

Impact:

• IDE validation cannot warn users about this conflict
• External schema validators will accept invalid configurations
• Users only discover the error at compile time, not at edit time

Documentation Status: ✅ Partially documented in errors.md:353 and command-triggers.md:40

2. Label-Only Event Exception Not Documented

Location: pkg/parser/schema.go:553-590 and pkg/workflow/compiler.go:1338

Issue: The implementation includes a nuanced exception - issues and pull_request events CAN be combined with command triggers IF they only contain labeled or unlabeled types. This exception is implemented but not documented.

Code Evidence:

func IsLabelOnlyEvent(eventValue any) bool {
    // Event can be a map with types field
    eventMap, isMap := eventValue.(map[string]any)
    if !isMap {
        return false
    }
    // ... checks if all types are labeled or unlabeled
}

Impact:

• Documentation says "cannot combine" without mentioning the exception
• Users may avoid valid configurations thinking they're forbidden
• Advanced pattern (command + label events) remains undiscovered

Documentation Gap: command-triggers.md:40 states categorical prohibition without exception

3. branches vs branches-ignore Mutual Exclusivity Missing

Location: Edge case test 20 revealed

Issue: GitHub Actions specifies that branches and branches-ignore are mutually exclusive, but gh-aw provides no validation for this GitHub Actions constraint.

Impact:

• Workflows compile successfully but may behave unexpectedly at runtime
• GitHub Actions will resolve the conflict using undocumented precedence rules
• User intent unclear when both filters present

Real World: 5+ workflows use branches, none use branches-ignore (no conflicts found in practice)

Recommendation: Add compile-time warning or error for this GitHub Actions anti-pattern

4. Logical Conflicts Not Detected (draft + ready_for_review)

Location: Edge case test 24 revealed

Issue: Logically contradictory PR filter combinations compile without warning:

• draft: true - only draft pull requests
• types: ["ready_for_review"] - only PRs marked ready for review

These conditions are mutually exclusive by definition (draft PRs cannot be ready for review), creating a filter that will never match.

Impact:

• Workflow compiles but will never trigger
• Silent logic error with no diagnostic feedback
• User debugging required to discover the issue

Real World: 12 workflows use draft: true, none combine with ready_for_review (good practices in actual usage)

Moderate Issues

5. Empty Schedule Array Accepted

Location: Edge case test 19

Issue: Schema accepts schedule: [] without validation

Impact: Workflow compiles but will never run on schedule

Fix: Add minItems: 1 constraint to schedule array in schema

6. Conditional Default Not Schema-Documented

Location: pkg/workflow/compiler.go:1328-1332

Issue: Command name defaults to filename only when command field exists but name not specified. This conditional default behavior is not documented in schema metadata.

Impact: IDE autocomplete cannot hint at automatic behavior

Code:

if workflowData.Command == "" {
    baseName := strings.TrimSuffix(filepath.Base(markdownPath), ".md")
    workflowData.Command = baseName
}

Positive Findings

✅ Well-Documented Conflicts

1. Command trigger conflicts documented in:

   • docs/troubleshooting/errors.md:353 (error message explanation)
   • docs/reference/command-triggers.md:40 (usage warning)

2. MCP container/command exclusivity documented in:

   • docs/troubleshooting/errors.md:437

✅ Clean Real Workflows

Real workflow analysis revealed excellent practices:

• No branches/branches-ignore conflicts
• No draft/ready_for_review contradictions
• No command + comment event conflicts
• 5+ workflows successfully use branches filters
• 12+ workflows appropriately use draft filtering

✅ Sophisticated Implementation

The IsLabelOnlyEvent() function demonstrates thoughtful implementation handling edge cases correctly, even if not fully documented.

Schema Limitations

JSON Schema cannot express:

1. Context-dependent mutual exclusivity (command conflicts only with certain event configurations)
2. Logical implication rules (draft implies not ready_for_review)
3. Cross-platform constraints (GitHub Actions inherent restrictions)
4. Conditional defaults (filename fallback only when command exists)

These limitations mean some validation MUST remain in the compiler, but should be documented in schema $comment fields or descriptions.

Recommendations

High Priority

1. Add schema $comment annotations for conditional rules:

   "command": {
     "$comment": "Cannot be combined with issues, issue_comment, pull_request, or pull_request_review_comment events (except when those events only use labeled/unlabeled types)",
     "description": "..."
   }

2. Document label-only exception in command-triggers.md:

   • Add section explaining the labeled/unlabeled exception
   • Provide example of valid command + issues.types: [labeled] combination

3. Add compile-time warnings for logical conflicts:

   • branches + branches-ignore combination
   • draft: true + types: [ready_for_review] combination

4. Add minItems constraints:

   • schedule: minItems: 1
   • runs-on (array form): minItems: 1

Medium Priority

5. Enhance schema descriptions with conditional behavior:

   • Command name defaults to filename (conditional)
   • Feature compatibility notes (engine-specific)

6. Add validation testing for conditional rules:

   • Unit tests for label-only event detection
   • Integration tests for conflict detection

Low Priority

7. Consider linter tool for advanced validation:
   • Logical conflict detection
   • GitHub Actions best practices
   • Platform-specific constraint checking

Conclusions

The analysis revealed a validation gap between compile-time enforcement and schema-time validation. While the compiler correctly implements sophisticated conditional logic, the JSON Schema cannot express these rules, creating challenges for:

• IDE autocomplete and validation
• External validation tooling
• Pre-commit hooks and CI/CD checks
• User discoverability of valid patterns

Key Insight: The label-only event exception exemplifies the limits of JSON Schema - the rule is too nuanced for declarative schema expression, requiring imperative validation logic.

Positive Note: Real workflows demonstrate excellent practices with zero instances of the identified anti-patterns, suggesting user documentation and error messages are effective despite schema gaps.

Strategy Performance

• Strategy Used: Strategy-005 (Conditional Logic Analysis)
• Success Count: 5 runs
• Findings This Run: 8
• Effectiveness: Very High
• Should Reuse: Yes - every 3-4 analyses

Novel Discoveries: This run integrated edge case testing insights (tests 19, 20, 24) with conditional analysis, revealing new issues beyond previous runs. The label-only exception discovery is particularly valuable for advanced users.

Next Steps

• Complete analysis
• Add schema $comment annotations for conditional rules
• Document label-only event exception in command-triggers.md
• Add minItems constraints to schedule and runs-on arrays
• Consider compile-time warnings for logical conflicts
• Enhance schema descriptions with conditional behavior notes

---

Analysis Date: 2025-11-26
Workflow Run: §19690567479
Strategy: Conditional Logic & Feature Flag Analysis
Effectiveness: Very High (8 findings, 4 critical)

AI generated by Schema Consistency Checker

[github-actions[bot]]

⚓ Avast! This discussion be marked as outdated by Schema Consistency Checker.
🗺️ A newer treasure map awaits ye at Discussion #5254.
Fair winds, matey! 🏴‍☠️