[Schema Consistency] Cross-Schema Type Inconsistencies: Version Fields and Feature Availability

[github-actions[bot]]

🔍 Schema Consistency Check - 2025-12-01

This analysis reveals critical inconsistencies between the three JSON schemas that govern workflow validation. The main schema diverges significantly from both the included file schema and MCP config schema, creating validation gaps and undocumented feature limitations.

Executive Summary

Key Findings:

• 4 Critical Issues: Type mismatches and missing fields create validation inconsistencies
• 1 Moderate Issue: Validation requirement discrepancies
• 2 Positive Confirmations: Some definitions maintain perfect consistency
• Strategy Used: Cross-Schema Consistency & Type Analysis (Strategy-004)
• Analysis Approach: Proven strategy from cache (70% selection rate, day 335 mod 10 = 5)

Impact: Workflows that validate against one schema may fail validation against another. Advanced features advertised in main schema are silently unavailable in included files without documentation.

Full Report Details

Critical Issues

1. MCP Tool Version Type Mismatch

Problem: The three schemas disagree on what types the version field accepts for MCP tools.

Schema Comparison:

# Main schema - stdio_mcp_tool.version
type: ["string", "number"]
description: "...Numeric values are automatically converted to strings at runtime."
examples: ["latest", "v1.0.0", 20, 3.11]

# Included schema - stdio_mcp_tool.version
type: "string"
description: "Optional version/tag for the container image (e.g., 'latest', 'v1.0.0')"

# MCP config schema - version
type: "string"
description: "Version or tag for container images"

Impact:

• A workflow using version: 20 validates against main schema but fails against included/MCP schemas
• Creates cross-schema validation inconsistency
• External validation tools (IDEs, linters) may reject valid workflows
• No documentation warns users about this difference

Location:

• pkg/parser/schemas/main_workflow_schema.json - $defs.stdio_mcp_tool.version
• pkg/parser/schemas/included_file_schema.json - $defs.stdio_mcp_tool.version
• pkg/parser/schemas/mcp_config_schema.json - properties.version

Recommendation: Unify type definitions. Either:

• Option A: All schemas accept ["string", "number"] with documented type coercion
• Option B: All schemas accept "string" only and document numeric version rejection

---

2. Engine Configuration Field Mismatch

Problem: Main schema defines 10+ engine configuration fields, but included schema only supports 6 basic fields.

Missing from Included Schema:

• concurrency - Controls concurrent workflow execution
• user-agent - Custom user agent for GitHub MCP (codex engine)
• error_patterns - Custom error pattern validation
• config - Additional TOML configuration (codex engine)
• args - Command-line arguments for engine CLI

Schema Comparison:

# Main schema - engine_config (10+ fields)
properties:
  id, version, model, max-turns, env, steps
  concurrency, user-agent, error_patterns, config, args  # Missing in included

# Included schema - engine (6 fields only)
properties:
  id, version, model, max-turns, env, steps
  # concurrency, user-agent, error_patterns, config, args NOT DEFINED

Impact:

• Advanced engine features unavailable in included files
• No schema validation error - fields silently ignored
• Users cannot configure concurrency or error patterns in shared configurations
• Undocumented limitation - documentation doesn't explain included file restrictions

Location:

• pkg/parser/schemas/main_workflow_schema.json - $defs.engine_config
• pkg/parser/schemas/included_file_schema.json - properties.engine

Recommendation:

1. Document which engine fields are main-workflow-only vs included-file-compatible
2. Add $comment to included schema explaining the limitation
3. Consider: Should included files support full engine configuration?

---

3. Network Field Structural Difference

Problem: Main schema has rich network configuration with AWF (Agent Workflow Firewall) support, but included schema has minimal configuration.

Schema Comparison:

# Main schema - network
oneOf:
  - type: string, enum: ["defaults"]  # enum, not const
  - type: object
    properties:
      allowed: [array of domains]
      firewall:  # AWF configuration - 40+ lines
        oneOf: [null, boolean, "disable", object]
        object.properties: [args, version, log-level]

# Included schema - network
oneOf:
  - type: string, const: "defaults"  # const, not enum
  - type: object
    properties:
      allowed: [array of domains]
      # NO firewall field

Differences:

1. String constant: Main uses enum: ["defaults"], Included uses const: "defaults"
2. Firewall field: Main has extensive AWF configuration, Included has none
3. Documentation: Main has 20+ lines about ecosystem identifiers, Included minimal
4. Version types: Main firewall.version accepts ["string", "number"], Included N/A

Impact:

• AWF firewall (network egress control) unavailable in included files
• Security feature limitation not documented
• const vs enum creates minor validation difference (both work but semantically different)

Location:

• pkg/parser/schemas/main_workflow_schema.json - properties.network
• pkg/parser/schemas/included_file_schema.json - properties.network

Recommendation:

1. Document that AWF firewall is main-workflow-only
2. Explain why included files have simplified network configuration
3. Consider: Should included files support firewall configuration?

---

4. MCP Config anyOf Requirement Discrepancy

Problem: MCP config schema has permissive validation requirements that differ from main/included schemas.

Schema Comparison:

# MCP config schema - anyOf
anyOf:
  - required: ["type"]      # Type-only valid
  - required: ["url"]       # URL-only valid
  - required: ["command"]   # Command-only valid
  - required: ["container"] # Container-only valid

# Main/Included stdio_mcp_tool - anyOf
anyOf:
  - required: ["command"]   # Command required
  - required: ["container"] # Or container required
# Type-only and URL-only NOT valid

Impact:

• MCP config schema accepts incomplete configurations (type-only, url-only)
• Main/included schemas require command or container for stdio tools
• Creates validation gap where MCP config accepts configs that main/included reject
• May allow invalid MCP server definitions through initial validation

Location:

• pkg/parser/schemas/mcp_config_schema.json - root anyOf
• pkg/parser/schemas/main_workflow_schema.json - $defs.stdio_mcp_tool.anyOf
• pkg/parser/schemas/included_file_schema.json - $defs.stdio_mcp_tool.anyOf

Recommendation: Fix MCP config anyOf to match stdio_mcp_tool requirements:

"anyOf": [
  {"required": ["command"]},
  {"required": ["container"]}
]

Or document why MCP config intentionally has different requirements.

Moderate Issues

5. Version Field Description Inconsistency

Problem: Version field descriptions don't consistently document type coercion behavior.

Comparison:

• Main engine.version: "...Numeric values are automatically converted to strings at runtime."
• Included engine.version: "Optional version of the action" (no coercion mention)
• Main MCP version: "...Numeric values automatically converted to strings"
• Included MCP version: "Optional version/tag for the container image" (no coercion mention)

Impact: Users reading included schema don't know if numeric versions work

Recommendation: Standardize descriptions to always mention type coercion (if supported) or explicitly state "string only"

Positive Findings

✅ http_mcp_tool Definition Identical

The http_mcp_tool definition is perfectly consistent across main and included schemas:

• All 5 fields match exactly: type, registry, url, headers, allowed
• Property types identical
• Validation rules identical
• Excellent reference example of schema consistency

✅ Permissions Field Present in Both

Both main and included schemas define the permissions field with GitHub Actions permission scopes. Structure appears consistent (deeper validation needed).

Type Coercion Pattern Analysis

Main Schema Pattern: Consistently uses ["string", "number"] for version fields with documented runtime conversion:

• engine.version
• engine.firewall.version
• stdio_mcp_tool.version

Included/MCP Schema Pattern: Only accepts "string" type, no numeric support documented

This suggests an intentional design where:

• Main schema is more permissive (accepts both types)
• Included/MCP schemas are stricter (string-only)
• Runtime likely coerces anyway, but schema validation differs

Schema Architecture Insights

The three schemas serve different purposes:

1. Main Workflow Schema (main_workflow_schema.json)

   • Most comprehensive, 10+ fields in engine config
   • Supports advanced features: AWF firewall, concurrency, error patterns
   • Accepts flexible types with runtime coercion
   • Target audience: Primary workflow authors

2. Included File Schema (included_file_schema.json)

   • Simplified subset of main schema
   • 6 basic engine fields only
   • No firewall configuration
   • Target audience: Shared configuration authors

3. MCP Config Schema (mcp_config_schema.json)

   • Standalone MCP tool definition
   • Permissive validation (type-only valid)
   • String-only version type
   • Target audience: MCP server configuration

Design Question: Is the feature limitation in included files intentional or accidental?

Recommendations by Priority

Priority 1: CRITICAL - Fix Type Definitions

Action: Unify version field type definitions across all schemas

• Option A: All accept ["string", "number"] with documented coercion
• Option B: All accept "string" only with clear documentation

Files:

• pkg/parser/schemas/main_workflow_schema.json
• pkg/parser/schemas/included_file_schema.json
• pkg/parser/schemas/mcp_config_schema.json

Priority 2: CRITICAL - Document Feature Availability

Action: Add documentation explaining which features are available in main vs included files

Changes:

1. Add $comment fields to included schema explaining limitations:

   "engine": {
     "$comment": "Included files support basic engine config only. For advanced features (concurrency, error_patterns, args), use main workflow schema.",
     "oneOf": [...]
   }

2. Update frontmatter documentation (docs/src/content/docs/reference/frontmatter.md)

3. Add "Included File Limitations" section to docs

Priority 3: HIGH - Unify Network Configuration

Action: Either add firewall to included schema OR document why it's main-only

Decision needed: Should included files support AWF firewall configuration?

• Yes: Copy firewall definition from main to included
• No: Add documentation explaining security model difference

Priority 4: MEDIUM - Fix MCP Config Validation

Action: Update MCP config schema anyOf requirements to match stdio_mcp_tool

File: pkg/parser/schemas/mcp_config_schema.json

Change:

"anyOf": [
  {"required": ["command"]},
  {"required": ["container"]}
]

Priority 5: LOW - Standardize Descriptions

Action: Use consistent language for version field descriptions across all schemas

Pattern: Always mention type coercion if supported, or explicitly state "string only"

Testing Recommendations

To prevent future schema drift:

1. Create Schema Consistency Tests

   • Validate that stdio_mcp_tool version types match across schemas
   • Validate that shared definitions have identical field sets
   • Flag when main and included definitions diverge

2. Add Documentation Validation

   • Test that documented features exist in all relevant schemas
   • Test that schema limitations are documented

3. Create Schema Diff Tool

   • Automated detection of definition drift
   • Report when $defs diverge between schemas

Strategy Performance

Strategy Used: Strategy-004 (Cross-Schema Consistency & Type Analysis)

• Effectiveness: Very High
• Success Count: 5 (including this run)
• Findings: 7 total (4 critical, 1 moderate, 2 positive)
• Unique Value: Only strategy focused on inter-schema comparison
• Should Reuse: YES - Continue using every 4-5 analyses

Key Strengths:

• Reveals schema drift invisible to single-schema validation
• Catches structural mismatches and duplication issues
• Essential for multi-schema projects
• Complements field-level validation strategies

Next Steps

• Decide: Unify version types OR document string-only requirement
• Decide: Add advanced engine fields to included schema OR document limitation
• Decide: Add AWF firewall to included schema OR document main-only
• Fix MCP config anyOf requirements
• Add schema consistency tests to CI/CD
• Update documentation with included file limitations

---

Analysis Method: Strategy-004 selected from cache using deterministic selection (day 335 mod 10 = 5, within proven strategy range). Previous run 2025-11-28 found 6 issues. This run found 7 issues with different focus areas, confirming strategy's continued effectiveness.

Strategy Database: Updated with success count 5, findings 7, last_used 2025-12-01

AI generated by Schema Consistency Checker

[github-actions[bot]]

⚓ Avast! This discussion be marked as outdated by Schema Consistency Checker.
🗺️ A newer treasure map awaits ye at Discussion #5254.
Fair winds, matey! 🏴‍☠️