[Schema Consistency] Schema Consistency Check - Cross-Schema Inconsistencies (2025-11-28)

[github-actions[bot]]

Analysis of cross-schema consistency between main_workflow_schema.json, included_file_schema.json, and mcp_config_schema.json revealed 6 significant inconsistencies primarily related to MCP tool definitions, permissions handling, and engine configurations.

Key Findings: stdio_mcp_tool definition mismatch, missing permissions in main schema, engine enum differences, and $defs duplication without consistency.

Critical Issues (3 High/Medium Severity)

1. stdio_mcp_tool Definition Mismatch ⚠️ HIGH

Problem: The stdio_mcp_tool $def has inconsistent property definitions across schemas.

Details:

• Main schema: ✅ Correctly EXCLUDES headers and url properties (only in http_mcp_tool)
• Included schema: ❌ Incorrectly INCLUDES headers and url properties in stdio_mcp_tool
• MCP config schema: ✅ Includes both but validates against them for stdio type

Evidence:

# Main schema stdio_mcp_tool: 10 properties (no headers, no url)
# Included schema stdio_mcp_tool: 12 properties (has headers, has url)
# MCP config has allOf validation rejecting headers for stdio

Impact:

• Included file schemas allow invalid configurations (stdio with headers/url)
• IDE autocomplete shows invalid options for stdio connections
• External validation tools receive conflicting signals
• Users could write invalid configs that pass included schema validation

Location:

• pkg/parser/schemas/included_file_schema.json:450-578 (stdio_mcp_tool $def)

Recommendation: Remove headers and url from included schema's stdio_mcp_tool definition to match main schema.

---

2. Permissions Field Missing in Main Schema ⚠️ MEDIUM

Problem: The permissions property is missing entirely from main_workflow_schema.json but fully defined in included_file_schema.json.

Details:

• Main schema: Has permissions property but NO sub-properties defined (empty object)
• Included schema: Complete definition with 16 permission scopes (actions, checks, contents, deployments, discussions, id-token, issues, metadata, packages, pages, pull-requests, repository-projects, security-events, statuses, attestations, models)

Impact:

• Main workflow files get NO schema validation for permission scopes
• IDE autocomplete doesn't work for permissions in main workflows
• Inconsistent validation behavior between main and included files
• Users miss typos in permission names

Recommendation: Copy the complete permissions property definition from included schema to main schema.

---

3. Engine Configuration Mismatch ⚠️ MEDIUM

Problem: Engine enum values differ between main and included schemas.

Details:

• Main schema: Uses $ref to #/$defs/engine_config with enums: ["claude", "codex", "copilot", "custom"]
• Included schema: Inline enum with only ["claude", "codex", "custom"] (missing "copilot")

Impact:

• Included files cannot use "copilot" engine option
• Inconsistent engine availability between main and included files
• Default engine "copilot" not available in includes

Location: pkg/parser/schemas/included_file_schema.json:216-263

Recommendation: Add "copilot" to included schema engine enum, or use $ref to engine_config.

Moderate Issues (3 Medium/Low Severity)

4. anyOf Requirements Inconsistency

Severity: MEDIUM

Main schema's stdio_mcp_tool requires either command or container, but MCP config schema's anyOf is more flexible, allowing type inference from type, url, command, or container fields.

Recommendation: Align anyOf requirements - MCP config approach is more user-friendly.

---

5. Network Field Description Mismatch

Severity: LOW

• Main schema: "Network access control for AI engines using ecosystem identifiers and domain allowlists. Controls web fetch and search capabilities."
• Included schema: "Network permissions configuration for allowed domains"

Recommendation: Use main schema's comprehensive description in both places.

---

6. $defs Duplication Without Consistency

Severity: MEDIUM

Both main and included schemas define stdio_mcp_tool and http_mcp_tool in their $defs, but with different properties. This violates DRY principle and creates maintenance burden.

Impact:

• Definitions drift apart over time (already happening with stdio_mcp_tool)
• No single source of truth for MCP tool structure
• Changes must be synchronized manually

Recommendation: Extract common $defs to shared schema file and use $ref, or designate main schema as authoritative.

Positive Findings ✅

1. Mutual Exclusivity Validation: Both schemas correctly enforce that command and container cannot both be specified
2. Network + Container Dependency: Both correctly require container when network field is present
3. MCP Config Schema Quality: Excellent validation with runtime type checking
4. http_mcp_tool Consistency: Definition is identical between main and included schemas (5 properties each)
5. Documentation Accuracy: Correctly describes headers as HTTP-only (tools.md:183)

Technical Analysis Details

Comparison Matrix

Component	Main Schema	Included Schema	MCP Config	Parser
stdio headers	❌ No	✅ Yes (wrong)	✅ Yes + validation	✅ Accepts all
stdio url	❌ No	✅ Yes (wrong)	✅ Yes + validation	✅ Accepts all
permissions	❌ No properties	✅ 16 scopes	N/A	✅ Handles all
engine copilot	✅ Yes	❌ No	N/A	✅ Handles all
anyOf rules	Strict (2)	N/A	Flexible (4)	Permissive

Parser Implementation (mcp.go:66)

type MCPServerConfig struct {
    URL            string            `json:"url"`            // for http
    Headers        map[string]string `json:"headers"`        // for http
    Command        string            `json:"command"`        // for stdio
    Container      string            `json:"container"`      // for docker
    // ... other fields
}

The parser struct accepts both URL and Headers for all connection types (comments say "for http"). This permissive parsing relies on downstream validation, but included schema fails to restrict these fields properly.

Cross-Schema $defs

Main schema $defs: engine_config, github_token, http_mcp_tool, stdio_mcp_tool
Included schema $defs: http_mcp_tool, safe_job, stdio_mcp_tool

The overlapping stdio_mcp_tool and http_mcp_tool definitions should be identical but aren't.

Recommendations & Next Steps

High Priority

1. ✅ Fix stdio_mcp_tool in Included Schema

   • Remove headers and url properties from $defs.stdio_mcp_tool
   • Match main schema definition exactly
   • File: pkg/parser/schemas/included_file_schema.json:450-578

2. ✅ Add Permissions to Main Schema

   • Copy complete permissions property from included schema
   • Ensure all 16 scopes defined with read/write/none enums
   • Maintain GitHub Actions permission model consistency

3. ✅ Add "copilot" to Included Schema

   • Update engine enum to ["claude", "codex", "copilot", "custom"]
   • Or use $ref to #/$defs/engine_config (preferred)
   • File: pkg/parser/schemas/included_file_schema.json:228

Medium Priority

4. Unify $defs Across Schemas

   • Extract common definitions to shared schema
   • Use $ref to import shared definitions
   • Eliminate duplication and drift risk

5. Align anyOf Requirements

   • Make main schema's stdio_mcp_tool anyOf more flexible
   • Allow type inference from field presence

6. Standardize Descriptions

   • Use main schema descriptions as authoritative
   • Copy to included schema for consistency

Low Priority

7. Parser Validation Enhancement

   • Add runtime validation in mcp.go to reject headers/url for stdio
   • Don't rely solely on schema validation

8. Documentation Update

   • Document schema differences between main and included files
   • Clarify engine availability in each context

Strategy Performance

Strategy Used: Cross-Schema Consistency & Type Analysis (Strategy #4)

Effectiveness: VERY HIGH

Why It Worked:

• Comparing $defs across schemas revealed structural mismatches
• Side-by-side property comparison exposed missing/extra fields
• Type validation analysis showed enum differences
• Parser code cross-reference confirmed real-world impact

Key Insights:

• Schema drift occurs when $defs are duplicated across files
• Included schema serves different purpose but needs coordination with main
• MCP config schema has most sophisticated validation (runtime type checking)
• Parser is permissive (accepts all fields) but downstream validation matters
• Three schemas need better consistency maintenance

Should Reuse: YES - Use every 4-5 analyses to catch schema drift in multi-schema projects.

---

Analysis Date: 2025-11-28
Schemas Analyzed: 3 (main_workflow_schema.json, included_file_schema.json, mcp_config_schema.json)
Total Inconsistencies: 6 (3 high/medium severity, 3 moderate/low severity)
Method: Automated jq-based schema comparison + parser code analysis
Files Examined: Schema files, pkg/parser/mcp.go, docs/src/content/docs/reference/tools.md

AI generated by Schema Consistency Checker

[pelikhan]

/plan

[github-actions[bot]]

🎉 Yo ho ho! Plan Command found the treasure and completed successfully! ⚓💰

[github-actions[bot]]

⚓ Avast! This discussion be marked as outdated by Schema Consistency Checker.
🗺️ A newer treasure map awaits ye at Discussion #5254.
Fair winds, matey! 🏴‍☠️