[Schema Consistency] Cross-Schema Consistency Analysis - December 9, 2025

[github-actions[bot]]

Cross-Schema Consistency Analysis - December 9, 2025

This analysis identified 7 schema consistency issues across the three schema files, focusing on type mismatches, missing fields, and structural inconsistencies that could cause validation failures or limit feature availability.

Key Findings Overview

The cross-schema analysis revealed critical inconsistencies between main_workflow_schema.json, included_file_schema.json, and mcp_config_schema.json. The most significant issues are type mismatches in version fields that cause runtime behavior to diverge from schema validation, and missing fields in included schemas that prevent advanced features from being used in imported files.

Full Report Details

Analysis Metadata

• Date: 2025-12-09
• Strategy Used: Strategy-004 (Cross-Schema Consistency & Type Analysis)
• Strategy Performance: 7 findings (5 critical, 2 moderate)
• Previous Uses: 7 times (last: 2025-12-06)
• Effectiveness: Very High
• Schemas Analyzed: 3 files
  • pkg/parser/schemas/main_workflow_schema.json
  • pkg/parser/schemas/included_file_schema.json
  • pkg/parser/schemas/mcp_config_schema.json

---

Critical Issues

🔴 Issue 1: Engine Version Type Mismatch

Severity: Critical
Location: engine.version property
Files: main_workflow_schema.json vs included_file_schema.json

Problem:

Main schema:    type: ["string", "number"]  ✅ Accepts both "beta" and 20
Included schema: type: "string"             ❌ Only accepts "beta"

Impact:

• Workflows using numeric versions (e.g., version: 20) validate in main but fail in included files
• Runtime accepts both types, creating validation gap
• Users cannot use numeric versions in included files despite runtime support

Example:

# Valid in main_workflow_schema.json
engine:
  id: claude
  version: 20  # ✅ works

# Invalid in included_file_schema.json
engine:
  id: claude
  version: 20  # ❌ fails validation

Recommendation: Update included_file_schema.json engine.version to accept ["string", "number"]

Files to modify:

• pkg/parser/schemas/included_file_schema.json:455 - Change version type to array

---

🔴 Issue 2: MCP Version Type Inconsistency

Severity: Critical
Location: version property in MCP tool configurations
Files: mcp_config_schema.json vs main_workflow_schema.json & included_file_schema.json

Problem:

MCP config schema:        type: "string"           ❌ Only strings
Main schema MCP tool:     type: ["string", "number"] ✅ Accepts both
Included schema MCP tool: type: ["string", "number"] ✅ Accepts both

Impact:

• Standalone MCP config files cannot use numeric versions even though embedded MCP tools can
• Common pattern version: 3.11 (for Python) fails in standalone configs
• Inconsistent user experience across MCP configuration methods

Example:

# Embedded in workflow - WORKS
mcp-servers:
  python-server:
    container: python
    version: 3.11  # ✅ works in main/included schemas

# Standalone mcp_config.json - FAILS
{
  "container": "python",
  "version": 3.11  // ❌ fails in mcp_config_schema.json
}

Recommendation: Update mcp_config_schema.json version field to accept ["string", "number"]

Files to modify:

• pkg/parser/schemas/mcp_config_schema.json:95-98 - Change version type to array

---

🔴 Issue 3: Engine Fields Missing in Included Schema

Severity: Critical
Location: engine object properties
Files: included_file_schema.json missing fields from main_workflow_schema.json

Problem:

Field	Main Schema	Included Schema	Impact
args	✅	❌	Cannot pass custom engine arguments
concurrency	✅	❌	Cannot control agent job concurrency
config	✅	❌	Cannot provide engine-specific config
error_patterns	✅	❌	Cannot define custom error patterns
user-agent	✅	❌	Cannot customize user agent (codex)

Impact:

• Included files cannot use 5 advanced engine features
• Particularly problematic for concurrency - users may want to control parallel execution in included files
• user-agent for codex engine completely unavailable in includes

Example:

# main_workflow.md - WORKS
engine:
  id: claude
  concurrency:
    group: "gh-aw-claude"
    cancel-in-progress: false

# included_file.md - FAILS
engine:
  id: claude
  concurrency:  # ❌ Not in schema
    group: "gh-aw-claude"

Recommendation:

1. If intentional: Document why these fields are excluded from included files
2. If oversight: Add all 5 fields to included_file_schema.json engine definition
3. Likely intentional for simplicity, but at minimum add concurrency as it's commonly needed

Files to modify (if adding fields):

• pkg/parser/schemas/included_file_schema.json:437-486 - Expand engine properties

---

🔴 Issue 4: Network Firewall Property Missing

Severity: Critical
Location: network.firewall property
Files: included_file_schema.json missing from main_workflow_schema.json

Problem:

# Main schema network
network:
  allowed: ["defaults", "python"]
  firewall:              # ✅ Available
    version: "v1.0.0"
    log-level: "debug"

# Included schema network
network:
  allowed: ["defaults", "python"]
  # firewall property not in schema ❌

Impact:

• Included files cannot configure firewall version pinning
• Cannot set custom firewall log levels for debugging
• Advanced network security features unavailable in included files

Recommendation: Add firewall property to included_file_schema.json network configuration OR document why firewall is main-workflow-only

Files to modify (if adding):

• pkg/parser/schemas/included_file_schema.json:629-652 - Add firewall property to network object

---

🟡 Issue 5: Safe-Outputs Semantic Overload

Severity: Moderate (by design, but confusing)
Location: safe-outputs property
Files: Both schemas, different purposes

Problem: The safe-outputs property name is identical but has completely different meanings:

Main Workflow Schema:

safe-outputs:
  create-issue:     # Configure GitHub operations
    max: 1
    labels: ["bug"]
  create-pull-request:
    max: 1
  # ... 20+ operation types

Included File Schema:

safe-outputs:
  jobs:             # Define custom job templates
    my-custom-job:
      description: "Custom workflow job"
      inputs: {...}
      steps: [...]

Impact:

• Different semantics: Main = configure operations, Included = define job templates
• Users may expect consistent behavior across schemas
• Same property name, different structure causes confusion
• Not a bug - this is intentional design, but naming is misleading

Recommendation:

1. Documentation: Clearly document this distinction in both schemas' descriptions
2. Naming (breaking change): Consider renaming included schema's safe-outputs to safe-jobs or job-templates for clarity
3. Schema comments: Add $comment fields explaining the dual purpose

Files to modify:

• pkg/parser/schemas/main_workflow_schema.json - Add comment about dual use
• pkg/parser/schemas/included_file_schema.json:487-503 - Enhance description
• Documentation: docs/src/content/docs/reference/frontmatter.md

---

🟡 Issue 6: Permissions Description Inconsistency

Severity: Moderate
Location: permissions property description
Files: Both schemas, merge behavior unclear

Problem:

// Main schema
"description": "GitHub token permissions for the workflow. Controls what the GITHUB_TOKEN can access during execution."

// Included schema
"description": "GitHub Actions permissions for the workflow (merged with main workflow permissions)"

Impact:

• Included schema explicitly mentions "merged" but doesn't explain merge strategy
• Main schema doesn't mention merging at all
• Unclear how permissions combine: union, intersection, override, or error?

Example Questions:

# main.md
permissions:
  contents: read
  issues: write

# included.md
permissions:
  contents: write  # Override main? Error? Union?
  pull-requests: write

Recommendation:

1. Document merge semantics clearly in both schemas
2. Specify merge strategy (likely union with included taking precedence)
3. Add examples showing merge behavior

Files to modify:

• pkg/parser/schemas/main_workflow_schema.json - Update permissions description
• pkg/parser/schemas/included_file_schema.json:653-744 - Clarify merge behavior
• Add to documentation with examples

---

Additional Findings

ℹ️ Tools Configuration Subset (By Design)

Severity: Info
Observation: Included schema supports only 4 tools vs 13 in main schema

Tools in Main: agentic-workflows, bash, cache-memory, edit, github, playwright, repo-memory, safety-prompt, serena, startup-timeout, timeout, web-fetch, web-search (13 total)

Tools in Included: bash, cache-memory, github, repo-memory (4 total)

Impact:

• Included files missing 9 tools including: safety-prompt, timeout, startup-timeout
• Appears intentional (included files should be simpler)
• May surprise users expecting full tool availability

Recommendation: Document which tools are available in included files vs main workflows

---

Positive Findings ✅

1. MCP Tool Definitions Consistent

The $defs/stdio_mcp_tool and $defs/http_mcp_tool definitions are identical between main and included schemas (except for version type already reported). Properties, validation rules, and structure match perfectly.

2. Schema Organization Sound

The three schemas serve distinct purposes effectively:

• main_workflow_schema.json: Complete workflow configurations
• included_file_schema.json: Reusable/importable components
• mcp_config_schema.json: Standalone MCP server definitions

This architectural separation is well-designed.

---

Summary Statistics

Metric	Value
Total Issues	7
Critical	4
Moderate	2
Info	1
Type Mismatches	2
Missing Fields	6 (engine: 5, network: 1)
Structural Differences	2 (safe-outputs, tools)
Positive Validations	2

---

Recommendations

High Priority (Fix Soon)

1. Fix engine.version type mismatch

   • File: pkg/parser/schemas/included_file_schema.json:455
   • Change: "type": "string" → "type": ["string", "number"]
   • Impact: Enables numeric versions in included files

2. Fix MCP version type mismatch

   • File: pkg/parser/schemas/mcp_config_schema.json:95-98
   • Change: "type": "string" → "type": ["string", "number"]
   • Impact: Enables numeric versions like 3.11 in standalone MCP configs

3. Document engine field availability

   • Add schema $comment explaining why 5 fields excluded from included
   • Document in docs/src/content/docs/reference/frontmatter.md
   • Consider adding concurrency if limitation not intentional

4. Document network firewall availability

   • Add comment explaining firewall property availability
   • Update docs to clarify main-only vs included features

Medium Priority (Document/Clarify)

5. Clarify safe-outputs dual purpose

   • Enhance descriptions in both schemas
   • Add examples showing different uses
   • Consider renaming in future major version

6. Document permissions merge behavior

   • Specify merge strategy (union/override/intersection)
   • Add examples with main + included permissions
   • Document precedence rules

7. Document tools availability

   • List which tools work in included files
   • Explain why tools subset is intentional
   • Add warnings for missing tools users might expect

Low Priority (Future Improvement)

8. Schema validation in CI

   • Add automated cross-schema consistency checks
   • Prevent future type drift
   • Validate $defs remain synchronized

9. Schema versioning

   • Consider adding $id with version to schemas
   • Track breaking changes explicitly
   • Enable better tooling support

---

Test Cases Needed

Add test workflows validating:

1. ✅ Numeric version values in included files: version: 20
2. ✅ Numeric version values in MCP config: version: 3.11
3. ✅ Engine concurrency in included files (should work or have clear error)
4. ✅ Network firewall in included files (should work or have clear error)
5. ✅ Permissions merging: main read + included write = ?
6. ✅ Safe-outputs in both contexts
7. ✅ Tools availability in included files

---

Strategy Performance

• Strategy ID: strategy-004
• Strategy Name: Cross-Schema Consistency & Type Analysis
• Findings This Run: 7 (5 critical, 2 moderate, 1 info)
• Success Count: 7 total runs
• Last Used: 2025-12-09 (today)
• Previous Use: 2025-12-06 (3 days ago)
• Effectiveness: Very High ⭐⭐⭐⭐⭐

Why This Strategy Works:

• Focuses on inter-schema relationships vs single-schema validation
• Catches drift between related schemas over time
• Reveals type inconsistencies invisible to single-file analysis
• Finds structural differences that impact feature availability

When to Use: Every 3-4 analyses to maintain consistency across schema evolution. Essential for multi-schema projects where related schemas must stay synchronized.

---

Files Referenced

Schema Files

• pkg/parser/schemas/main_workflow_schema.json (48,833 tokens)
• pkg/parser/schemas/included_file_schema.json (1,044 lines)
• pkg/parser/schemas/mcp_config_schema.json (184 lines)

Documentation Files (To Update)

• docs/src/content/docs/reference/frontmatter.md
• docs/src/content/docs/reference/frontmatter-full.md
• docs/src/content/docs/reference/included-files.md (if exists)

Parser Files (For Context)

• pkg/parser/frontmatter.go
• pkg/workflow/compiler.go
• pkg/workflow/engine.go
• pkg/workflow/safe_outputs.go

---

Next Steps

1. ✅ Review findings with maintainers
2. ✅ Prioritize type mismatch fixes (high impact, low risk)
3. ✅ Document intentional design decisions (engine fields, firewall)
4. ✅ Add schema consistency tests to CI
5. ✅ Update documentation with merge behaviors and feature availability

Conclusion

This analysis revealed 4 critical type and field mismatches that should be addressed soon, plus 2 moderate documentation gaps that need clarification. The issues are straightforward to fix (mostly type updates) and will improve schema consistency significantly.

The most urgent fixes are the version type mismatches which cause runtime behavior to differ from validation, leading to confusing errors when users use numeric versions in included files or standalone MCP configs.

AI generated by Schema Consistency Checker

[github-actions[bot]]

⚓ Avast! This discussion be marked as outdated by Schema Consistency Checker.
🗺️ A newer treasure map awaits ye at Discussion #5991.
Fair winds, matey! 🏴‍☠️