From a10c0186947aba3729411054ab390deb385a69b6 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 17 Mar 2026 17:18:47 +0100 Subject: [PATCH 01/25] docs: add validation framework requirements draft MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Initial draft of the CAMARA Validation Framework requirements document covering use cases, validation contexts, rule architecture, bundling, caller workflow design, and rollout strategy. Work in progress — Sessions 5-6 (integration points, consolidation) still pending. --- ...AMARA-Validation-Framework-Requirements.md | 1002 +++++++++++++++++ 1 file changed, 1002 insertions(+) create mode 100644 documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md new file mode 100644 index 0000000..263f49c --- /dev/null +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md @@ -0,0 +1,1002 @@ +# Validation Framework — Requirements (Internal Draft) + +**Status**: Work in progress +**Last updated**: 2026-03-17 + +--- + +## 1. Use Cases + +### 1.1 Contributor (including dev agents) + +UC-01: Check a fix/feature branch on a fork before creating a PR (via GitHub workflow dispatch). + +UC-02: See validation results of a PR check in an understandable form, with hints how to resolve failures. + +UC-03: Run most validation rules locally (on local clones, via scripts). + +UC-04: Check the code base before starting work, to be aware of existing issues (relevant for dev agents and new contributors). + +### 1.2 Codeowner + +UC-05: Rely on PR checks — if green, the branch fulfills formal requirements for the intended release as defined in release-plan.yaml. + +UC-06: Check at any time if the main branch is ready to be released, or which issues remain open (via workflow dispatch on main). + +UC-07: See the consequences of a release-plan.yaml change — a PR that only changes release-plan.yaml triggers a re-run, surfacing newly relevant rules. + +### 1.3 Release Automation + +UC-08: Use validation as a strict gate before creating a release snapshot. + +UC-09: Use validation to check the release review PR (strict, also restricting allowed changes to README and CHANGELOG). + +### 1.4 Rule Developer + +UC-10: Run rule changes against existing release branches as regression tests or to assess impact. + +UC-11: Define rule applicability conditions and conditional severity based on context (see section 2.2). + +UC-12: Define fix hints for failed checks. + +### 1.5 CAMARA Admin + +UC-13: Introduce the validation framework incrementally, with central configuration of enabled functionality per repository (no per-repo config files that create change noise). + +UC-14: Validate that an API repository is correctly configured for the validation framework. + +UC-15: Test the validation framework on feature branches with pinned refs (same mechanism as release automation). + +### 1.6 Release Manager (independent) + +UC-16: See a dashboard indicating compliance of API repository main branches against their declared intent. *(Independent work — Release Progress Tracker can collect last validation run on merged PR or main branch, trigger validation as camara-release-automation bot.)* + +UC-17: Trigger a validation check on selected repositories to update dashboard status. *(Independent — requires validation to be operational but does not depend on specific framework version.)* + +--- + +## 2. Validation Contexts + +### 2.1 Validation Profiles + +Three profiles control blocking behavior. The framework selects the profile automatically based on how validation is invoked. + +| Profile | Blocking behavior | Typical trigger | +|---------|-------------------|-----------------| +| **advisory** | Nothing blocks; all findings shown | Local run, dispatch | +| **standard** | `error` blocks; `warn` and `hint` shown | PR (fork-to-upstream or upstream branch) | +| **strict** | `error` and `warn` block; `hint` shown | Release automation gates (snapshot, release review PR) | + +### 2.2 Rule Model + +Each rule defines two things: + +1. **Applicability conditions** — when the rule fires at all +2. **Conditional level** — what severity it produces, which can vary by context + +The profile (section 2.1) then determines which levels block. This separates three concerns cleanly: + +- **Rule developer** controls: when a rule applies and what level it produces +- **Framework** controls: which profile is active +- **Profile** controls: which levels block + +#### Levels + +| Level | Meaning | Blocked by profile | +|-------|---------|-------------------| +| **error** | Must be fixed | standard, strict | +| **warn** | Should be fixed | strict | +| **hint** | Recommendation, never blocking | *(none)* | +| **off** | Suppressed, finding not shown | *(n/a)* | + +#### Rule metadata + +Rule metadata is expressed in YAML. Fields that are not constrained are omitted (omitted = applies in all contexts for that dimension). The primary use of the metadata is **post-filter and severity mapping**: engines run and produce findings, then the framework applies applicability and conditional level to interpret the results in the current context. + +```yaml +id: "042" # flat sequential ID, stable across engine changes +name: path-kebab-case # human-readable name +engine: spectral # spectral | yamllint | gherkin | python | manual +engine_rule: "camara-parameter-casing-convention" # native engine rule ID (if applicable) +hint: "Use kebab-case for all path segments: /my-resource/{resourceId}" + +applicability: # only list fields that constrain; omitted = no constraint + branch_types: [main, release] + trigger_types: [pr, dispatch] + # ... further conditions as needed + +conditional_level: + default: error # always present + overrides: # only if level varies by context + - condition: + target_release_type: [pre-release-alpha] + level: hint +``` + +#### Applicability conditions + +A rule is skipped silently if its conditions don't match the current context. Multiple fields are combined with AND; multiple values within an array field are combined with OR. + +| Field | Type | Values | Source | +|-------|------|--------|--------| +| `branch_types` | array | `main`, `release`, `maintenance`, `feature` | Target branch (PR) or checked branch (dispatch) | +| `trigger_types` | array | `pr`, `dispatch`, `release-automation` | How validation was invoked | +| `target_release_type` | array | `none`, `pre-release-alpha`, `pre-release-rc`, `public-release`, `maintenance-release` | `repository.target_release_type` in release-plan.yaml | +| `target_api_status` | array | `draft`, `alpha`, `rc`, `public` | `apis[].target_api_status` in release-plan.yaml (per-API) | +| `target_api_maturity` | array | `initial` (0.x.y), `stable` (x.y.z, x>=1) | Derived from `apis[].target_api_version` | +| `api_pattern` | array | `request-response`, `implicit-subscription`, `explicit-subscription` | Detected from OpenAPI spec content (per-API) | +| `commonalities_release` | string | Range expression, e.g. `">=r3.4"` | `dependencies.commonalities_release` in release-plan.yaml | +| `release_plan_changed` | boolean | `true`, `false` | Whether release-plan.yaml is in the PR diff (PR trigger only) | + +Range comparison for `commonalities_release` uses `packaging.specifiers` (Python) with release tags normalized to comparable values. + +#### Conditional level + +`default` is always present. `overrides` is a list of `{condition, level}` pairs evaluated in order; first match wins. Conditions use the same field/value model as applicability (AND across fields, OR within arrays). The level `off` can be used in overrides to suppress a finding in specific contexts. + +Examples: + +```yaml +# "Test definition must be present" — hint by default, warn for RC/public of stable APIs +conditional_level: + default: hint + overrides: + - condition: + target_api_maturity: [stable] + target_release_type: [pre-release-rc, public-release] + level: warn + +# "Commonalities compliance check" — warn by default, suppressed for draft APIs +conditional_level: + default: warn + overrides: + - condition: + target_api_status: [draft] + level: off +``` + +#### Execution context + +The framework constructs a context object at runtime from the trigger, branch, and release-plan.yaml content. Rules are evaluated per-API for API-specific checks (a repository with three APIs at different statuses produces three evaluation contexts). + +| Context field | Type | Source | +|---------------|------|--------| +| `branch_type` | string | Derived from branch name | +| `trigger_type` | string | Workflow event | +| `release_plan_changed` | boolean | PR diff (PR trigger only) | +| `target_release_type` | string | release-plan.yaml `repository.target_release_type` | +| `commonalities_release` | string | release-plan.yaml `dependencies.commonalities_release` | +| `icm_release` | string | release-plan.yaml `dependencies.identity_consent_management_release` | +| `target_api_status` | string | release-plan.yaml `apis[].target_api_status` (per-API) | +| `target_api_maturity` | string | Derived from `apis[].target_api_version` (per-API) | +| `api_pattern` | string | Detected from OpenAPI spec content (per-API) | +| `is_release_review_pr` | boolean | Detected by framework (used for profile selection, not rule applicability) | + +`is_release_review_pr` is used at the framework level to select the strict profile. It is not a rule applicability condition. + +### 2.3 Execution Contexts + +The validation framework must support these execution contexts: + +| Context | Trigger | Profile | Token | Notes | +|---------|---------|---------|-------|-------| +| **PR (fork-to-upstream)** | `pull_request` event | standard | read-only | Default GitHub behavior: fork PRs get read-only GITHUB_TOKEN regardless of author's repo permissions. Limits output options (no check run annotations, no PR comments via token) | +| **PR (upstream branch)** | `pull_request` event | standard | write | PR from upstream branch. Note: codeowners should normally use forks too | +| **Dispatch (upstream repo)** | `workflow_dispatch` | advisory | write | Main (default), maintenance, release branches. Warning on non-standard branches | +| **Dispatch (fork repo)** | `workflow_dispatch` | advisory | write (fork scope) | Fork owner triggers on own fork. Inherited — no extra work if dispatch trigger exists | +| **Local** | CLI / script | advisory | n/a | No GitHub context; subset of rules | +| **Release automation: snapshot** | Called by release workflow | strict | write (app token) | Gate before snapshot creation | +| **Release automation: review PR** | `pull_request` event (push to PR branch) | strict | write or read-only | Same trigger as normal PR; profile is strict based on release review PR detection | + +**Profile is independent of token permissions.** A fork-to-upstream PR gets the same **standard** profile as an upstream-branch PR. The read-only token only limits *how results are surfaced* (e.g. no check run annotations via GITHUB_TOKEN), not validation strictness. Alternative output paths for read-only contexts (workflow summary, `pull_request_target`, bot token) are a Session 4 topic. + +**Dispatch is always advisory.** Dispatch runs have nothing to block (except the workflow itself). They surface errors, warnings, and hints for the user to review. + +**Who can dispatch**: Standard GitHub permission model — write access required. On upstream: codeowners and org-wide teams (admin, release_reviewer). On forks: fork owner. + +--- + +## 3. MVP Scope + +The MVP replaces `pr_validation` v0 and delivers the minimum useful validation on PRs and dispatch. + +### MVP includes + +- UC-01 through UC-07 (contributor and codeowner use cases) +- UC-13 (incremental rollout with central config) +- UC-15 (feature branch testing with pinned refs) +- Profiles: advisory and standard +- Execution contexts: PR (fork-to-upstream), PR (upstream branch), dispatch (upstream repo), dispatch (fork repo) +- Existing Spectral rules and YAML linting +- Understandable output with fix hints +- Caller workflow with all triggers from day one (PR, dispatch), deployed alongside `pr_validation` v0 +- Central enable/disable per repository (no per-repo config files) + +### Post-MVP priorities + +- UC-08, UC-09 (release automation strict gates) — high priority, requires strict profile; depends on PR integration being operational first +- UC-10 (regression testing against release branches) — rule developer tooling +- UC-14 (repository configuration validation) — admin tooling +- Automated cache synchronization for `code/common/` and strict version enforcement — bundling itself (ref resolution via `$ref`) is MVP scope (see section 7.3) +- Python-based consistency checks beyond what Spectral covers — Session 2 scope, may partially land in MVP + +### Independent work (not sequenced with MVP) + +- UC-16, UC-17 (dashboard) — Release Progress Tracker can collect last validation run results and trigger validation with appropriate permissions + +### MVP rollout model + +1. Deploy caller workflow with all triggers to repositories alongside `pr_validation` v0; reusable workflow starts as no-op for most triggers +2. Enable advisory checks on dispatch +3. Enable standard checks on PRs +4. Enable blocking via GitHub rulesets (errors only) +5. Expand repository set incrementally via central include list + +--- + +## 4. Check Inventory + +### 4.1 Authoritative Sources + +The validation rules are derived from the following upstream documents. Rules must be validated against the version of these documents matching the declared `commonalities_release` in release-plan.yaml. + +**Commonalities** (version-dependent — r3.4 = v0.6.x, r4.1/r4.2 = v0.7.x): +- `CAMARA-API-Design-Guide.md` — API design rules, versioning, error handling, naming conventions +- `CAMARA-API-Event-Subscription-and-Notification-Guide.md` — subscription and notification patterns + +**Release Management:** +- `release-plan-schema.yaml` — field definitions and allowed values for release-plan.yaml +- `release-metadata-schema.yaml` — field definitions for generated release-metadata.yaml on release branches +- `api-readiness-checklist.md` — release asset requirements matrix by API status and maturity + +### 4.2 Check Engines + +| Engine | Scope | Current status | +|--------|-------|----------------| +| **Spectral** | OpenAPI structural and naming rules against `code/API_definitions/*.yaml` | Active in `pr_validation` v0 (~47 rules) | +| **yamllint** | YAML formatting for `code/API_definitions/*.yaml` | Active in `pr_validation` v0 (via MegaLinter) | +| **gherkin-lint** | Test file structure for `code/Test_definitions/*.feature` | Active in `pr_validation` v0 (via MegaLinter) | +| **Python** | Cross-file consistency, release-plan validation, context-dependent checks | Partially active (release-plan validation only); api-review v0.6 deprecated | +| **manual** | Checks too semantic to automate; framework provides prompts for human/AI review | Not yet implemented | + +### 4.3 Implementation Classification + +Each check falls into one of five categories: + +| Category | Description | +|----------|-------------| +| **spectral** | Implementable as a Spectral rule (single-file OpenAPI checks, pattern matching, schema structure) | +| **python** | Requires Python implementation (cross-file checks, release-plan awareness, file existence, version consistency) | +| **manual** | Too semantic to fully automate (e.g., data minimization, meaningful descriptions). Framework generates a prompt or checklist | +| **obsolete** | Handled by release automation — validation framework should not re-check (e.g., readiness checklist file management, release tag creation, version field replacement on release branches) | +| **existing** | Already implemented in current Spectral/yamllint/gherkin rules. Severity mapping to be determined when individual rules are classified | + +### 4.4 Check Inventory Status + +The per-rule inventory is **not yet complete**. The following work is required: + +1. **Commonalities audit** (dependency): Examine `CAMARA-API-Design-Guide.md` and `CAMARA-API-Event-Subscription-and-Notification-Guide.md` at both r3.4 and r4.1/r4.2 versions to: + - Identify checks not yet covered by any engine + - Validate existing Spectral rules against the current design guide + - Identify rules that changed between Commonalities releases + +2. **Existing rule classification**: Map each current Spectral rule to the framework metadata model (applicability, conditional level, hints). The existing Spectral severity levels are assumed valid for now; detailed severity review is deferred. + +3. **api-review v0.6 coverage**: The deprecated `api_review_validator_v0_6.py` (~43 checks) was a monolithic implementation used manually by release reviewers for the Fall25 meta-release. Its checks serve as input for identifying Python-needed validations not covered by Spectral. + +### 4.5 Known Check Areas by Engine + +Summary of check areas identified so far, pending the Commonalities audit for completeness: + +**Spectral (existing + new):** +- OpenAPI version enforcement (3.0.3) +- Naming conventions (paths: kebab-case, schemas: PascalCase, operationId: camelCase) +- Required descriptions (operations, parameters, responses, properties) +- Reserved words detection +- Security: no secrets in path/query parameters +- HTTP method validity, no request body on GET/DELETE +- Unused components detection +- *New*: info.version format (wip/alpha.n/rc.n/public), XCorrelator pattern, phone number format, device object structure + +**Python (new):** +- Server URL version consistency with info.version (cross-field) +- Version must be wip on main, must not be wip on release branches (context-dependent) +- release-plan.yaml non-exclusivity check (PR diff analysis) +- release-plan.yaml schema and semantic validation (existing, to be integrated) +- Error response structure and code compliance (cross-schema) +- Test file existence and version alignment (cross-file) +- CHANGELOG format and link tag-locking (file content analysis) +- API pattern-specific checks: subscription endpoints, CloudEvents format, event type naming (structural + semantic) + +**Manual + prompt:** +- Data minimization compliance (GEN-009, GEN-010) +- Meaningful description quality (beyond presence checks) +- User story adequacy +- Breaking change justification + +**Obsolete (handled by release automation):** +- API Readiness Checklist file management (files should no longer be in the repository) +- Release tag creation and format +- Version field replacement on release branches (wip → actual version) +- release-metadata.yaml generation +- README update with release information + +--- + +## 5. Rule Architecture + +### 5.1 Overview + +The framework uses a **post-filter and severity mapping** architecture: + +1. Framework determines the execution context (branch type, trigger, release-plan.yaml content) +2. Engines run (Spectral, yamllint, gherkin-lint, Python checks) and produce findings +3. Framework collects all findings +4. For each finding: look up rule metadata → is it applicable in this context? → what level? → apply profile (advisory/standard/strict) to determine blocking +5. Produce unified output with hints + +Individual checks may use the context object as a pre-condition for expensive operations (e.g., release-plan validation only runs if `release_plan_changed` is true on PR trigger, or always on `release-automation` trigger). This is a performance optimization, not a generalized orchestration model. + +### 5.2 Rule Metadata Model + +See section 2.2 for the full metadata model. Key design decisions: + +- **Flat ID namespace**: Sequential IDs (e.g., `042`) that are stable across engine changes. A rule migrating from Python to Spectral keeps its ID. +- **Engine as metadata**: The `engine` field records which engine implements the check. The `engine_rule` field maps to the native rule ID (e.g., Spectral rule name) for configuration synchronization. +- **Condition language**: Plain YAML with AND across fields, OR within arrays. Range comparison on `commonalities_release` uses `packaging.specifiers` (Python). No custom DSL or policy engine. +- **Per-API evaluation**: API-specific rules are evaluated once per API in the repository, using the API's own `target_api_status`, `target_api_maturity`, and `api_pattern` from release-plan.yaml and OpenAPI spec content. + +### 5.3 Spectral Pass-Through Principle + +The framework uses Spectral's severity names (`error`, `warn`, `hint`) as its native level values. This gives identity mapping for the primary engine: + +| Spectral | Framework | Notes | +|----------|-----------|-------| +| `error` | `error` | Identity | +| `warn` | `warn` | Identity | +| `hint` | `hint` | Identity | +| `info` | `hint` | Mapped (rarely used) | +| `off` | `off` | Identity (disable rule) | + +Spectral rules already include `message` fields with fix guidance. Therefore, **Spectral rules that do not need context-dependent severity or applicability filtering do not require explicit framework metadata entries**. Their findings pass through with direct severity mapping and native messages. + +Framework metadata is only needed for Spectral rules when: +- The level should change based on context (e.g., error on release branch, hint on feature branch) +- The rule should be suppressed in certain contexts (applicability filtering) +- The fix hint should be overridden or augmented + +This minimizes the metadata surface: only rules with context-dependent behavior need explicit entries. + +The framework consumes Spectral output as structured data (JSON), not terminal text. This enables programmatic post-filtering, severity remapping, and merging with findings from other engines. + +Spectral does not resolve `$ref` references before linting — it validates the document as-is. Checks that depend on the content of referenced schemas (e.g., from CAMARA_common.yaml) require either a pre-bundled input spec or a Python implementation with explicit ref resolution. See Session 3 (Bundling & Refs) for implications. + +For further Spectral-specific implementation details, see [spectral-integration-notes.md](../reviews/spectral-integration-notes.md). + +### 5.4 Condition Evaluation + +``` +applicability match: + for each field in rule.applicability: + if field is array: context value must be IN the array (OR) + if field is range string: context value must satisfy the range expression + if field is boolean: context value must equal the field value + all fields must match (AND) + omitted fields are unconstrained (always match) + +conditional level: + for each override in rule.conditional_level.overrides (in order): + if override.condition matches context (same logic as applicability): + return override.level + return rule.conditional_level.default +``` + +### 5.5 Derived Context Fields + +Two per-API context fields are derived from content rather than declared in release-plan.yaml: + +**`target_api_maturity`**: Derived from `apis[].target_api_version` — `initial` if major version is 0 (v0.x.y), `stable` if major version >= 1 (vx.y.z). Determines which asset requirements apply per the API Readiness Checklist (e.g., stable public APIs require enhanced test cases and user stories). + +**`api_pattern`**: Detected from OpenAPI spec content — `request-response`, `implicit-subscription`, or `explicit-subscription`. Detection logic examines paths (subscription endpoints), callbacks, schema names, and content types. Multiple pattern-specific rule sets (REQ, IMP, EXP, EVT categories) depend on this classification. This detection is a cross-cutting capability used by many rules. + +### 5.6 Spectral Migration Potential + +Analysis of the deprecated `api_review_validator_v0_6.py` shows that approximately 40% of its checks are implementable as Spectral rules (single-file OpenAPI pattern matching), and an additional 15% could use Spectral custom JavaScript functions. This includes: + +- Mandatory error response checks (400, 401, 403) +- Server URL format validation +- info.version format validation +- License and security scheme validation +- ErrorInfo and XCorrelator schema presence +- Error response structure validation + +The main blocker for migrating ~20% of checks to Spectral is the dependency on `api_pattern` detection — Spectral cannot natively apply rules conditionally based on detected API type. These checks either need custom JS functions that embed the detection logic, or remain as Python checks that use `api_pattern` from the context. + +The Commonalities audit (OQ-05) should evaluate each candidate check against the current design guide version before migration. + +### 5.7 Authoritative Schema References + +The execution context fields and their allowed values are defined by the following schemas, which are the authoritative sources: + +- **release-plan.yaml**: `upstream/traversals/ReleaseManagement/artifacts/metadata-schemas/schemas/release-plan-schema.yaml` +- **release-metadata.yaml**: `upstream/traversals/ReleaseManagement/artifacts/metadata-schemas/schemas/release-metadata-schema.yaml` + +The framework must accept exactly the values defined in these schemas. Any change to the schemas must be reflected in the framework's context model. + +--- + +## 6. Open Questions + +OQ-01: *(Resolved — fork dispatch is inherited automatically if the caller workflow supports dispatch.)* + +OQ-02: *(Resolved — bundling for `CAMARA_common.yaml` via `$ref` is within MVP scope. Automated cache sync and strict version enforcement are post-MVP. See section 7.3.)* + +OQ-03: What is the minimum set of checks for MVP? Just current Spectral + YAML lint, or also a subset of Python checks? *(Partially addressed by Session 2: the Commonalities audit will identify which checks are needed and which are feasible without bundling.)* + +OQ-04: *(Resolved — layered token resolution: validation app bot as default, GITHUB_TOKEN write as fallback, read-only as last resort. Dedicated validation GitHub App separate from camara-release-automation. `pull_request_target` rejected due to security risk at scale. See section 9.1.)* + +OQ-05: Commonalities audit — per-rule validation of existing Spectral rules and identification of missing checks against `CAMARA-API-Design-Guide.md` and `CAMARA-API-Event-Subscription-and-Notification-Guide.md` at r3.4 and r4.1/r4.2 versions. Required before the check inventory can be completed. + +OQ-06: Sub-project common dependency declaration format — should sub-project dependencies (repository, release tag, array of files) be declared within `release-plan.yaml` or as a separate manifest? See section 7.4. + +OQ-07: API-aware change summary tooling — evaluate oasdiff or alternatives for semantic OpenAPI diff summaries. Post-MVP. See section 8.6. + +OQ-08: OIDC token availability for fork PRs — does `id-token: write` work when a fork PR triggers the reusable workflow? If not, the hardcoded version fallback (section 9.5) handles ref resolution, but the exact OIDC behavior should be confirmed empirically as part of WP-01 validation. + +--- + +## 7. Bundling & Refs + +The overall bundling model — controlled local copy on `main`, source-only `main`, bundled release artifacts — is defined in the [Commonalities Consumption and Bundling Design](https://github.com/camaraproject/ReleaseManagement/pull/436) document. This section captures what the validation framework checks about that model, not the model itself. + +### 7.1 Repository Model Validation + +The framework validates the repository layout and `$ref` patterns used in API source files. + +#### Layout validation + +When any API source file in `code/API_definitions/` contains a `$ref` to `../common/` or `../modules/`, the framework validates: + +- The referenced file exists at the expected path +- `code/common/` contains only cache copies from other repositories (e.g., `CAMARA_common.yaml` from Commonalities). These files are CI-managed, not manually edited. Content validation is covered in section 7.4. +- `code/modules/` contains project-local reusable schemas used to make the API source more readable and structured. No sync validation is needed for these files. + +#### Ref pattern validation + +- API source files must use only relative `$ref` for normative schema consumption — into `code/common/`, `code/modules/`, or same-directory references +- Remote URL-based `$ref` (e.g., `https://raw.githubusercontent.com/...`) used for normative schema content is an error +- Internal component references (`#/components/...`) are always valid + +#### Transition handling + +Repositories not yet using `code/common/` (i.e., the current copy-paste model with all schemas inline) remain valid. The layout and ref pattern checks only fire when `$ref` to `../common/` or `../modules/` is detected in source files. The framework does not force migration to the local copy model. + +### 7.2 Source vs Bundled Validation + +The framework applies a uniform validation flow regardless of context: + +1. **Pre-bundling validation** (always runs on source files): + - YAML validity (yamllint) + - Ref existence and pattern validation (section 7.1 — do referenced files exist? are refs relative and canonical?) + - Release-plan.yaml consistency checks (Python) + - Cross-file checks that do not depend on schema content (file existence, version field presence) + +2. **Bundling** (if `$ref` to `code/common/` or `code/modules/` is detected): + Resolve all local refs and produce a standalone spec per API. The framework invokes an external bundling tool — it does not implement its own OpenAPI bundler/dereferencer. Tool choice is an implementation detail. + +3. **Full validation** (runs on the *effective input* — bundled output when refs are present, source directly when no refs): + - Full Spectral ruleset — all schemas are inline/resolved + - Python checks that depend on schema content + - Standalone API spec validation + +4. **Artifact surfacing**: Upload bundled specs as workflow artifacts or make them available for further processing (see section 8) + +This flow applies to all contexts — PR, dispatch on any branch type, release automation. The validation **profile** (advisory, standard, strict) controls which findings block; it does not change which steps run. + +#### Bundling failure + +If the bundling step fails (e.g., unresolvable `$ref`, missing file), the framework reports the failure as an error finding. Full validation (step 3) is skipped; only pre-bundling results are available. + +#### Repositories without `$ref` + +Repositories using the current copy-paste model (all schemas inline) skip step 2. Source files are standalone by construction, so the full Spectral ruleset runs directly on source in step 3. No bundling overhead is introduced for repositories that have not adopted the local copy model. + +### 7.3 Spectral and `$ref` Interaction + +#### Bundling vs full dereferencing + +The framework uses **bundling** (external ref resolution only), not full dereferencing: + +- **Bundling**: Resolves external `$ref` — pulls content from `code/common/`, `code/modules/`, and other local files into the document. Internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`) are preserved. +- **Full dereferencing**: Resolves all `$ref` including internal ones, producing a flat document with zero `$ref` and massive duplication. The framework must **not** use full dereferencing. + +Preserving internal `$ref` ensures that: + +- Spectral rules checking component structure, `$ref` patterns, and `#/components/` organization continue to work on bundled output +- Bundled output remains readable and structurally equivalent to what reviewers expect +- No Spectral rule changes are needed between copy-paste and bundled models + +Any constraints on where API designers may use external vs internal `$ref` are defined in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436), not by the validation framework. The framework enforces whatever ref patterns the design document specifies. + +#### Transition period + +During migration from copy-paste to the local copy model, both repository types coexist: + +- **Copy-paste repos**: All schemas inline. Spectral runs directly on source (step 3 of section 7.2). No bundling needed. +- **`$ref` repos**: Spectral runs on bundled output. All external refs resolved, internal refs preserved. Structurally equivalent to copy-paste. + +No rule changes are needed between the two models — bundling normalizes external refs while preserving the internal structure that Spectral rules depend on. Rule IDs remain stable across the transition (flat namespace from section 5.2). + +#### OQ-02 resolution — bundling is MVP scope + +Bundling support for `CAMARA_common.yaml` via `$ref` is **within MVP scope**. Commonalities 0.7.x requires updated common schemas, and the ability to consume them via `$ref` — with the framework handling bundling transparently — is the key additional value of the validation framework v1 for codeowners. This avoids repeating the difficult-to-validate copy-paste pattern. + +In the MVP, some parts may still be manual — providing the correct copy in `code/common/` and ensuring it matches the declared `commonalities_release` version. But the `$ref` option is available for early adopters, and the framework handles bundling when `$ref` is detected. Automated cache synchronization and strict version enforcement are post-MVP enhancements. + +### 7.4 Cache Synchronization and Dependency File Mapping + +#### Cache sync validation + +When `code/common/` exists, the framework validates that cached files match the declared dependency versions in `release-plan.yaml`. Mismatch severity depends on the profile: + +- **standard** (PR): warning — codeowner is informed, merge is not blocked +- **strict** (release automation): error — snapshot creation is blocked + +If no `code/common/` directory exists (repo has not adopted the local copy model), the sync check is skipped. + +In the MVP, cache management may be manual. The validation check still applies — it reports whether the current cache matches the declared dependency, regardless of how the cache was populated. + +#### Dependency categories + +Three categories of shared schema dependencies exist, each with different characteristics: + +**Commonalities** (well-known, hardcoded in tooling): +The Commonalities repository provides shared schemas that are well-known to automation tooling. Currently two files are relevant: +- `CAMARA_common.yaml` — common data types, error responses, headers +- `notification-as-cloud-event.yaml` — CloudEvents notification schema + +These files, their source location in the Commonalities repository, and the mapping from `release-plan.yaml.dependencies.commonalities_release` to the correct version are built into the tooling. No per-repository configuration is needed. + +**ICM** (version compatibility constraint): +Identity and Consent Management schemas are currently contained within Commonalities files — there are no separate ICM files to cache. The `dependencies.identity_consent_management_release` in `release-plan.yaml` is a version compatibility constraint (potentially `>= x.y.z`) rather than a file-caching relationship. The exact nature of this dependency requires further discussion. + +**Sub-project commons** (extensible, declared per repository): +Sub-projects may define common schemas shared across their API repositories (e.g., a device API family sharing common device type definitions). These dependencies must be declarable without requiring changes to the automation tooling. Each sub-project dependency requires: +- Source repository +- Release tag or version +- Array of files to consume + +This extensible model requires a dependency declaration format, either within `release-plan.yaml` or as a separate manifest. The schema design is a follow-up topic for the bundling design document. + +#### File caching strategy + +Which files are cached in `code/common/` — demand-driven (only files actually `$ref`'d) vs declaration-driven (all files from declared dependencies) — is a sync mechanism concern defined in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436), not a validation framework decision. + +The framework's checks are the same regardless: cached files must match their declared source version, and `$ref` targets must exist. + +### 7.5 Commonalities Version Matrix + +#### Active versions + +The framework must support validation rules that vary by Commonalities version. Active versions at the time of writing: + +- **r3.4** (Commonalities v0.6.x) — Fall25 meta-release, frozen, maintenance releases only +- **r4.x** (Commonalities v0.7.x) — Spring26 meta-release. r4.1 is the release candidate (available now); r4.2 is the upcoming public release and will replace r4.1 + +Within a version line (r4.x), the latest release is always authoritative. When r4.2 is available, r4.1 becomes obsolete — new releases must target r4.2. If a maintenance release r4.3 follows, it replaces r4.2 for validation purposes. + +Future Commonalities major versions (e.g., r5.x for v1.0.0) will add further version lines. The architecture must not assume a fixed number of active versions. + +#### Spectral ruleset selection (pre-selection) + +Each Commonalities major version line gets its own Spectral ruleset (e.g., `.spectral-r3.4.yaml`, `.spectral-r4.yaml`). The framework reads `commonalities_release` from `release-plan.yaml` and selects the matching ruleset before running Spectral. + +This avoids running contradicting rules from different Commonalities versions simultaneously, which would produce confusing Spectral output even if the results were filtered afterwards. The r3.4 ruleset is effectively frozen — only maintenance fixes. New rule development targets the current r4.x ruleset. + +#### Framework rule metadata (post-filter with conditionals) + +Framework rule metadata uses a single ruleset with `commonalities_release` range conditions (defined in section 2.2) for version-specific behavior. This is appropriate because: + +- Python checks are framework-controlled and do not produce confusing intermediate output +- Most framework rules apply across versions; only a minority are version-specific +- Duplicating shared rules into per-version files would create drift risk + +The Commonalities audit (OQ-05) will identify which rules changed between r3.4 and r4.x. Those rules receive `commonalities_release` range conditions in their metadata. + +### 7.6 Placeholder Handling + +#### Current state + +The current `CAMARA_common.yaml` contains placeholder patterns (e.g., `{{SPECIFIC_CODE}}`) that have no defined resolution rules. These should be removed from Commonalities, with API repositories extending shared schemas via `allOf` instead (per the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436)). + +#### Future direction + +Placeholder replacement with defined values could be introduced together with bundling as part of a broader transformation pipeline. This could include dynamic variables such as `api_version`, `commonalities_release`, `commonalities_version`, effectively replacing the current "wip" and "/main/" substitutions done by the snapshot transformer. In this model, bundling + transformation (including placeholder replacement) would produce the release-ready artifact. + +#### Framework requirements + +- The framework detects unreplaced placeholder patterns (`{{...}}`) in bundled output and reports them as errors — this is a permanent safety net +- As of now, placeholder replacement is not a framework responsibility. If placeholder replacement becomes part of the bundling/transformation pipeline, it is a post-MVP enhancement +- Detection of undefined/unresolvable placeholders remains an error regardless of whether replacement is implemented + +### 7.7 Rule Architecture Integration + +Bundling integrates into the rule architecture (section 5) without requiring changes to the context model or rule metadata: + +- **Step assignment**: Each rule runs in either step 1 (pre-bundling validation) or step 3 (full validation). Assignment is an implementation detail — the framework knows which checks belong to which step. +- **No new context fields**: The context model from section 2.2 is sufficient. Whether external refs existed and were resolved is an implementation concern, not a rule applicability condition. +- **Cache sync is a check, not context**: The cache synchronization validation (section 7.4) produces findings (warning or error depending on profile). It is not a context field consumed by other rules. +- **Spectral ruleset selection**: The `commonalities_release` field (already in the context model) drives Spectral ruleset pre-selection (section 7.5). No additional metadata is needed. + +#### Relationship to snapshot creation + +How bundling and validation interact with release automation's snapshot branch creation — whether the framework produces the bundled content that becomes the snapshot, or validates an already-created snapshot — is a WS03 integration topic (Session 5). + +## 8. Artifact Surfacing + +This section covers how bundled artifacts are surfaced for reviewer visibility. Findings surfacing (how validation errors, warnings, and hints are presented) is a separate topic covered in Session 4 (OQ-04). + +### 8.1 PR Review Surfaces + +The [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436) defines a priority order for reviewer visibility. The framework produces: + +1. **Source diff** — primary review surface, no framework action needed (standard git diff) +2. **Bundled artifact** — the framework uploads the bundled standalone API spec as a GitHub workflow artifact for each API affected by the PR +3. **Bundled diff** — a diff between the bundled API from the PR base and the bundled API from the PR head, uploaded as a workflow artifact or included in the workflow summary +4. **API-aware summary** — optional semantic change summary (see section 8.5) + +### 8.2 Workflow Artifacts + +- Bundled specs are uploaded as GitHub workflow artifacts with a naming convention that identifies the API name, branch, and commit SHA +- Bundled files include a header comment: `# For information only - DO NOT EDIT` +- Workflow artifact retention uses the GitHub default (90 days) + +### 8.3 Line Number Mapping + +When checks run on bundled output (step 3 of section 7.2), findings report line numbers in the bundled file. These do not correspond to the source files where the issue should be fixed. The framework must map finding locations in the bundled output back to source file and line number, so that findings are actionable for contributors and codeowners. + +### 8.4 Temporary Branch Model + +Workflow artifacts replace the temporary branch model (`/tmp/bundled/-`) for MVP. Temporary branches may be revisited post-MVP if reviewers need a browsable view of bundled content. + +### 8.5 "wip" Version Handling + +Bundling on `main` leaves `info.version` as-is — it contains `wip` as expected for unreleased code. Version replacement on release branches is handled by release automation (snapshot transformer), not by the validation framework's bundling step. The framework validates version correctness per branch type — this is an existing check (section 4.5), not a new bundling-specific requirement. + +### 8.6 API-Aware Change Summaries + +The framework should support pluggable diff tools for generating semantic change summaries (breaking changes, new endpoints, modified schemas). oasdiff is a candidate implementation. This capability is post-MVP — source diff, bundled artifact, and bundled diff provide sufficient reviewer visibility for MVP. + +--- + +## 9. Caller Workflow Design + +### 9.1 Findings Surfacing + +#### Surfacing categories + +Two categories of output are surfaced to users: + +**A. Findings output** (validation errors, warnings, hints with fix guidance): + +| Surface | Description | Requires write token | +|---------|-------------|---------------------| +| **Workflow summary** | Structured markdown in the Actions run summary. Primary detailed surface with all findings, fix hints, and links to diagnostic artifacts | No | +| **Check run annotations** | Inline annotations in the PR diff, mapped to file and line. Findings appear directly where the issue is | Yes | +| **PR comment** | Concise summary on the PR: error/warn/hint counts, blocking status, link to full workflow summary | Yes | +| **Commit status** | Per-check context visible in the PR checks list (e.g., `--> Validate: release-plan.yaml`). Continuation of the v0 pattern | Yes | + +**B. Diagnostic artifacts** (log files, detailed engine output): + +| Artifact | Description | +|----------|-------------| +| **Spectral JSON log** | Full structured Spectral output for debugging rule behavior | +| **Engine reports** | Detailed per-engine output (replaces v0 MegaLinter report artifacts) | +| **Bundled spec artifacts** | Standalone bundled API specs (see section 8.2) | + +Diagnostic artifacts are always uploaded as GitHub workflow artifacts regardless of token permissions. They have no fork PR limitation. + +#### Workflow summary size limit + +GitHub limits workflow step summaries to 1 MB per step and 1 MB total per job. For repositories with many APIs or many findings, this limit may be exceeded. The framework must: + +- Truncate the summary when approaching the limit, showing a count ("50 of 127 findings shown") with a link to the full diagnostic artifact +- Prioritize errors over warnings over hints in the truncated view +- Always include the full findings in the Spectral JSON log artifact (no size limitation on workflow artifacts) + +#### Token resolution + +The framework uses a layered token resolution strategy. The order prioritizes consistent branding (all findings come from the same bot identity) over using whichever token happens to be available: + +1. **Snapshot context**: `camara-release-automation` app token — provided by the calling release workflow. The validation framework does not mint this token; it is passed in by the release automation caller. +2. **Validation default**: Dedicated validation app bot token — the framework mints an installation token for the current repository. This is the primary path for all PR and dispatch contexts, ensuring consistent bot identity on annotations and comments. +3. **Fallback**: `GITHUB_TOKEN` with write access — used when the validation app is not installed (e.g., dispatch in a fork, or repositories not yet onboarded to the app). Write capability is probed at runtime. +4. **Read-only**: Workflow summary and diagnostic artifacts only — when no write token is available. + +In normal operation, both upstream PRs and fork PRs show findings from the validation bot. The `GITHUB_TOKEN` fallback and read-only mode are degraded paths, not the expected default. + +#### Validation GitHub App + +A dedicated GitHub App handles write surfaces for validation. This is a **separate app** from `camara-release-automation` — it has a narrower permission scope and a different purpose. + +| Aspect | Validation App | camara-release-automation | +|--------|---------------|--------------------------| +| **Purpose** | PR annotations, comments, commit status | Release snapshot creation, branch management | +| **Permissions** | `checks: write`, `pull-requests: write`, `statuses: write` | `contents: write`, `workflows: write`, plus release management | +| **Commits/pushes** | Never | Yes (snapshot branches, tags, release assets) | +| **EasyCLA** | Not needed (no commits) | Required (commits to repos with CLA enforcement) | + +Org-level configuration: +- `vars.VALIDATION_APP_ID` — app ID (org variable) +- `secrets.VALIDATION_APP_PRIVATE_KEY` — app private key (org secret) + +The validation app is introduced from day one (MVP) to establish consistent bot identity and avoid caller workflow changes later. + +#### Surfaces by resolved capability + +| Token capability | Findings output | Diagnostic artifacts | +|-----------------|-----------------|---------------------| +| **Write** (validation app, camara-release-automation, or GITHUB_TOKEN) | Workflow summary + check run annotations + PR comment + commit status | Workflow artifacts (always) | +| **Read-only** (all write paths failed) | Workflow summary only | Workflow artifacts (always) | + +#### Relationship to v0 surfacing + +The v0 workflow surfaces findings via MegaLinter's built-in reporters (`GITHUB_COMMENT_REPORTER`, `GITHUB_STATUS_REPORTER`) and custom `actions/github-script` steps for release-plan validation. The v1 framework replaces all of these with its own unified surfacing layer. MegaLinter is no longer used as the orchestration layer. + +### 9.2 Trigger Design + +#### PR trigger + +```yaml +on: + pull_request: + branches: + - main + - release-snapshot/** + - maintenance/** +``` + +- **`main`**: Standard development PRs. Profile: standard (or strict if release review PR detected) +- **`release-snapshot/**`**: Release review PRs created by release automation on snapshot branches. Profile: strict +- **`maintenance/**`**: Maintenance branch PRs. Profile: standard + +Default event types (`opened`, `synchronize`, `reopened`) are sufficient. The framework validates code content, not PR metadata — `edited` (title/body changes) is not needed. + +#### Dispatch trigger + +```yaml + workflow_dispatch: +``` + +Dispatch runs on whatever branch the user selects in the GitHub UI. The framework derives branch type, release context, and all validation parameters from the checked-out branch content. Dispatch inputs are discussed in section 9.4. + +#### Concurrency + +```yaml +concurrency: + group: ${{ github.ref }}-${{ github.workflow }} + cancel-in-progress: true +``` + +Same model as v0: a new push to a PR branch cancels the previous validation run. Dispatch behaves identically — a second dispatch on the same branch cancels the first. + +### 9.3 Permissions + +The caller workflow declares the maximum permission set. The reusable workflow inherits these as a ceiling — it cannot elevate above what the caller declares. + +```yaml +permissions: + checks: write + pull-requests: write + issues: write + contents: read + statuses: write + id-token: write +``` + +| Permission | Purpose | Fork PR behavior | +|------------|---------|-----------------| +| `checks: write` | Check run annotations (findings inline in PR diff) | Restricted by GitHub; validation app token used instead | +| `pull-requests: write` | PR review interactions | Restricted by GitHub; validation app token used instead | +| `issues: write` | PR comments (PRs use the Issues API for comments) | Restricted by GitHub; validation app token used instead | +| `contents: read` | Repository checkout | Available (read-only) | +| `statuses: write` | Commit status (per-check context in checks list) | Restricted by GitHub; validation app token used instead | +| `id-token: write` | OIDC token for tooling ref resolution (WP-01 pattern) | May not be granted for fork PRs — see section 9.5 | + +For fork PRs, `GITHUB_TOKEN` write permissions are restricted by GitHub regardless of what the caller declares. The validation app token (section 9.1) bypasses this restriction because it is minted from the app's own credentials, independent of `GITHUB_TOKEN`. + +### 9.4 Input Design Principle + +The reusable workflow does not accept inputs that duplicate information derivable from the checked-out branch. This prevents contradictions where an input says one thing but branch content says another. + +**Example of the problem avoided**: If the workflow accepted a `release_type` input, a user could dispatch on `main` with `release_type: public-release` while `release-plan.yaml` on `main` says `target_release_type: pre-release-alpha`. The framework would need reconciliation logic, and the user would get confusing results. + +**The rule**: The framework reads branch type from the branch name, and all release context from `release-plan.yaml` on the checked-out branch. These values are derived at runtime, never accepted as inputs. + +**Forbidden inputs**: `branch_type`, `release_type`, `api_status`, `commonalities_version`, `configurations` — all derivable from branch content or from the central configuration file (section 10.2). + +**Consequence for the caller workflow**: No per-repo inputs exist. All per-repo configuration (linting config subfolder, enabled features, rollout stage) lives in the central config file read by the reusable workflow. The caller workflow is identical across all repositories, with no `with:` block needed in standard operation. This makes it protectable via CODEOWNERS or rulesets — nobody ever needs to edit it. + +### 9.5 Ref Resolution + +#### OIDC-based ref resolution (primary) + +The reusable workflow resolves its own tooling repository and commit SHA via OIDC claims (`job_workflow_sha`), following the pattern established in WP-01 (tooling#121). This ensures all internal checkouts (linting config, shared actions at runtime) use the same tooling version that the caller specified. + +The caller workflow declares `id-token: write` to enable OIDC token generation. + +#### Hardcoded version fallback + +If OIDC token generation fails (e.g., fork PRs where `id-token: write` may not be granted), the reusable workflow falls back to its own hardcoded version tag (e.g., `v1`). This is the same pattern as v0's hardcoded `ref: v0`. + +This fallback is acceptable because: +- **Fork PRs**: Contributors do not need pinned-SHA ref resolution — the release version tag (`v1`) is correct for production validation +- **Feature branch testing**: Done by admins and rule developers who have write access, so OIDC works +- **Release automation**: Always triggered by codeowners with write access, so OIDC works + +The fallback means fork PR validation always uses the published version of the tooling, not a feature branch. This is the expected behavior — only admins test unreleased tooling versions. + +#### Break-glass override + +`tooling_ref_override` — a 40-character SHA input to the reusable workflow. Takes precedence over both OIDC and the hardcoded fallback. Documented as pilot/break-glass only. Same mechanism as release automation. + +#### Resolution order + +1. `tooling_ref_override` input (if set) — explicit SHA, highest priority +2. OIDC `job_workflow_sha` claim (if `id-token` available) — exact commit SHA +3. Hardcoded version tag in the reusable workflow (e.g., `v1`) — always available + +### 9.6 Reusable Workflow Contract + +#### Inputs + +| Input | Type | Required | Default | Description | +|-------|------|----------|---------|-------------| +| `tooling_ref_override` | string | no | *(empty)* | 40-character SHA for non-standard tooling ref. Takes highest priority in ref resolution (section 9.5). Supports testing in contexts where OIDC may not work. Documented as pilot/break-glass only. | +| `profile` | choice | no | *(auto)* | Validation profile: `advisory`, `standard`, `strict`. Default: framework selects based on context — advisory for dispatch, standard for PR, strict for release review PR or release automation. Dispatch users can set explicitly to see what a different profile would flag. | + +No per-repo inputs. All per-repo configuration lives in the central config file (section 10.2). + +#### Caller workflow + +The caller workflow is identical across all repositories: + +```yaml +name: CAMARA Validation + +on: + pull_request: + branches: + - main + - release-snapshot/** + - maintenance/** + workflow_dispatch: + +concurrency: + group: ${{ github.ref }}-${{ github.workflow }} + cancel-in-progress: true + +permissions: + checks: write + pull-requests: write + issues: write + contents: read + statuses: write + id-token: write + +jobs: + validation: + uses: camaraproject/tooling/.github/workflows/validation.yml@v1 + secrets: inherit +``` + +No `with:` block in standard operation. The caller is a thin pass-through that provides triggers, permissions, and concurrency. All validation logic, configuration, and surfacing are handled by the reusable workflow. + +#### Version tagging + +The reusable workflow uses a floating version tag (`v1`) analogous to v0's `v0` tag. The tag is moved forward as the framework evolves within the v1 major version. Breaking changes (new required permissions, changed caller contract) require a new major version tag. + +#### Secrets + +The caller passes `secrets: inherit`. The reusable workflow uses: +- `GITHUB_TOKEN` — inherited, for checkout and fallback write surfaces +- Org secrets for validation app token minting (`VALIDATION_APP_PRIVATE_KEY`) — accessed via `secrets` context +- Org variables for app identity (`VALIDATION_APP_ID`) — accessed via `vars` context + +## 10. Rollout Strategy + +### 10.1 v0/v1 Coexistence + +The v0 caller (`pr_validation_caller.yml`) and v1 caller (new file) coexist in each repository during the transition period. Both run on every PR and appear in the PR checks list. + +#### Why separate callers + +- The v0 reusable workflow has a fundamentally different structure (MegaLinter-based, single job, different permissions and output model). A single caller with version switching would require complex conditional logic. +- Separate callers allow independent lifecycle: v0 can be removed per-repo after v1 is proven, without coordinating a simultaneous switch. +- GitHub rulesets can reference the v1 check name independently of v0. + +#### Lifecycle per repository + +1. **v1 deployed**: v1 caller added alongside v0. Both run on every PR. Codeowners see both check results. +2. **v1 validated**: Codeowners confirm v1 findings are correct and consistent with expectations. +3. **v1 required**: GitHub ruleset makes the v1 check blocking for PR merge. +4. **v0 removed**: v0 caller file deleted from the repository. Any v0-specific ruleset removed. + +The transition from stage 1 to stage 4 is per-repository. Repositories move through stages independently based on pilot experience and codeowner readiness. + +#### WP-01 relationship + +WP-01 (tooling#121) fixes ref consistency in the existing v0 reusable workflow. It validates the OIDC ref resolution pattern that v1 reuses and adds the `tooling_ref_override` break-glass input. WP-01 does not change the v0 caller — callers still call `@v0`. The v1 reusable workflow reuses the same ref resolution pattern with the hardcoded version fallback (section 9.5). + +### 10.2 Central Enable/Disable + +#### Mechanism: tooling config file + +A YAML configuration file in the tooling repository maps each API repository to its validation settings. The reusable workflow reads this file from its own checkout (which it already performs for linting configs and shared actions). + +**Rationale for config file over alternatives:** + +- **Org variable with repo list** (rejected): JSON arrays in org variables become unwieldy at 60+ repos and hit variable size limits. Not PR-reviewable. +- **Per-repo variable** (rejected): Requires touching each repository to enable. Violates UC-13 — central administration without per-repo configuration changes. +- **Caller version tag** (rejected): Would require editing the caller workflow per-repo, undermining the identical-caller-across-all-repos design (section 9.6). +- **Tooling config file** (chosen): Version-controlled, PR-reviewable, scalable. Adding a repo is one line in a YAML file. Can hold per-repo settings beyond enable/disable (linting config subfolder, rollout stage). Satisfies UC-13 — no per-repo config changes needed. + +The config file schema is an implementation detail. At minimum it maps repository name to rollout stage and per-repo settings (e.g., linting config subfolder previously handled by the `configurations` input in v0). + +#### Repositories not listed + +A repository not listed in the config file is treated as `disabled` — the reusable workflow exits immediately with a notice. This is the safe default for the 60+ repos that have the caller deployed but are not yet onboarded. + +### 10.3 Stage Model + +The config file controls the validation stage per repository: + +| Stage | Config value | Behavior | +|-------|-------------|----------| +| **0: dark** | `disabled` (or not listed) | Caller deployed, reusable workflow exits immediately | +| **1: advisory** | `advisory` | Runs on dispatch only, all findings shown, nothing blocks | +| **2: standard** | `standard` | Runs on PRs and dispatch, standard profile on PRs | +| **3: blocking** | `standard` + GitHub ruleset | Same as stage 2, plus GitHub ruleset requires the v1 check to pass for PR merge | + +Stage 3 is the combination of the config file setting (`standard`) and a GitHub ruleset. The config file does not control blocking — rulesets do. This separation keeps the config file focused on validation behavior and rulesets focused on merge policy. + +### 10.4 GitHub Rulesets for Blocking + +A new ruleset (org-level or per-repo) requires the v1 validation check to pass before PR merge. The pattern follows the existing `release-snapshot-protection` ruleset. + +- The ruleset references the v1 workflow by check name (workflow name or job name) +- The `camara-release-automation` app can be a bypass actor for automated release PRs that need to merge without validation +- Ruleset management can reuse the existing admin script pattern (`apply-release-rulesets.sh`) + +### 10.5 Rollout Sequence + +1. **Test repo**: `ReleaseTest` — full cycle through stages 0-3, validates all surfacing paths +2. **Template**: `Template_API_Repository` — ensures new repos get v1 caller from creation +3. **Pilot API repos**: 2-3 active repos with engaged codeowners +4. **Batch rollout**: Remaining repos, coordinated with v0 removal (section 10.1) + +### 10.6 Feature Branch Testing + +Admins and rule developers test validation changes on feature branches before merging to main and tagging (UC-15). + +The caller workflow in a test repo is temporarily pointed at the feature branch: + +```yaml +uses: camaraproject/tooling/.github/workflows/validation.yml@feature-branch +``` + +Ref resolution (section 9.5) ensures internal checkouts match — admins have write access, so OIDC resolves the exact SHA. `tooling_ref_override` is available as break-glass for composite action changes not on the workflow branch. + +Rule developers can dispatch validation on existing release branches in a test repo while calling the feature-branch version of the reusable workflow. This validates rule changes against known-good content before merging (UC-10). + +No special framework support is needed — pinned refs are a standard GitHub Actions feature. The framework's only requirement is correct ref resolution (section 9.5). + +### 10.7 Caller Update Strategy + +The v1 caller workflow is deployed by copying from `Template_API_Repository` to each API repo. Since the caller is identical across all repos (section 9.6), deployment is a mechanical copy — no per-repo customization. + +Deployment can be batched using the existing admin tooling pattern (scripted multi-repo operations). The caller can be deployed to all repos at once in stage 0 (dark) — it has no effect until the repo is listed in the config file. + +--- + +*Sections to be added: Release Automation Integration (S5), Operational Views (S5).* From 19a1ba021bb37a5ca940ddc7b7c0c84936b82649 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 17 Mar 2026 21:56:19 +0100 Subject: [PATCH 02/25] docs: add release automation integration and operational views (Session 5) Add sections 11 (Release Automation Integration) and 12 (Operational Views) to the validation framework requirements document. Update cross-references in sections 2.2, 2.3, 3, and 7.7. Resolve deferred bundling/snapshot interaction topic. --- ...AMARA-Validation-Framework-Requirements.md | 191 +++++++++++++++++- 1 file changed, 183 insertions(+), 8 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md index 263f49c..81a387c 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md @@ -170,9 +170,9 @@ The framework constructs a context object at runtime from the trigger, branch, a | `target_api_status` | string | release-plan.yaml `apis[].target_api_status` (per-API) | | `target_api_maturity` | string | Derived from `apis[].target_api_version` (per-API) | | `api_pattern` | string | Detected from OpenAPI spec content (per-API) | -| `is_release_review_pr` | boolean | Detected by framework (used for profile selection, not rule applicability) | +| `is_release_review_pr` | boolean | Detected by framework (used for profile selection; also available as applicability condition, see section 11.4) | -`is_release_review_pr` is used at the framework level to select the strict profile. It is not a rule applicability condition. +`is_release_review_pr` is used at the framework level to select the strict profile. It is also available as an optional applicability condition for checks specific to the release review context (section 11.4). ### 2.3 Execution Contexts @@ -185,8 +185,8 @@ The validation framework must support these execution contexts: | **Dispatch (upstream repo)** | `workflow_dispatch` | advisory | write | Main (default), maintenance, release branches. Warning on non-standard branches | | **Dispatch (fork repo)** | `workflow_dispatch` | advisory | write (fork scope) | Fork owner triggers on own fork. Inherited — no extra work if dispatch trigger exists | | **Local** | CLI / script | advisory | n/a | No GitHub context; subset of rules | -| **Release automation: snapshot** | Called by release workflow | strict | write (app token) | Gate before snapshot creation | -| **Release automation: review PR** | `pull_request` event (push to PR branch) | strict | write or read-only | Same trigger as normal PR; profile is strict based on release review PR detection | +| **Release automation: snapshot** | Called by release workflow | strict | write (app token) | Gate before snapshot creation (section 11.1) | +| **Release automation: review PR** | `pull_request` event (push to PR branch) | strict | write or read-only | Same trigger as normal PR; profile is strict based on release review PR detection (section 11.3) | **Profile is independent of token permissions.** A fork-to-upstream PR gets the same **standard** profile as an upstream-branch PR. The read-only token only limits *how results are surfaced* (e.g. no check run annotations via GITHUB_TOKEN), not validation strictness. Alternative output paths for read-only contexts (workflow summary, `pull_request_target`, bot token) are a Session 4 topic. @@ -214,7 +214,7 @@ The MVP replaces `pr_validation` v0 and delivers the minimum useful validation o ### Post-MVP priorities -- UC-08, UC-09 (release automation strict gates) — high priority, requires strict profile; depends on PR integration being operational first +- UC-08, UC-09 (release automation strict gates, section 11) — high priority, requires strict profile; depends on PR integration being operational first - UC-10 (regression testing against release branches) — rule developer tooling - UC-14 (repository configuration validation) — admin tooling - Automated cache synchronization for `code/common/` and strict version enforcement — bundling itself (ref resolution via `$ref`) is MVP scope (see section 7.3) @@ -222,7 +222,7 @@ The MVP replaces `pr_validation` v0 and delivers the minimum useful validation o ### Independent work (not sequenced with MVP) -- UC-16, UC-17 (dashboard) — Release Progress Tracker can collect last validation run results and trigger validation with appropriate permissions +- UC-16, UC-17 (dashboard, section 12.3) — Release Progress Tracker can collect last validation run results and trigger validation with appropriate permissions ### MVP rollout model @@ -630,7 +630,7 @@ Bundling integrates into the rule architecture (section 5) without requiring cha #### Relationship to snapshot creation -How bundling and validation interact with release automation's snapshot branch creation — whether the framework produces the bundled content that becomes the snapshot, or validates an already-created snapshot — is a WS03 integration topic (Session 5). +Resolved in section 11.2. The validation framework produces the bundled API specs as part of its validation pipeline (steps 2-3 above). These bundled specs are consumed by release automation for the snapshot branch — bundling happens once, during validation, and the output is handed off to release automation. ## 8. Artifact Surfacing @@ -999,4 +999,179 @@ Deployment can be batched using the existing admin tooling pattern (scripted mul --- -*Sections to be added: Release Automation Integration (S5), Operational Views (S5).* +## 11. Release Automation Integration + +The validation framework integrates with CAMARA release automation at two points in the release lifecycle: + +1. **Pre-snapshot gate** (UC-08) — validation runs on the base branch before snapshot creation. Failure blocks the release process. +2. **Release review PR validation** (UC-09) — validation runs on the release review PR with strict profile and content restrictions. + +Both touchpoints use the strict profile (section 2.1): errors and warnings block. + +### 11.1 Pre-Snapshot Validation Gate + +Release automation invokes validation as part of the `/create-snapshot` command, before creating any branches. The validation runs on the current HEAD of the base branch (`main` or maintenance) — this is exactly the content that will become the snapshot. + +**Timing and lifecycle position**: The release state is PLANNED when `/create-snapshot` is invoked. If validation fails, no snapshot branch is created and the state remains PLANNED. The codeowner fixes the reported issues and re-invokes `/create-snapshot`. + +**Profile**: Strict — both errors and warnings block snapshot creation. + +**Rationale for strict**: Once a snapshot is created, mechanical transformations are applied (version replacement, server URLs) and the content becomes immutable. Issues found post-snapshot require discarding and recreating the snapshot. Blocking on warnings at snapshot time avoids this expensive retry cycle. + +**Scope**: The validation framework runs all applicable checks — Spectral rules, Python consistency checks, bundling validation, and release-plan semantic checks. This subsumes the existing `validate-release-plan.py` preflight. The framework applies rule applicability conditions (section 5) with the full release context derived from `release-plan.yaml` on the checked-out branch. + +**Token**: The `camara-release-automation` app token is passed by the release workflow. This is token priority 1 in the layered resolution (section 9.1). The validation framework does not mint this token; it receives it from the caller. + +**Findings output**: Validation findings are reported in the bot's response comment on the Release Issue. The comment includes a structured findings section with error and warning counts, individual findings with fix hints, and a link to the full workflow run for diagnostic artifacts. + +| Aspect | Current release-plan preflight | With validation framework | +|--------|-------------------------------|---------------------------| +| **Checks** | release-plan.yaml schema + semantics | Full framework: Spectral, Python, bundling, release-plan | +| **Blocking** | Errors only | Errors and warnings (strict profile) | +| **Findings output** | Bot comment with error list | Bot comment with structured findings + fix hints | +| **Token** | camara-release-automation app | Same (passed to framework) | + +### 11.2 Bundling and Snapshot Interaction + +This subsection resolves the deferred topic from section 7.7: how bundling and validation interact with release automation's snapshot branch creation. + +**The validation framework produces the bundled API specs as part of its validation pipeline (section 7.2, steps 2-3). These bundled specs are the same artifacts that become the release content on the snapshot branch.** Bundling happens exactly once — during validation. Release automation consumes the bundled output rather than re-bundling independently. + +On the snapshot branch, source API definition files (which contain `$ref` to `code/common/` and `code/modules/`) are **replaced** with the bundled standalone specs produced by the validation framework. This is the "swap strategy" described in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436): the familiar filename (`api-name.yaml`) is retained, but the content is the fully resolved, consumer-ready artifact. + +#### Handoff model + +Two handoff models are viable: + +**Model A — Validation creates the snapshot branch**: The validation framework creates the snapshot branch with bundled API specs already in place. Release automation takes over the branch for mechanical transformations (version replacement, server URLs, `release-metadata.yaml` generation) and the rest of the release lifecycle. + +**Model B — Artifact handoff**: The validation framework uploads bundled specs as workflow artifacts (section 7.2, step 4). Release automation downloads these artifacts and commits them to the snapshot branch as part of snapshot creation. + +Both models avoid duplicate bundling. The choice between them is an implementation detail that depends on workflow architecture constraints. The requirement is: **bundling runs once, during validation, and the output is consumed by release automation for the snapshot.** + +#### Mechanical transformations after bundling + +Regardless of the handoff model, the mechanical transformer applies version-specific changes on top of the bundled content: + +- `info.version` replacement (`wip` → calculated release version) +- Server URL version updates +- `x-camara-commonalities` version field +- Feature file version updates +- Link replacements + +These transformations are release automation's responsibility. The validation framework validates the source content (including bundling); the mechanical transformer produces the final release-ready content. + +#### Cache sync at snapshot time + +A cache synchronization mismatch (section 7.4) is an error in strict profile, blocking snapshot creation. This ensures that `code/common/` content matches the declared `commonalities_release` version before the bundled output is produced and becomes immutable on the snapshot branch. + +### 11.3 Release Review PR Validation + +A release review PR is created by release automation on the `release-review/rX.Y-` branch, targeting the `release-snapshot/rX.Y-` branch. It contains only CHANGELOG and README changes — API specs and other files are immutable on the snapshot branch. + +**Detection**: The framework detects a release review PR by its target branch pattern (`release-snapshot/**`). The `is_release_review_pr` context field (section 2.2) is set to `true`, which selects the strict profile. + +**Profile**: Strict — same as the pre-snapshot gate. Errors and warnings block the PR. + +**Which checks run**: Only a subset of checks is meaningful on a release review PR: + +| Check category | Runs on release review PR | Rationale | +|----------------|--------------------------|-----------| +| CHANGELOG format validation | Yes | CHANGELOG is editable on the review branch | +| README content validation | Yes | README is editable on the review branch | +| File restriction check | Yes | Only CHANGELOG and README may be modified | +| Spectral / API definition checks | No | API specs are immutable on the snapshot branch | +| release-plan.yaml checks | No | Already validated at snapshot creation time | +| Bundling validation | No | Source files are immutable | +| Cache sync validation | No | Already validated at snapshot creation time | + +**File restriction check**: This is a release-review-specific rule. The framework examines the PR diff and produces an error if any file outside `CHANGELOG.md` (or `CHANGELOG/` directory) and `README.md` is modified. This enforces the separation between immutable snapshot content and reviewable documentation. + +**Token**: Standard PR trigger — the validation app token or `GITHUB_TOKEN` fallback. The `camara-release-automation` app token is not used here because this is a regular `pull_request` event, not a release workflow call. + +### 11.4 Context Model Notes + +The context field `is_release_review_pr` (section 2.2) is documented as "used for profile selection, not rule applicability." The file restriction check (section 11.3) requires `is_release_review_pr` as an applicability condition — the check applies only when this field is `true`. + +**Resolution**: `is_release_review_pr` retains its primary role as profile selector. It is also available as an optional applicability condition in rule metadata for checks that are specific to the release review context. This is the only check currently using it as an applicability condition. + +```yaml +# File restriction check — release review PR only +id: "060" +name: release-review-file-restriction +engine: python +applicability: + is_release_review_pr: true +conditional_level: + default: error +description: "Release review PR may only modify CHANGELOG and README files" +hint: "Only CHANGELOG.md (or CHANGELOG/ directory) and README.md may be modified on the release review branch. API specs and other files are immutable on the snapshot branch." +``` + +### 11.5 Pre-Snapshot Invocation + +The pre-snapshot gate follows the same input design principle as PR and dispatch contexts (section 9.4): the framework derives all release context from `release-plan.yaml` on the checked-out branch. No branch name or release tag inputs are needed. + +Release automation invokes the validation framework on the same branch on which `/create-snapshot` was called. The framework reads `release-plan.yaml` from that branch to derive all context fields (target release type, API statuses, Commonalities version, etc.). + +The only distinction is the `mode` — the framework needs to know this is a pre-snapshot invocation rather than a dispatch or PR trigger. When `mode` is `pre-snapshot`, the framework: +- Sets `trigger_type` to `release-automation` +- Selects the strict profile +- Produces bundled API specs as output for consumption by release automation (section 11.2) +- Formats findings for inclusion in a Release Issue comment (not a PR comment) + +The detailed output model (findings format, artifact structure) will be defined consistently across all execution contexts during document consolidation. + +### 11.6 Post-MVP Extensions + +The following integration enhancements are post-MVP: + +- **API-aware change summaries for release notes**: The framework could use oasdiff (section 8.6) to generate semantic change summaries (breaking changes, new endpoints, modified schemas) for inclusion in release notes or the Release Issue. +- **Snapshot transformer validation**: The framework could validate the transformer's configuration before snapshot creation — verifying that version replacement patterns and server URL formats are correct. This goes beyond API content validation into release tooling self-validation. + +--- + +## 12. Operational Views + +### 12.1 Repository Configuration Validation (UC-14) + +An admin needs to verify that an API repository is correctly configured for the validation framework. The following aspects must be checkable: + +- **Caller workflow**: The v1 caller workflow file exists in `.github/workflows/` and matches the expected template content +- **Central config listing**: The repository is listed in the tooling config file (section 10.2) with a valid stage value +- **GitHub ruleset** (stage 3 only): The v1 validation check is required in the applicable ruleset +- **Validation app installation**: The validation GitHub App (DEC-005) is installed for the repository +- **v0 cleanup** (post-transition): The v0 caller file has been removed after v1 is stable at stage 3 + +### 12.2 Minimal Change Noise Principle + +The design choices throughout this document follow a consistent principle: **minimize the number of changes that require codeowner interaction in API repositories.** The only per-repo change required for onboarding is adding the v1 caller workflow file — a mechanical copy from `Template_API_Repository`. After that, all configuration changes happen centrally in the tooling repository (section 10.2). + +### 12.3 Release Manager Dashboard (UC-16, UC-17) + +UC-16 and UC-17 are **independent work** — the Release Progress Tracker already exists and is operational. This section defines the integration points between the validation framework and the tracker. + +**Compliance indication**: For each repository with validation enabled, the dashboard shows a compliance status derived from the most recent validation run on `main`: + +| Status | Meaning | +|--------|---------| +| **compliant** | No errors, no warnings (would pass strict profile) | +| **issues** | Has warnings but no errors (would pass standard, not strict) | +| **failing** | Has errors (would fail standard profile) | +| **unknown** | No validation data available | + +**Data collection**: The tracker queries the GitHub API for the most recent completed run of the v1 validation workflow on the default branch (conclusion, URL, timestamp). No changes to the validation framework are needed for this. + +**On-demand trigger** (UC-17): The tracker or an admin can dispatch validation on selected repositories via `workflow_dispatch` (section 9.2) to update dashboard status. + +### 12.4 Cross-System Integration Map + +| System | Integration point | Data flow | Priority | +|--------|------------------|-----------|----------| +| Release automation (`/create-snapshot`) | Pre-snapshot gate (11.1) | Validation → release automation: pass/fail + findings + bundled specs | Post-MVP (high) | +| Release automation (release review PR) | PR trigger with strict profile (11.3) | Standard PR validation flow, profile auto-selected | Post-MVP (high) | +| Release Progress Tracker | Workflow run query (12.3) | Tracker reads validation run data from GitHub API | Independent | +| Release Progress Tracker | Dispatch trigger (12.3) | Tracker dispatches validation via GitHub API | Independent | +| Tooling config file | Central enable/disable (10.2) | Validation reads config at runtime | MVP | +| GitHub rulesets | Blocking enforcement (10.4) | Ruleset references validation check name | Stage 3 | +| Validation GitHub App | Token minting (9.1) | App provides write token for findings surfacing | MVP | From cfc4458da0ea01c0466b73771ae9158eade33c4a Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 17 Mar 2026 23:15:35 +0100 Subject: [PATCH 03/25] docs: split requirements into requirements + detailed design (Session 6) Split the 1,178-line combined document into a focused requirements document (522 lines) and a companion detailed design document (634 lines). The requirements document is now reviewable by WG members without implementation detail. The detailed design document preserves all architecture decisions, processing flows, YAML structures, and workflow templates for implementers. --- ...RA-Validation-Framework-Detailed-Design.md | 634 ++++++++++++++ ...AMARA-Validation-Framework-Requirements.md | 819 ++---------------- 2 files changed, 716 insertions(+), 737 deletions(-) create mode 100644 documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md new file mode 100644 index 0000000..01b59a9 --- /dev/null +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -0,0 +1,634 @@ +# Validation Framework — Detailed Design + +**Status**: Work in progress +**Last updated**: 2026-03-17 + +> This document supplements the [Validation Framework Requirements](CAMARA-Validation-Framework-Requirements.md) with design and implementation detail for developers and architects. It is expected to migrate to the `tooling` repository alongside the implementation. + +--- + +## 1. Rule Metadata Model + +### 1.1 YAML Structure and Examples + +Rule metadata is expressed in YAML. Fields that are not constrained are omitted (omitted = applies in all contexts for that dimension). The primary use of the metadata is **post-filter and severity mapping**: engines run and produce findings, then the framework applies applicability and conditional level to interpret the results in the current context. + +```yaml +id: "042" # flat sequential ID, stable across engine changes +name: path-kebab-case # human-readable name +engine: spectral # spectral | yamllint | gherkin | python | manual +engine_rule: "camara-parameter-casing-convention" # native engine rule ID (if applicable) +hint: "Use kebab-case for all path segments: /my-resource/{resourceId}" + +applicability: # only list fields that constrain; omitted = no constraint + branch_types: [main, release] + trigger_types: [pr, dispatch] + # ... further conditions as needed + +conditional_level: + default: error # always present + overrides: # only if level varies by context + - condition: + target_release_type: [pre-release-alpha] + level: hint +``` + +Conditional level examples: + +```yaml +# "Test definition must be present" — hint by default, warn for RC/public of stable APIs +conditional_level: + default: hint + overrides: + - condition: + target_api_maturity: [stable] + target_release_type: [pre-release-rc, public-release] + level: warn + +# "Commonalities compliance check" — warn by default, suppressed for draft APIs +conditional_level: + default: warn + overrides: + - condition: + target_api_status: [draft] + level: off +``` + +### 1.2 Condition Evaluation + +``` +applicability match: + for each field in rule.applicability: + if field is array: context value must be IN the array (OR) + if field is range string: context value must satisfy the range expression + if field is boolean: context value must equal the field value + all fields must match (AND) + omitted fields are unconstrained (always match) + +conditional level: + for each override in rule.conditional_level.overrides (in order): + if override.condition matches context (same logic as applicability): + return override.level + return rule.conditional_level.default +``` + +### 1.3 Spectral Pass-Through Principle + +The framework uses Spectral's severity names (`error`, `warn`, `hint`) as its native level values. This gives identity mapping for the primary engine: + +| Spectral | Framework | Notes | +|----------|-----------|-------| +| `error` | `error` | Identity | +| `warn` | `warn` | Identity | +| `hint` | `hint` | Identity | +| `info` | `hint` | Mapped (rarely used) | +| `off` | `off` | Identity (disable rule) | + +Spectral rules already include `message` fields with fix guidance. Therefore, **Spectral rules that do not need context-dependent severity or applicability filtering do not require explicit framework metadata entries**. Their findings pass through with direct severity mapping and native messages. + +Framework metadata is only needed for Spectral rules when: +- The level should change based on context (e.g., error on release branch, hint on feature branch) +- The rule should be suppressed in certain contexts (applicability filtering) +- The fix hint should be overridden or augmented + +This minimizes the metadata surface: only rules with context-dependent behavior need explicit entries. + +The framework consumes Spectral output as structured data (JSON), not terminal text. This enables programmatic post-filtering, severity remapping, and merging with findings from other engines. + +Spectral does not resolve `$ref` references before linting — it validates the document as-is. Checks that depend on the content of referenced schemas (e.g., from CAMARA_common.yaml) require either a pre-bundled input spec or a Python implementation with explicit ref resolution. See section 3.1 (Spectral and `$ref` Interaction) for implications. + +For further Spectral-specific details, see [spectral-integration-notes.md](../reviews/spectral-integration-notes.md). + +### 1.4 Derived Context Fields + +Two per-API context fields are derived from content rather than declared in release-plan.yaml: + +**`target_api_maturity`**: Derived from `apis[].target_api_version` — `initial` if major version is 0 (v0.x.y), `stable` if major version >= 1 (vx.y.z). Determines which asset requirements apply per the API Readiness Checklist (e.g., stable public APIs require enhanced test cases and user stories). + +**`api_pattern`**: Detected from OpenAPI spec content — `request-response`, `implicit-subscription`, or `explicit-subscription`. Detection logic examines paths (subscription endpoints), callbacks, schema names, and content types. Multiple pattern-specific rule sets (REQ, IMP, EXP, EVT categories) depend on this classification. This detection is a cross-cutting capability used by many rules. + +### 1.5 Spectral Migration Potential + +Analysis of the deprecated `api_review_validator_v0_6.py` shows that approximately 40% of its checks are implementable as Spectral rules (single-file OpenAPI pattern matching), and an additional 15% could use Spectral custom JavaScript functions. This includes: + +- Mandatory error response checks (400, 401, 403) +- Server URL format validation +- info.version format validation +- License and security scheme validation +- ErrorInfo and XCorrelator schema presence +- Error response structure validation + +The main blocker for migrating ~20% of checks to Spectral is the dependency on `api_pattern` detection — Spectral cannot natively apply rules conditionally based on detected API type. These checks either need custom JS functions that embed the detection logic, or remain as Python checks that use `api_pattern` from the context. + +The Commonalities audit should evaluate each candidate check against the current design guide version before migration. + +### 1.6 Authoritative Schema References + +The execution context fields and their allowed values are defined by the following schemas, which are the authoritative sources: + +- **release-plan.yaml**: `artifacts/metadata-schemas/schemas/release-plan-schema.yaml` (in ReleaseManagement) +- **release-metadata.yaml**: `artifacts/metadata-schemas/schemas/release-metadata-schema.yaml` (in ReleaseManagement) + +The framework must accept exactly the values defined in these schemas. Any change to the schemas must be reflected in the framework's context model. + +--- + +## 2. Check Inventory Detail + +### 2.1 Inventory Status + +The per-rule inventory is **not yet complete**. The following work is required: + +1. **Commonalities audit** (dependency): Examine `CAMARA-API-Design-Guide.md` and `CAMARA-API-Event-Subscription-and-Notification-Guide.md` at both r3.4 and r4.1/r4.2 versions to: + - Identify checks not yet covered by any engine + - Validate existing Spectral rules against the current design guide + - Identify rules that changed between Commonalities releases + +2. **Existing rule classification**: Map each current Spectral rule to the framework metadata model (applicability, conditional level, hints). The existing Spectral severity levels are assumed valid for now; detailed severity review is deferred. + +3. **api-review v0.6 coverage**: The deprecated `api_review_validator_v0_6.py` (~43 checks) was a monolithic implementation used manually by release reviewers for the Fall25 meta-release. Its checks serve as input for identifying Python-needed validations not covered by Spectral. + +### 2.2 Known Check Areas by Engine + +Summary of check areas identified so far, pending the Commonalities audit for completeness: + +**Spectral (existing + new):** +- OpenAPI version enforcement (3.0.3) +- Naming conventions (paths: kebab-case, schemas: PascalCase, operationId: camelCase) +- Required descriptions (operations, parameters, responses, properties) +- Reserved words detection +- Security: no secrets in path/query parameters +- HTTP method validity, no request body on GET/DELETE +- Unused components detection +- *New*: info.version format (wip/alpha.n/rc.n/public), XCorrelator pattern, phone number format, device object structure + +**Python (new):** +- Server URL version consistency with info.version (cross-field) +- Version must be wip on main, must not be wip on release branches (context-dependent) +- release-plan.yaml non-exclusivity check (PR diff analysis) +- release-plan.yaml schema and semantic validation (existing, to be integrated) +- Error response structure and code compliance (cross-schema) +- Test file existence and version alignment (cross-file) +- CHANGELOG format and link tag-locking (file content analysis) +- API pattern-specific checks: subscription endpoints, CloudEvents format, event type naming (structural + semantic) + +**Manual + prompt:** +- Data minimization compliance (GEN-009, GEN-010) +- Meaningful description quality (beyond presence checks) +- User story adequacy +- Breaking change justification + +**Obsolete (handled by release automation):** +- API Readiness Checklist file management (files should no longer be in the repository) +- Release tag creation and format +- Version field replacement on release branches (wip → actual version) +- release-metadata.yaml generation +- README update with release information + +--- + +## 3. Bundling Pipeline + +### 3.1 Spectral and `$ref` Interaction + +#### Bundling vs full dereferencing + +The framework uses **bundling** (external ref resolution only), not full dereferencing: + +- **Bundling**: Resolves external `$ref` — pulls content from `code/common/`, `code/modules/`, and other local files into the document. Internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`) are preserved. +- **Full dereferencing**: Resolves all `$ref` including internal ones, producing a flat document with zero `$ref` and massive duplication. The framework must **not** use full dereferencing. + +Preserving internal `$ref` ensures that: + +- Spectral rules checking component structure, `$ref` patterns, and `#/components/` organization continue to work on bundled output +- Bundled output remains readable and structurally equivalent to what reviewers expect +- No Spectral rule changes are needed between copy-paste and bundled models + +Any constraints on where API designers may use external vs internal `$ref` are defined in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436), not by the validation framework. The framework enforces whatever ref patterns the design document specifies. + +#### Transition period + +During migration from copy-paste to the local copy model, both repository types coexist: + +- **Copy-paste repos**: All schemas inline. Spectral runs directly on source. No bundling needed. +- **`$ref` repos**: Spectral runs on bundled output. All external refs resolved, internal refs preserved. Structurally equivalent to copy-paste. + +No rule changes are needed between the two models — bundling normalizes external refs while preserving the internal structure that Spectral rules depend on. Rule IDs remain stable across the transition (flat namespace from Requirements section 5). + +#### Bundling is MVP scope + +Bundling support for `CAMARA_common.yaml` via `$ref` is **within MVP scope**. Commonalities 0.7.x requires updated common schemas, and the ability to consume them via `$ref` — with the framework handling bundling transparently — is the key additional value of the validation framework v1 for codeowners. This avoids repeating the difficult-to-validate copy-paste pattern. + +In the MVP, some parts may still be manual — providing the correct copy in `code/common/` and ensuring it matches the declared `commonalities_release` version. But the `$ref` option is available for early adopters, and the framework handles bundling when `$ref` is detected. Automated cache synchronization and strict version enforcement are post-MVP enhancements. + +### 3.2 Dependency Categories and File Mapping + +Three categories of shared schema dependencies exist, each with different characteristics: + +**Commonalities** (well-known, hardcoded in tooling): +The Commonalities repository provides shared schemas that are well-known to automation tooling. Currently two files are relevant: +- `CAMARA_common.yaml` — common data types, error responses, headers +- `notification-as-cloud-event.yaml` — CloudEvents notification schema + +These files, their source location in the Commonalities repository, and the mapping from `release-plan.yaml.dependencies.commonalities_release` to the correct version are built into the tooling. No per-repository configuration is needed. + +**ICM** (version compatibility constraint): +Identity and Consent Management schemas are currently contained within Commonalities files — there are no separate ICM files to cache. The `dependencies.identity_consent_management_release` in `release-plan.yaml` is a version compatibility constraint (potentially `>= x.y.z`) rather than a file-caching relationship. The exact nature of this dependency requires further discussion. + +**Sub-project commons** (extensible, declared per repository): +Sub-projects may define common schemas shared across their API repositories (e.g., a device API family sharing common device type definitions). These dependencies must be declarable without requiring changes to the automation tooling. Each sub-project dependency requires: +- Source repository +- Release tag or version +- Array of files to consume + +This extensible model requires a dependency declaration format, either within `release-plan.yaml` or as a separate manifest. The schema design is a follow-up topic for the bundling design document. + +#### File caching strategy + +Which files are cached in `code/common/` — demand-driven (only files actually `$ref`'d) vs declaration-driven (all files from declared dependencies) — is a sync mechanism concern defined in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436), not a validation framework decision. + +The framework's checks are the same regardless: cached files must match their declared source version, and `$ref` targets must exist. + +### 3.3 Commonalities Version Matrix + +#### Active versions + +The framework must support validation rules that vary by Commonalities version. Active versions at the time of writing: + +- **r3.4** (Commonalities v0.6.x) — Fall25 meta-release, frozen, maintenance releases only +- **r4.x** (Commonalities v0.7.x) — Spring26 meta-release. r4.1 is the release candidate (available now); r4.2 is the upcoming public release and will replace r4.1 + +Within a version line (r4.x), the latest release is always authoritative. When r4.2 is available, r4.1 becomes obsolete — new releases must target r4.2. If a maintenance release r4.3 follows, it replaces r4.2 for validation purposes. + +Future Commonalities major versions (e.g., r5.x for v1.0.0) will add further version lines. The architecture must not assume a fixed number of active versions. + +#### Spectral ruleset selection (pre-selection) + +Each Commonalities major version line gets its own Spectral ruleset (e.g., `.spectral-r3.4.yaml`, `.spectral-r4.yaml`). The framework reads `commonalities_release` from `release-plan.yaml` and selects the matching ruleset before running Spectral. + +This avoids running contradicting rules from different Commonalities versions simultaneously, which would produce confusing Spectral output even if the results were filtered afterwards. The r3.4 ruleset is effectively frozen — only maintenance fixes. New rule development targets the current r4.x ruleset. + +#### Framework rule metadata (post-filter with conditionals) + +Framework rule metadata uses a single ruleset with `commonalities_release` range conditions for version-specific behavior. This is appropriate because: + +- Python checks are framework-controlled and do not produce confusing intermediate output +- Most framework rules apply across versions; only a minority are version-specific +- Duplicating shared rules into per-version files would create drift risk + +The Commonalities audit will identify which rules changed between r3.4 and r4.x. Those rules receive `commonalities_release` range conditions in their metadata. + +### 3.4 Placeholder Handling + +#### Current state + +The current `CAMARA_common.yaml` contains placeholder patterns (e.g., `{{SPECIFIC_CODE}}`) that have no defined resolution rules. These should be removed from Commonalities, with API repositories extending shared schemas via `allOf` instead (per the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436)). + +#### Future direction + +Placeholder replacement with defined values could be introduced together with bundling as part of a broader transformation pipeline. This could include dynamic variables such as `api_version`, `commonalities_release`, `commonalities_version`, effectively replacing the current "wip" and "/main/" substitutions done by the snapshot transformer. In this model, bundling + transformation (including placeholder replacement) would produce the release-ready artifact. + +### 3.5 Rule Architecture Integration + +Bundling integrates into the rule architecture (Requirements section 5) without requiring changes to the context model or rule metadata: + +- **Step assignment**: Each rule runs in either step 1 (pre-bundling validation) or step 3 (full validation). Assignment is an implementation detail — the framework knows which checks belong to which step. +- **No new context fields**: The context model from Requirements section 2.2 is sufficient. Whether external refs existed and were resolved is an implementation concern, not a rule applicability condition. +- **Cache sync is a check, not context**: The cache synchronization validation (section 3.2) produces findings (warning or error depending on profile). It is not a context field consumed by other rules. +- **Spectral ruleset selection**: The `commonalities_release` field (already in the context model) drives Spectral ruleset pre-selection (section 3.3). No additional metadata is needed. + +--- + +## 4. Artifact Surfacing Detail + +### 4.1 Workflow Artifact Naming + +- Bundled specs are uploaded as GitHub workflow artifacts with a naming convention that identifies the API name, branch, and commit SHA +- Bundled files include a header comment: `# For information only - DO NOT EDIT` +- Workflow artifact retention uses the GitHub default (90 days) + +### 4.2 Temporary Branch Model + +Workflow artifacts replace the temporary branch model (`/tmp/bundled/-`) for MVP. Temporary branches may be revisited post-MVP if reviewers need a browsable view of bundled content. + +### 4.3 "wip" Version Handling + +Bundling on `main` leaves `info.version` as-is — it contains `wip` as expected for unreleased code. Version replacement on release branches is handled by release automation (snapshot transformer), not by the validation framework's bundling step. The framework validates version correctness per branch type — this is an existing check (section 2.2), not a new bundling-specific requirement. + +--- + +## 5. Caller Workflow Design + +### 5.1 Token Resolution Strategy + +The framework uses a layered token resolution strategy. The order prioritizes consistent branding (all findings come from the same bot identity) over using whichever token happens to be available: + +1. **Snapshot context**: `camara-release-automation` app token — provided by the calling release workflow. The validation framework does not mint this token; it is passed in by the release automation caller. +2. **Validation default**: Dedicated validation app bot token — the framework mints an installation token for the current repository. This is the primary path for all PR and dispatch contexts, ensuring consistent bot identity on annotations and comments. +3. **Fallback**: `GITHUB_TOKEN` with write access — used when the validation app is not installed (e.g., dispatch in a fork, or repositories not yet onboarded to the app). Write capability is probed at runtime. +4. **Read-only**: Workflow summary and diagnostic artifacts only — when no write token is available. + +In normal operation, both upstream PRs and fork PRs show findings from the validation bot. The `GITHUB_TOKEN` fallback and read-only mode are degraded paths, not the expected default. + +### 5.2 Validation GitHub App + +A dedicated GitHub App handles write surfaces for validation. This is a **separate app** from `camara-release-automation` — it has a narrower permission scope and a different purpose. + +| Aspect | Validation App | camara-release-automation | +|--------|---------------|--------------------------| +| **Purpose** | PR annotations, comments, commit status | Release snapshot creation, branch management | +| **Permissions** | `checks: write`, `pull-requests: write`, `statuses: write` | `contents: write`, `workflows: write`, plus release management | +| **Commits/pushes** | Never | Yes (snapshot branches, tags, release assets) | +| **EasyCLA** | Not needed (no commits) | Required (commits to repos with CLA enforcement) | + +Org-level configuration: +- `vars.VALIDATION_APP_ID` — app ID (org variable) +- `secrets.VALIDATION_APP_PRIVATE_KEY` — app private key (org secret) + +The validation app is introduced from day one (MVP) to establish consistent bot identity and avoid caller workflow changes later. + +#### Relationship to v0 surfacing + +The v0 workflow surfaces findings via MegaLinter's built-in reporters (`GITHUB_COMMENT_REPORTER`, `GITHUB_STATUS_REPORTER`) and custom `actions/github-script` steps for release-plan validation. The v1 framework replaces all of these with its own unified surfacing layer. MegaLinter is no longer used as the orchestration layer. + +### 5.3 Trigger and Concurrency YAML + +#### PR trigger + +```yaml +on: + pull_request: + branches: + - main + - release-snapshot/** + - maintenance/** +``` + +- **`main`**: Standard development PRs. Profile: standard (or strict if release review PR detected) +- **`release-snapshot/**`**: Release review PRs created by release automation on snapshot branches. Profile: strict +- **`maintenance/**`**: Maintenance branch PRs. Profile: standard + +Default event types (`opened`, `synchronize`, `reopened`) are sufficient. The framework validates code content, not PR metadata — `edited` (title/body changes) is not needed. + +#### Dispatch trigger + +```yaml + workflow_dispatch: +``` + +Dispatch runs on whatever branch the user selects in the GitHub UI. The framework derives branch type, release context, and all validation parameters from the checked-out branch content. + +#### Concurrency + +```yaml +concurrency: + group: ${{ github.ref }}-${{ github.workflow }} + cancel-in-progress: true +``` + +Same model as v0: a new push to a PR branch cancels the previous validation run. Dispatch behaves identically — a second dispatch on the same branch cancels the first. + +### 5.4 Permissions Detail + +The caller workflow declares the maximum permission set. The reusable workflow inherits these as a ceiling — it cannot elevate above what the caller declares. + +```yaml +permissions: + checks: write + pull-requests: write + issues: write + contents: read + statuses: write + id-token: write +``` + +| Permission | Purpose | Fork PR behavior | +|------------|---------|-----------------| +| `checks: write` | Check run annotations (findings inline in PR diff) | Restricted by GitHub; validation app token used instead | +| `pull-requests: write` | PR review interactions | Restricted by GitHub; validation app token used instead | +| `issues: write` | PR comments (PRs use the Issues API for comments) | Restricted by GitHub; validation app token used instead | +| `contents: read` | Repository checkout | Available (read-only) | +| `statuses: write` | Commit status (per-check context in checks list) | Restricted by GitHub; validation app token used instead | +| `id-token: write` | OIDC token for tooling ref resolution (section 5.6) | May not be granted for fork PRs — see section 5.6 | + +For fork PRs, `GITHUB_TOKEN` write permissions are restricted by GitHub regardless of what the caller declares. The validation app token (section 5.1) bypasses this restriction because it is minted from the app's own credentials, independent of `GITHUB_TOKEN`. + +### 5.5 Input Design Detail + +The reusable workflow does not accept inputs that duplicate information derivable from the checked-out branch. This prevents contradictions where an input says one thing but branch content says another. + +**Example of the problem avoided**: If the workflow accepted a `release_type` input, a user could dispatch on `main` with `release_type: public-release` while `release-plan.yaml` on `main` says `target_release_type: pre-release-alpha`. The framework would need reconciliation logic, and the user would get confusing results. + +**Forbidden inputs**: `branch_type`, `release_type`, `api_status`, `commonalities_version`, `configurations` — all derivable from branch content or from the central configuration file (Requirements section 10). + +**Consequence for the caller workflow**: No per-repo inputs exist. All per-repo configuration (linting config subfolder, enabled features, rollout stage) lives in the central config file read by the reusable workflow. The caller workflow is identical across all repositories, with no `with:` block needed in standard operation. This makes it protectable via CODEOWNERS or rulesets — nobody ever needs to edit it. + +### 5.6 Ref Resolution + +#### OIDC-based ref resolution (primary) + +The reusable workflow resolves its own tooling repository and commit SHA via OIDC claims (`job_workflow_sha`), following the pattern established in tooling#121. This ensures all internal checkouts (linting config, shared actions at runtime) use the same tooling version that the caller specified. + +The caller workflow declares `id-token: write` to enable OIDC token generation. + +#### Hardcoded version fallback + +If OIDC token generation fails (e.g., fork PRs where `id-token: write` may not be granted), the reusable workflow falls back to its own hardcoded version tag (e.g., `v1`). This is the same pattern as v0's hardcoded `ref: v0`. + +This fallback is acceptable because: +- **Fork PRs**: Contributors do not need pinned-SHA ref resolution — the release version tag (`v1`) is correct for production validation +- **Feature branch testing**: Done by admins and rule developers who have write access, so OIDC works +- **Release automation**: Always triggered by codeowners with write access, so OIDC works + +The fallback means fork PR validation always uses the published version of the tooling, not a feature branch. This is the expected behavior — only admins test unreleased tooling versions. + +#### Break-glass override + +`tooling_ref_override` — a 40-character SHA input to the reusable workflow. Takes precedence over both OIDC and the hardcoded fallback. Documented as pilot/break-glass only. Same mechanism as release automation. + +#### Resolution order + +1. `tooling_ref_override` input (if set) — explicit SHA, highest priority +2. OIDC `job_workflow_sha` claim (if `id-token` available) — exact commit SHA +3. Hardcoded version tag in the reusable workflow (e.g., `v1`) — always available + +### 5.7 Caller Workflow Template + +The caller workflow is identical across all repositories: + +```yaml +name: CAMARA Validation + +on: + pull_request: + branches: + - main + - release-snapshot/** + - maintenance/** + workflow_dispatch: + +concurrency: + group: ${{ github.ref }}-${{ github.workflow }} + cancel-in-progress: true + +permissions: + checks: write + pull-requests: write + issues: write + contents: read + statuses: write + id-token: write + +jobs: + validation: + uses: camaraproject/tooling/.github/workflows/validation.yml@v1 + secrets: inherit +``` + +No `with:` block in standard operation. The caller is a thin pass-through that provides triggers, permissions, and concurrency. All validation logic, configuration, and surfacing are handled by the reusable workflow. + +### 5.8 Version Tagging and Secrets + +#### Version tagging + +The reusable workflow uses a floating version tag (`v1`) analogous to v0's `v0` tag. The tag is moved forward as the framework evolves within the v1 major version. Breaking changes (new required permissions, changed caller contract) require a new major version tag. + +#### Secrets + +The caller passes `secrets: inherit`. The reusable workflow uses: +- `GITHUB_TOKEN` — inherited, for checkout and fallback write surfaces +- Org secrets for validation app token minting (`VALIDATION_APP_PRIVATE_KEY`) — accessed via `secrets` context +- Org variables for app identity (`VALIDATION_APP_ID`) — accessed via `vars` context + +--- + +## 6. Rollout Implementation + +### 6.1 Why Separate Callers + +The v0 reusable workflow has a fundamentally different structure (MegaLinter-based, single job, different permissions and output model). A single caller with version switching would require complex conditional logic. + +Separate callers allow independent lifecycle: v0 can be removed per-repo after v1 is proven, without coordinating a simultaneous switch. GitHub rulesets can reference the v1 check name independently of v0. + +### 6.2 Central Config Alternatives Analysis + +**Rationale for config file over alternatives:** + +- **Org variable with repo list** (rejected): JSON arrays in org variables become unwieldy at 60+ repos and hit variable size limits. Not PR-reviewable. +- **Per-repo variable** (rejected): Requires touching each repository to enable. Violates UC-13 — central administration without per-repo configuration changes. +- **Caller version tag** (rejected): Would require editing the caller workflow per-repo, undermining the identical-caller-across-all-repos design (section 5.7). +- **Tooling config file** (chosen): Version-controlled, PR-reviewable, scalable. Adding a repo is one line in a YAML file. Can hold per-repo settings beyond enable/disable (linting config subfolder, rollout stage). Satisfies UC-13 — no per-repo config changes needed. + +The config file schema is an implementation detail. At minimum it maps repository name to rollout stage and per-repo settings (e.g., linting config subfolder previously handled by the `configurations` input in v0). + +### 6.3 GitHub Rulesets for Blocking + +A new ruleset (org-level or per-repo) requires the v1 validation check to pass before PR merge. The pattern follows the existing `release-snapshot-protection` ruleset. + +- The ruleset references the v1 workflow by check name (workflow name or job name) +- The `camara-release-automation` app can be a bypass actor for automated release PRs that need to merge without validation +- Ruleset management can reuse the existing admin script pattern (`apply-release-rulesets.sh`) + +### 6.4 Rollout Sequence + +1. **Test repo**: `ReleaseTest` — full cycle through stages 0-3, validates all surfacing paths +2. **Template**: `Template_API_Repository` — ensures new repos get v1 caller from creation +3. **Pilot API repos**: 2-3 active repos with engaged codeowners +4. **Batch rollout**: Remaining repos, coordinated with v0 removal + +### 6.5 Feature Branch Testing + +Admins and rule developers test validation changes on feature branches before merging to main and tagging (UC-15). + +The caller workflow in a test repo is temporarily pointed at the feature branch: + +```yaml +uses: camaraproject/tooling/.github/workflows/validation.yml@feature-branch +``` + +Ref resolution (section 5.6) ensures internal checkouts match — admins have write access, so OIDC resolves the exact SHA. `tooling_ref_override` is available as break-glass for composite action changes not on the workflow branch. + +Rule developers can dispatch validation on existing release branches in a test repo while calling the feature-branch version of the reusable workflow. This validates rule changes against known-good content before merging (UC-10). + +No special framework support is needed — pinned refs are a standard GitHub Actions feature. The framework's only requirement is correct ref resolution (section 5.6). + +### 6.6 Caller Update Strategy + +The v1 caller workflow is deployed by copying from `Template_API_Repository` to each API repo. Since the caller is identical across all repos (section 5.7), deployment is a mechanical copy — no per-repo customization. + +Deployment can be batched using the existing admin tooling pattern (scripted multi-repo operations). The caller can be deployed to all repos at once in stage 0 (dark) — it has no effect until the repo is listed in the config file. + +### 6.7 WP-01 Relationship + +WP-01 (tooling#121) fixes ref consistency in the existing v0 reusable workflow. It validates the OIDC ref resolution pattern that v1 reuses and adds the `tooling_ref_override` break-glass input. WP-01 does not change the v0 caller — callers still call `@v0`. The v1 reusable workflow reuses the same ref resolution pattern with the hardcoded version fallback (section 5.6). + +--- + +## 7. Release Automation Implementation + +### 7.1 Bundling and Snapshot Interaction + +The validation framework produces the bundled API specs as part of its validation pipeline (Requirements section 6.2, steps 2-3). These bundled specs are the same artifacts that become the release content on the snapshot branch. Bundling happens exactly once — during validation. Release automation consumes the bundled output rather than re-bundling independently. + +On the snapshot branch, source API definition files (which contain `$ref` to `code/common/` and `code/modules/`) are **replaced** with the bundled standalone specs produced by the validation framework. This is the "swap strategy" described in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436): the familiar filename (`api-name.yaml`) is retained, but the content is the fully resolved, consumer-ready artifact. + +#### Handoff model + +Two handoff models are viable: + +**Model A — Validation creates the snapshot branch**: The validation framework creates the snapshot branch with bundled API specs already in place. Release automation takes over the branch for mechanical transformations (version replacement, server URLs, `release-metadata.yaml` generation) and the rest of the release lifecycle. + +**Model B — Artifact handoff**: The validation framework uploads bundled specs as workflow artifacts. Release automation downloads these artifacts and commits them to the snapshot branch as part of snapshot creation. + +Both models avoid duplicate bundling. The choice between them is an implementation detail that depends on workflow architecture constraints. The requirement is: **bundling runs once, during validation, and the output is consumed by release automation for the snapshot.** + +#### Mechanical transformations after bundling + +Regardless of the handoff model, the mechanical transformer applies version-specific changes on top of the bundled content: + +- `info.version` replacement (`wip` → calculated release version) +- Server URL version updates +- `x-camara-commonalities` version field +- Feature file version updates +- Link replacements + +These transformations are release automation's responsibility. The validation framework validates the source content (including bundling); the mechanical transformer produces the final release-ready content. + +#### Cache sync at snapshot time + +A cache synchronization mismatch is an error in strict profile, blocking snapshot creation. This ensures that `code/common/` content matches the declared `commonalities_release` version before the bundled output is produced and becomes immutable on the snapshot branch. + +### 7.2 Token and Findings Output for Pre-Snapshot + +**Token**: The `camara-release-automation` app token is passed by the release workflow. This is token priority 1 in the layered resolution (section 5.1). The validation framework does not mint this token; it receives it from the caller. + +**Findings output**: Validation findings are reported in the bot's response comment on the Release Issue. The comment includes a structured findings section with error and warning counts, individual findings with fix hints, and a link to the full workflow run for diagnostic artifacts. + +### 7.3 File Restriction Check + +The context field `is_release_review_pr` (Requirements section 2.2) serves dual roles: profile selection and applicability condition. The file restriction check is the only check currently using it as an applicability condition. + +```yaml +# File restriction check — release review PR only +id: "060" +name: release-review-file-restriction +engine: python +applicability: + is_release_review_pr: true +conditional_level: + default: error +description: "Release review PR may only modify CHANGELOG and README files" +hint: "Only CHANGELOG.md (or CHANGELOG/ directory) and README.md may be modified on the release review branch. API specs and other files are immutable on the snapshot branch." +``` + +### 7.4 Pre-Snapshot Invocation Detail + +Release automation invokes the validation framework on the same branch on which `/create-snapshot` was called. The framework reads `release-plan.yaml` from that branch to derive all context fields (target release type, API statuses, Commonalities version, etc.). + +The only distinction is the `mode` — the framework needs to know this is a pre-snapshot invocation rather than a dispatch or PR trigger. When `mode` is `pre-snapshot`, the framework: +- Sets `trigger_type` to `release-automation` +- Selects the strict profile +- Produces bundled API specs as output for consumption by release automation (section 7.1) +- Formats findings for inclusion in a Release Issue comment (not a PR comment) + +The detailed output model (findings format, artifact structure) will be defined consistently across all execution contexts during implementation. diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md index 81a387c..9c861ff 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md @@ -1,8 +1,10 @@ -# Validation Framework — Requirements (Internal Draft) +# Validation Framework — Requirements **Status**: Work in progress **Last updated**: 2026-03-17 +> For design and implementation detail, see [Validation Framework — Detailed Design](CAMARA-Validation-Framework-Detailed-Design.md). + --- ## 1. Use Cases @@ -89,30 +91,6 @@ The profile (section 2.1) then determines which levels block. This separates thr | **hint** | Recommendation, never blocking | *(none)* | | **off** | Suppressed, finding not shown | *(n/a)* | -#### Rule metadata - -Rule metadata is expressed in YAML. Fields that are not constrained are omitted (omitted = applies in all contexts for that dimension). The primary use of the metadata is **post-filter and severity mapping**: engines run and produce findings, then the framework applies applicability and conditional level to interpret the results in the current context. - -```yaml -id: "042" # flat sequential ID, stable across engine changes -name: path-kebab-case # human-readable name -engine: spectral # spectral | yamllint | gherkin | python | manual -engine_rule: "camara-parameter-casing-convention" # native engine rule ID (if applicable) -hint: "Use kebab-case for all path segments: /my-resource/{resourceId}" - -applicability: # only list fields that constrain; omitted = no constraint - branch_types: [main, release] - trigger_types: [pr, dispatch] - # ... further conditions as needed - -conditional_level: - default: error # always present - overrides: # only if level varies by context - - condition: - target_release_type: [pre-release-alpha] - level: hint -``` - #### Applicability conditions A rule is skipped silently if its conditions don't match the current context. Multiple fields are combined with AND; multiple values within an array field are combined with OR. @@ -134,27 +112,6 @@ Range comparison for `commonalities_release` uses `packaging.specifiers` (Python `default` is always present. `overrides` is a list of `{condition, level}` pairs evaluated in order; first match wins. Conditions use the same field/value model as applicability (AND across fields, OR within arrays). The level `off` can be used in overrides to suppress a finding in specific contexts. -Examples: - -```yaml -# "Test definition must be present" — hint by default, warn for RC/public of stable APIs -conditional_level: - default: hint - overrides: - - condition: - target_api_maturity: [stable] - target_release_type: [pre-release-rc, public-release] - level: warn - -# "Commonalities compliance check" — warn by default, suppressed for draft APIs -conditional_level: - default: warn - overrides: - - condition: - target_api_status: [draft] - level: off -``` - #### Execution context The framework constructs a context object at runtime from the trigger, branch, and release-plan.yaml content. Rules are evaluated per-API for API-specific checks (a repository with three APIs at different statuses produces three evaluation contexts). @@ -170,9 +127,7 @@ The framework constructs a context object at runtime from the trigger, branch, a | `target_api_status` | string | release-plan.yaml `apis[].target_api_status` (per-API) | | `target_api_maturity` | string | Derived from `apis[].target_api_version` (per-API) | | `api_pattern` | string | Detected from OpenAPI spec content (per-API) | -| `is_release_review_pr` | boolean | Detected by framework (used for profile selection; also available as applicability condition, see section 11.4) | - -`is_release_review_pr` is used at the framework level to select the strict profile. It is also available as an optional applicability condition for checks specific to the release review context (section 11.4). +| `is_release_review_pr` | boolean | Detected by framework (profile selection + applicability condition for release-review-specific checks) | ### 2.3 Execution Contexts @@ -186,9 +141,9 @@ The validation framework must support these execution contexts: | **Dispatch (fork repo)** | `workflow_dispatch` | advisory | write (fork scope) | Fork owner triggers on own fork. Inherited — no extra work if dispatch trigger exists | | **Local** | CLI / script | advisory | n/a | No GitHub context; subset of rules | | **Release automation: snapshot** | Called by release workflow | strict | write (app token) | Gate before snapshot creation (section 11.1) | -| **Release automation: review PR** | `pull_request` event (push to PR branch) | strict | write or read-only | Same trigger as normal PR; profile is strict based on release review PR detection (section 11.3) | +| **Release automation: review PR** | `pull_request` event (push to PR branch) | strict | write or read-only | Same trigger as normal PR; profile is strict based on release review PR detection (section 11.2) | -**Profile is independent of token permissions.** A fork-to-upstream PR gets the same **standard** profile as an upstream-branch PR. The read-only token only limits *how results are surfaced* (e.g. no check run annotations via GITHUB_TOKEN), not validation strictness. Alternative output paths for read-only contexts (workflow summary, `pull_request_target`, bot token) are a Session 4 topic. +**Profile is independent of token permissions.** A fork-to-upstream PR gets the same **standard** profile as an upstream-branch PR. The read-only token only limits *how results are surfaced* (e.g. no check run annotations via GITHUB_TOKEN), not validation strictness. **Dispatch is always advisory.** Dispatch runs have nothing to block (except the workflow itself). They surface errors, warnings, and hints for the user to review. @@ -217,8 +172,8 @@ The MVP replaces `pr_validation` v0 and delivers the minimum useful validation o - UC-08, UC-09 (release automation strict gates, section 11) — high priority, requires strict profile; depends on PR integration being operational first - UC-10 (regression testing against release branches) — rule developer tooling - UC-14 (repository configuration validation) — admin tooling -- Automated cache synchronization for `code/common/` and strict version enforcement — bundling itself (ref resolution via `$ref`) is MVP scope (see section 7.3) -- Python-based consistency checks beyond what Spectral covers — Session 2 scope, may partially land in MVP +- Automated cache synchronization for `code/common/` and strict version enforcement — bundling itself (ref resolution via `$ref`) is MVP scope (section 6) +- Python-based consistency checks beyond what Spectral covers — may partially land in MVP ### Independent work (not sequenced with MVP) @@ -249,6 +204,8 @@ The validation rules are derived from the following upstream documents. Rules mu - `release-metadata-schema.yaml` — field definitions for generated release-metadata.yaml on release branches - `api-readiness-checklist.md` — release asset requirements matrix by API status and maturity +The `release-plan-schema.yaml` and `release-metadata-schema.yaml` are the authoritative source for execution context field values. The framework must accept exactly the values defined in these schemas. + ### 4.2 Check Engines | Engine | Scope | Current status | @@ -271,61 +228,11 @@ Each check falls into one of five categories: | **obsolete** | Handled by release automation — validation framework should not re-check (e.g., readiness checklist file management, release tag creation, version field replacement on release branches) | | **existing** | Already implemented in current Spectral/yamllint/gherkin rules. Severity mapping to be determined when individual rules are classified | -### 4.4 Check Inventory Status - -The per-rule inventory is **not yet complete**. The following work is required: - -1. **Commonalities audit** (dependency): Examine `CAMARA-API-Design-Guide.md` and `CAMARA-API-Event-Subscription-and-Notification-Guide.md` at both r3.4 and r4.1/r4.2 versions to: - - Identify checks not yet covered by any engine - - Validate existing Spectral rules against the current design guide - - Identify rules that changed between Commonalities releases - -2. **Existing rule classification**: Map each current Spectral rule to the framework metadata model (applicability, conditional level, hints). The existing Spectral severity levels are assumed valid for now; detailed severity review is deferred. - -3. **api-review v0.6 coverage**: The deprecated `api_review_validator_v0_6.py` (~43 checks) was a monolithic implementation used manually by release reviewers for the Fall25 meta-release. Its checks serve as input for identifying Python-needed validations not covered by Spectral. - -### 4.5 Known Check Areas by Engine - -Summary of check areas identified so far, pending the Commonalities audit for completeness: - -**Spectral (existing + new):** -- OpenAPI version enforcement (3.0.3) -- Naming conventions (paths: kebab-case, schemas: PascalCase, operationId: camelCase) -- Required descriptions (operations, parameters, responses, properties) -- Reserved words detection -- Security: no secrets in path/query parameters -- HTTP method validity, no request body on GET/DELETE -- Unused components detection -- *New*: info.version format (wip/alpha.n/rc.n/public), XCorrelator pattern, phone number format, device object structure - -**Python (new):** -- Server URL version consistency with info.version (cross-field) -- Version must be wip on main, must not be wip on release branches (context-dependent) -- release-plan.yaml non-exclusivity check (PR diff analysis) -- release-plan.yaml schema and semantic validation (existing, to be integrated) -- Error response structure and code compliance (cross-schema) -- Test file existence and version alignment (cross-file) -- CHANGELOG format and link tag-locking (file content analysis) -- API pattern-specific checks: subscription endpoints, CloudEvents format, event type naming (structural + semantic) - -**Manual + prompt:** -- Data minimization compliance (GEN-009, GEN-010) -- Meaningful description quality (beyond presence checks) -- User story adequacy -- Breaking change justification - -**Obsolete (handled by release automation):** -- API Readiness Checklist file management (files should no longer be in the repository) -- Release tag creation and format -- Version field replacement on release branches (wip → actual version) -- release-metadata.yaml generation -- README update with release information - --- ## 5. Rule Architecture -### 5.1 Overview +### 5.1 Processing Model The framework uses a **post-filter and severity mapping** architecture: @@ -335,345 +242,79 @@ The framework uses a **post-filter and severity mapping** architecture: 4. For each finding: look up rule metadata → is it applicable in this context? → what level? → apply profile (advisory/standard/strict) to determine blocking 5. Produce unified output with hints -Individual checks may use the context object as a pre-condition for expensive operations (e.g., release-plan validation only runs if `release_plan_changed` is true on PR trigger, or always on `release-automation` trigger). This is a performance optimization, not a generalized orchestration model. - -### 5.2 Rule Metadata Model - -See section 2.2 for the full metadata model. Key design decisions: +### 5.2 Key Design Decisions - **Flat ID namespace**: Sequential IDs (e.g., `042`) that are stable across engine changes. A rule migrating from Python to Spectral keeps its ID. - **Engine as metadata**: The `engine` field records which engine implements the check. The `engine_rule` field maps to the native rule ID (e.g., Spectral rule name) for configuration synchronization. -- **Condition language**: Plain YAML with AND across fields, OR within arrays. Range comparison on `commonalities_release` uses `packaging.specifiers` (Python). No custom DSL or policy engine. +- **Condition language**: Plain YAML with AND across fields, OR within arrays. Range comparison on `commonalities_release`. No custom DSL or policy engine. - **Per-API evaluation**: API-specific rules are evaluated once per API in the repository, using the API's own `target_api_status`, `target_api_maturity`, and `api_pattern` from release-plan.yaml and OpenAPI spec content. -### 5.3 Spectral Pass-Through Principle - -The framework uses Spectral's severity names (`error`, `warn`, `hint`) as its native level values. This gives identity mapping for the primary engine: - -| Spectral | Framework | Notes | -|----------|-----------|-------| -| `error` | `error` | Identity | -| `warn` | `warn` | Identity | -| `hint` | `hint` | Identity | -| `info` | `hint` | Mapped (rarely used) | -| `off` | `off` | Identity (disable rule) | - -Spectral rules already include `message` fields with fix guidance. Therefore, **Spectral rules that do not need context-dependent severity or applicability filtering do not require explicit framework metadata entries**. Their findings pass through with direct severity mapping and native messages. - -Framework metadata is only needed for Spectral rules when: -- The level should change based on context (e.g., error on release branch, hint on feature branch) -- The rule should be suppressed in certain contexts (applicability filtering) -- The fix hint should be overridden or augmented - -This minimizes the metadata surface: only rules with context-dependent behavior need explicit entries. - -The framework consumes Spectral output as structured data (JSON), not terminal text. This enables programmatic post-filtering, severity remapping, and merging with findings from other engines. - -Spectral does not resolve `$ref` references before linting — it validates the document as-is. Checks that depend on the content of referenced schemas (e.g., from CAMARA_common.yaml) require either a pre-bundled input spec or a Python implementation with explicit ref resolution. See Session 3 (Bundling & Refs) for implications. - -For further Spectral-specific implementation details, see [spectral-integration-notes.md](../reviews/spectral-integration-notes.md). - -### 5.4 Condition Evaluation - -``` -applicability match: - for each field in rule.applicability: - if field is array: context value must be IN the array (OR) - if field is range string: context value must satisfy the range expression - if field is boolean: context value must equal the field value - all fields must match (AND) - omitted fields are unconstrained (always match) - -conditional level: - for each override in rule.conditional_level.overrides (in order): - if override.condition matches context (same logic as applicability): - return override.level - return rule.conditional_level.default -``` - -### 5.5 Derived Context Fields - -Two per-API context fields are derived from content rather than declared in release-plan.yaml: - -**`target_api_maturity`**: Derived from `apis[].target_api_version` — `initial` if major version is 0 (v0.x.y), `stable` if major version >= 1 (vx.y.z). Determines which asset requirements apply per the API Readiness Checklist (e.g., stable public APIs require enhanced test cases and user stories). - -**`api_pattern`**: Detected from OpenAPI spec content — `request-response`, `implicit-subscription`, or `explicit-subscription`. Detection logic examines paths (subscription endpoints), callbacks, schema names, and content types. Multiple pattern-specific rule sets (REQ, IMP, EXP, EVT categories) depend on this classification. This detection is a cross-cutting capability used by many rules. - -### 5.6 Spectral Migration Potential - -Analysis of the deprecated `api_review_validator_v0_6.py` shows that approximately 40% of its checks are implementable as Spectral rules (single-file OpenAPI pattern matching), and an additional 15% could use Spectral custom JavaScript functions. This includes: - -- Mandatory error response checks (400, 401, 403) -- Server URL format validation -- info.version format validation -- License and security scheme validation -- ErrorInfo and XCorrelator schema presence -- Error response structure validation - -The main blocker for migrating ~20% of checks to Spectral is the dependency on `api_pattern` detection — Spectral cannot natively apply rules conditionally based on detected API type. These checks either need custom JS functions that embed the detection logic, or remain as Python checks that use `api_pattern` from the context. - -The Commonalities audit (OQ-05) should evaluate each candidate check against the current design guide version before migration. - -### 5.7 Authoritative Schema References - -The execution context fields and their allowed values are defined by the following schemas, which are the authoritative sources: - -- **release-plan.yaml**: `upstream/traversals/ReleaseManagement/artifacts/metadata-schemas/schemas/release-plan-schema.yaml` -- **release-metadata.yaml**: `upstream/traversals/ReleaseManagement/artifacts/metadata-schemas/schemas/release-metadata-schema.yaml` - -The framework must accept exactly the values defined in these schemas. Any change to the schemas must be reflected in the framework's context model. - ---- - -## 6. Open Questions - -OQ-01: *(Resolved — fork dispatch is inherited automatically if the caller workflow supports dispatch.)* - -OQ-02: *(Resolved — bundling for `CAMARA_common.yaml` via `$ref` is within MVP scope. Automated cache sync and strict version enforcement are post-MVP. See section 7.3.)* - -OQ-03: What is the minimum set of checks for MVP? Just current Spectral + YAML lint, or also a subset of Python checks? *(Partially addressed by Session 2: the Commonalities audit will identify which checks are needed and which are feasible without bundling.)* - -OQ-04: *(Resolved — layered token resolution: validation app bot as default, GITHUB_TOKEN write as fallback, read-only as last resort. Dedicated validation GitHub App separate from camara-release-automation. `pull_request_target` rejected due to security risk at scale. See section 9.1.)* - -OQ-05: Commonalities audit — per-rule validation of existing Spectral rules and identification of missing checks against `CAMARA-API-Design-Guide.md` and `CAMARA-API-Event-Subscription-and-Notification-Guide.md` at r3.4 and r4.1/r4.2 versions. Required before the check inventory can be completed. - -OQ-06: Sub-project common dependency declaration format — should sub-project dependencies (repository, release tag, array of files) be declared within `release-plan.yaml` or as a separate manifest? See section 7.4. - -OQ-07: API-aware change summary tooling — evaluate oasdiff or alternatives for semantic OpenAPI diff summaries. Post-MVP. See section 8.6. - -OQ-08: OIDC token availability for fork PRs — does `id-token: write` work when a fork PR triggers the reusable workflow? If not, the hardcoded version fallback (section 9.5) handles ref resolution, but the exact OIDC behavior should be confirmed empirically as part of WP-01 validation. - --- -## 7. Bundling & Refs +## 6. Bundling & Refs The overall bundling model — controlled local copy on `main`, source-only `main`, bundled release artifacts — is defined in the [Commonalities Consumption and Bundling Design](https://github.com/camaraproject/ReleaseManagement/pull/436) document. This section captures what the validation framework checks about that model, not the model itself. -### 7.1 Repository Model Validation +### 6.1 Repository Model Validation -The framework validates the repository layout and `$ref` patterns used in API source files. +The framework validates the repository layout and `$ref` patterns used in API source files: -#### Layout validation +- **Layout**: When API source files contain `$ref` to `../common/` or `../modules/`, the framework validates that referenced files exist, that `code/common/` contains only cache copies from external repositories, and that `code/modules/` contains project-local reusable schemas. +- **Ref patterns**: API source files must use only relative `$ref` for normative schema consumption. Remote URL-based `$ref` used for normative schema content is an error. Internal component references (`#/components/...`) are always valid. +- **Transition**: Repositories not yet using `code/common/` (current copy-paste model) remain valid. Layout and ref pattern checks only fire when `$ref` to `../common/` or `../modules/` is detected. The framework does not force migration. -When any API source file in `code/API_definitions/` contains a `$ref` to `../common/` or `../modules/`, the framework validates: - -- The referenced file exists at the expected path -- `code/common/` contains only cache copies from other repositories (e.g., `CAMARA_common.yaml` from Commonalities). These files are CI-managed, not manually edited. Content validation is covered in section 7.4. -- `code/modules/` contains project-local reusable schemas used to make the API source more readable and structured. No sync validation is needed for these files. - -#### Ref pattern validation - -- API source files must use only relative `$ref` for normative schema consumption — into `code/common/`, `code/modules/`, or same-directory references -- Remote URL-based `$ref` (e.g., `https://raw.githubusercontent.com/...`) used for normative schema content is an error -- Internal component references (`#/components/...`) are always valid - -#### Transition handling - -Repositories not yet using `code/common/` (i.e., the current copy-paste model with all schemas inline) remain valid. The layout and ref pattern checks only fire when `$ref` to `../common/` or `../modules/` is detected in source files. The framework does not force migration to the local copy model. - -### 7.2 Source vs Bundled Validation +### 6.2 Validation Pipeline The framework applies a uniform validation flow regardless of context: -1. **Pre-bundling validation** (always runs on source files): - - YAML validity (yamllint) - - Ref existence and pattern validation (section 7.1 — do referenced files exist? are refs relative and canonical?) - - Release-plan.yaml consistency checks (Python) - - Cross-file checks that do not depend on schema content (file existence, version field presence) - -2. **Bundling** (if `$ref` to `code/common/` or `code/modules/` is detected): - Resolve all local refs and produce a standalone spec per API. The framework invokes an external bundling tool — it does not implement its own OpenAPI bundler/dereferencer. Tool choice is an implementation detail. - -3. **Full validation** (runs on the *effective input* — bundled output when refs are present, source directly when no refs): - - Full Spectral ruleset — all schemas are inline/resolved - - Python checks that depend on schema content - - Standalone API spec validation - -4. **Artifact surfacing**: Upload bundled specs as workflow artifacts or make them available for further processing (see section 8) - -This flow applies to all contexts — PR, dispatch on any branch type, release automation. The validation **profile** (advisory, standard, strict) controls which findings block; it does not change which steps run. - -#### Bundling failure - -If the bundling step fails (e.g., unresolvable `$ref`, missing file), the framework reports the failure as an error finding. Full validation (step 3) is skipped; only pre-bundling results are available. - -#### Repositories without `$ref` - -Repositories using the current copy-paste model (all schemas inline) skip step 2. Source files are standalone by construction, so the full Spectral ruleset runs directly on source in step 3. No bundling overhead is introduced for repositories that have not adopted the local copy model. - -### 7.3 Spectral and `$ref` Interaction +1. **Pre-bundling validation** (always runs on source files): YAML validity, ref existence and pattern validation, release-plan.yaml consistency checks, cross-file checks that do not depend on schema content +2. **Bundling** (if `$ref` to `code/common/` or `code/modules/` is detected): Resolve all external refs and produce a standalone spec per API. Internal `$ref` (`#/components/...`) are preserved. The framework uses bundling (external ref resolution only), not full dereferencing. +3. **Full validation** (runs on the effective input — bundled output when refs are present, source directly when no refs): Full Spectral ruleset, Python checks that depend on schema content, standalone API spec validation +4. **Artifact surfacing**: Upload bundled specs as workflow artifacts or make them available for further processing (section 7) -#### Bundling vs full dereferencing +The validation **profile** (advisory, standard, strict) controls which findings block; it does not change which steps run. -The framework uses **bundling** (external ref resolution only), not full dereferencing: +**Bundling failure**: If the bundling step fails (e.g., unresolvable `$ref`, missing file), the framework reports the failure as an error. Full validation is skipped; only pre-bundling results are available. -- **Bundling**: Resolves external `$ref` — pulls content from `code/common/`, `code/modules/`, and other local files into the document. Internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`) are preserved. -- **Full dereferencing**: Resolves all `$ref` including internal ones, producing a flat document with zero `$ref` and massive duplication. The framework must **not** use full dereferencing. +**Repositories without `$ref`**: Repositories using the copy-paste model skip step 2. Source files are standalone by construction, so the full Spectral ruleset runs directly on source. No bundling overhead is introduced. -Preserving internal `$ref` ensures that: +**Bundling happens once**: The bundled API specs produced during validation are consumed by release automation for the snapshot branch. Release automation does not re-bundle independently. -- Spectral rules checking component structure, `$ref` patterns, and `#/components/` organization continue to work on bundled output -- Bundled output remains readable and structurally equivalent to what reviewers expect -- No Spectral rule changes are needed between copy-paste and bundled models - -Any constraints on where API designers may use external vs internal `$ref` are defined in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436), not by the validation framework. The framework enforces whatever ref patterns the design document specifies. - -#### Transition period - -During migration from copy-paste to the local copy model, both repository types coexist: - -- **Copy-paste repos**: All schemas inline. Spectral runs directly on source (step 3 of section 7.2). No bundling needed. -- **`$ref` repos**: Spectral runs on bundled output. All external refs resolved, internal refs preserved. Structurally equivalent to copy-paste. - -No rule changes are needed between the two models — bundling normalizes external refs while preserving the internal structure that Spectral rules depend on. Rule IDs remain stable across the transition (flat namespace from section 5.2). - -#### OQ-02 resolution — bundling is MVP scope - -Bundling support for `CAMARA_common.yaml` via `$ref` is **within MVP scope**. Commonalities 0.7.x requires updated common schemas, and the ability to consume them via `$ref` — with the framework handling bundling transparently — is the key additional value of the validation framework v1 for codeowners. This avoids repeating the difficult-to-validate copy-paste pattern. - -In the MVP, some parts may still be manual — providing the correct copy in `code/common/` and ensuring it matches the declared `commonalities_release` version. But the `$ref` option is available for early adopters, and the framework handles bundling when `$ref` is detected. Automated cache synchronization and strict version enforcement are post-MVP enhancements. - -### 7.4 Cache Synchronization and Dependency File Mapping - -#### Cache sync validation +### 6.3 Cache Synchronization When `code/common/` exists, the framework validates that cached files match the declared dependency versions in `release-plan.yaml`. Mismatch severity depends on the profile: - **standard** (PR): warning — codeowner is informed, merge is not blocked - **strict** (release automation): error — snapshot creation is blocked -If no `code/common/` directory exists (repo has not adopted the local copy model), the sync check is skipped. - -In the MVP, cache management may be manual. The validation check still applies — it reports whether the current cache matches the declared dependency, regardless of how the cache was populated. - -#### Dependency categories - -Three categories of shared schema dependencies exist, each with different characteristics: - -**Commonalities** (well-known, hardcoded in tooling): -The Commonalities repository provides shared schemas that are well-known to automation tooling. Currently two files are relevant: -- `CAMARA_common.yaml` — common data types, error responses, headers -- `notification-as-cloud-event.yaml` — CloudEvents notification schema +If no `code/common/` directory exists, the sync check is skipped. In the MVP, cache management may be manual; the validation check still applies regardless of how the cache was populated. -These files, their source location in the Commonalities repository, and the mapping from `release-plan.yaml.dependencies.commonalities_release` to the correct version are built into the tooling. No per-repository configuration is needed. - -**ICM** (version compatibility constraint): -Identity and Consent Management schemas are currently contained within Commonalities files — there are no separate ICM files to cache. The `dependencies.identity_consent_management_release` in `release-plan.yaml` is a version compatibility constraint (potentially `>= x.y.z`) rather than a file-caching relationship. The exact nature of this dependency requires further discussion. - -**Sub-project commons** (extensible, declared per repository): -Sub-projects may define common schemas shared across their API repositories (e.g., a device API family sharing common device type definitions). These dependencies must be declarable without requiring changes to the automation tooling. Each sub-project dependency requires: -- Source repository -- Release tag or version -- Array of files to consume - -This extensible model requires a dependency declaration format, either within `release-plan.yaml` or as a separate manifest. The schema design is a follow-up topic for the bundling design document. - -#### File caching strategy - -Which files are cached in `code/common/` — demand-driven (only files actually `$ref`'d) vs declaration-driven (all files from declared dependencies) — is a sync mechanism concern defined in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436), not a validation framework decision. - -The framework's checks are the same regardless: cached files must match their declared source version, and `$ref` targets must exist. - -### 7.5 Commonalities Version Matrix - -#### Active versions - -The framework must support validation rules that vary by Commonalities version. Active versions at the time of writing: - -- **r3.4** (Commonalities v0.6.x) — Fall25 meta-release, frozen, maintenance releases only -- **r4.x** (Commonalities v0.7.x) — Spring26 meta-release. r4.1 is the release candidate (available now); r4.2 is the upcoming public release and will replace r4.1 - -Within a version line (r4.x), the latest release is always authoritative. When r4.2 is available, r4.1 becomes obsolete — new releases must target r4.2. If a maintenance release r4.3 follows, it replaces r4.2 for validation purposes. - -Future Commonalities major versions (e.g., r5.x for v1.0.0) will add further version lines. The architecture must not assume a fixed number of active versions. - -#### Spectral ruleset selection (pre-selection) - -Each Commonalities major version line gets its own Spectral ruleset (e.g., `.spectral-r3.4.yaml`, `.spectral-r4.yaml`). The framework reads `commonalities_release` from `release-plan.yaml` and selects the matching ruleset before running Spectral. - -This avoids running contradicting rules from different Commonalities versions simultaneously, which would produce confusing Spectral output even if the results were filtered afterwards. The r3.4 ruleset is effectively frozen — only maintenance fixes. New rule development targets the current r4.x ruleset. - -#### Framework rule metadata (post-filter with conditionals) - -Framework rule metadata uses a single ruleset with `commonalities_release` range conditions (defined in section 2.2) for version-specific behavior. This is appropriate because: - -- Python checks are framework-controlled and do not produce confusing intermediate output -- Most framework rules apply across versions; only a minority are version-specific -- Duplicating shared rules into per-version files would create drift risk - -The Commonalities audit (OQ-05) will identify which rules changed between r3.4 and r4.x. Those rules receive `commonalities_release` range conditions in their metadata. - -### 7.6 Placeholder Handling - -#### Current state - -The current `CAMARA_common.yaml` contains placeholder patterns (e.g., `{{SPECIFIC_CODE}}`) that have no defined resolution rules. These should be removed from Commonalities, with API repositories extending shared schemas via `allOf` instead (per the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436)). - -#### Future direction - -Placeholder replacement with defined values could be introduced together with bundling as part of a broader transformation pipeline. This could include dynamic variables such as `api_version`, `commonalities_release`, `commonalities_version`, effectively replacing the current "wip" and "/main/" substitutions done by the snapshot transformer. In this model, bundling + transformation (including placeholder replacement) would produce the release-ready artifact. - -#### Framework requirements +### 6.4 Placeholder Handling - The framework detects unreplaced placeholder patterns (`{{...}}`) in bundled output and reports them as errors — this is a permanent safety net -- As of now, placeholder replacement is not a framework responsibility. If placeholder replacement becomes part of the bundling/transformation pipeline, it is a post-MVP enhancement +- Placeholder replacement is not a framework responsibility. If placeholder replacement becomes part of a bundling/transformation pipeline, it is a post-MVP enhancement - Detection of undefined/unresolvable placeholders remains an error regardless of whether replacement is implemented -### 7.7 Rule Architecture Integration - -Bundling integrates into the rule architecture (section 5) without requiring changes to the context model or rule metadata: - -- **Step assignment**: Each rule runs in either step 1 (pre-bundling validation) or step 3 (full validation). Assignment is an implementation detail — the framework knows which checks belong to which step. -- **No new context fields**: The context model from section 2.2 is sufficient. Whether external refs existed and were resolved is an implementation concern, not a rule applicability condition. -- **Cache sync is a check, not context**: The cache synchronization validation (section 7.4) produces findings (warning or error depending on profile). It is not a context field consumed by other rules. -- **Spectral ruleset selection**: The `commonalities_release` field (already in the context model) drives Spectral ruleset pre-selection (section 7.5). No additional metadata is needed. - -#### Relationship to snapshot creation - -Resolved in section 11.2. The validation framework produces the bundled API specs as part of its validation pipeline (steps 2-3 above). These bundled specs are consumed by release automation for the snapshot branch — bundling happens once, during validation, and the output is handed off to release automation. - -## 8. Artifact Surfacing - -This section covers how bundled artifacts are surfaced for reviewer visibility. Findings surfacing (how validation errors, warnings, and hints are presented) is a separate topic covered in Session 4 (OQ-04). +--- -### 8.1 PR Review Surfaces +## 7. Artifact Surfacing -The [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436) defines a priority order for reviewer visibility. The framework produces: +The framework produces bundled artifacts for reviewer visibility. The [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436) defines a priority order: 1. **Source diff** — primary review surface, no framework action needed (standard git diff) 2. **Bundled artifact** — the framework uploads the bundled standalone API spec as a GitHub workflow artifact for each API affected by the PR 3. **Bundled diff** — a diff between the bundled API from the PR base and the bundled API from the PR head, uploaded as a workflow artifact or included in the workflow summary -4. **API-aware summary** — optional semantic change summary (see section 8.5) - -### 8.2 Workflow Artifacts - -- Bundled specs are uploaded as GitHub workflow artifacts with a naming convention that identifies the API name, branch, and commit SHA -- Bundled files include a header comment: `# For information only - DO NOT EDIT` -- Workflow artifact retention uses the GitHub default (90 days) - -### 8.3 Line Number Mapping - -When checks run on bundled output (step 3 of section 7.2), findings report line numbers in the bundled file. These do not correspond to the source files where the issue should be fixed. The framework must map finding locations in the bundled output back to source file and line number, so that findings are actionable for contributors and codeowners. - -### 8.4 Temporary Branch Model +4. **API-aware summary** — optional semantic change summary (post-MVP) -Workflow artifacts replace the temporary branch model (`/tmp/bundled/-`) for MVP. Temporary branches may be revisited post-MVP if reviewers need a browsable view of bundled content. +**Line number mapping**: When checks run on bundled output, findings report line numbers in the bundled file. The framework must map finding locations back to source file and line number, so that findings are actionable. -### 8.5 "wip" Version Handling - -Bundling on `main` leaves `info.version` as-is — it contains `wip` as expected for unreleased code. Version replacement on release branches is handled by release automation (snapshot transformer), not by the validation framework's bundling step. The framework validates version correctness per branch type — this is an existing check (section 4.5), not a new bundling-specific requirement. - -### 8.6 API-Aware Change Summaries - -The framework should support pluggable diff tools for generating semantic change summaries (breaking changes, new endpoints, modified schemas). oasdiff is a candidate implementation. This capability is post-MVP — source diff, bundled artifact, and bundled diff provide sufficient reviewer visibility for MVP. +**API-aware change summaries**: The framework should support pluggable diff tools for generating semantic change summaries (breaking changes, new endpoints, modified schemas). This capability is post-MVP. --- -## 9. Caller Workflow Design - -### 9.1 Findings Surfacing +## 8. Findings Surfacing -#### Surfacing categories +### 8.1 Output Surfaces Two categories of output are surfaced to users: @@ -692,258 +333,70 @@ Two categories of output are surfaced to users: |----------|-------------| | **Spectral JSON log** | Full structured Spectral output for debugging rule behavior | | **Engine reports** | Detailed per-engine output (replaces v0 MegaLinter report artifacts) | -| **Bundled spec artifacts** | Standalone bundled API specs (see section 8.2) | +| **Bundled spec artifacts** | Standalone bundled API specs (see section 7) | -Diagnostic artifacts are always uploaded as GitHub workflow artifacts regardless of token permissions. They have no fork PR limitation. +Diagnostic artifacts are always uploaded as GitHub workflow artifacts regardless of token permissions. -#### Workflow summary size limit +### 8.2 Summary Size Limit -GitHub limits workflow step summaries to 1 MB per step and 1 MB total per job. For repositories with many APIs or many findings, this limit may be exceeded. The framework must: +GitHub limits workflow step summaries to 1 MB per step and 1 MB total per job. The framework must: - Truncate the summary when approaching the limit, showing a count ("50 of 127 findings shown") with a link to the full diagnostic artifact - Prioritize errors over warnings over hints in the truncated view - Always include the full findings in the Spectral JSON log artifact (no size limitation on workflow artifacts) -#### Token resolution - -The framework uses a layered token resolution strategy. The order prioritizes consistent branding (all findings come from the same bot identity) over using whichever token happens to be available: - -1. **Snapshot context**: `camara-release-automation` app token — provided by the calling release workflow. The validation framework does not mint this token; it is passed in by the release automation caller. -2. **Validation default**: Dedicated validation app bot token — the framework mints an installation token for the current repository. This is the primary path for all PR and dispatch contexts, ensuring consistent bot identity on annotations and comments. -3. **Fallback**: `GITHUB_TOKEN` with write access — used when the validation app is not installed (e.g., dispatch in a fork, or repositories not yet onboarded to the app). Write capability is probed at runtime. -4. **Read-only**: Workflow summary and diagnostic artifacts only — when no write token is available. - -In normal operation, both upstream PRs and fork PRs show findings from the validation bot. The `GITHUB_TOKEN` fallback and read-only mode are degraded paths, not the expected default. - -#### Validation GitHub App - -A dedicated GitHub App handles write surfaces for validation. This is a **separate app** from `camara-release-automation` — it has a narrower permission scope and a different purpose. - -| Aspect | Validation App | camara-release-automation | -|--------|---------------|--------------------------| -| **Purpose** | PR annotations, comments, commit status | Release snapshot creation, branch management | -| **Permissions** | `checks: write`, `pull-requests: write`, `statuses: write` | `contents: write`, `workflows: write`, plus release management | -| **Commits/pushes** | Never | Yes (snapshot branches, tags, release assets) | -| **EasyCLA** | Not needed (no commits) | Required (commits to repos with CLA enforcement) | - -Org-level configuration: -- `vars.VALIDATION_APP_ID` — app ID (org variable) -- `secrets.VALIDATION_APP_PRIVATE_KEY` — app private key (org secret) - -The validation app is introduced from day one (MVP) to establish consistent bot identity and avoid caller workflow changes later. - -#### Surfaces by resolved capability +### 8.3 Surfaces by Resolved Capability | Token capability | Findings output | Diagnostic artifacts | |-----------------|-----------------|---------------------| | **Write** (validation app, camara-release-automation, or GITHUB_TOKEN) | Workflow summary + check run annotations + PR comment + commit status | Workflow artifacts (always) | | **Read-only** (all write paths failed) | Workflow summary only | Workflow artifacts (always) | -#### Relationship to v0 surfacing - -The v0 workflow surfaces findings via MegaLinter's built-in reporters (`GITHUB_COMMENT_REPORTER`, `GITHUB_STATUS_REPORTER`) and custom `actions/github-script` steps for release-plan validation. The v1 framework replaces all of these with its own unified surfacing layer. MegaLinter is no longer used as the orchestration layer. - -### 9.2 Trigger Design - -#### PR trigger - -```yaml -on: - pull_request: - branches: - - main - - release-snapshot/** - - maintenance/** -``` - -- **`main`**: Standard development PRs. Profile: standard (or strict if release review PR detected) -- **`release-snapshot/**`**: Release review PRs created by release automation on snapshot branches. Profile: strict -- **`maintenance/**`**: Maintenance branch PRs. Profile: standard - -Default event types (`opened`, `synchronize`, `reopened`) are sufficient. The framework validates code content, not PR metadata — `edited` (title/body changes) is not needed. - -#### Dispatch trigger - -```yaml - workflow_dispatch: -``` +A dedicated validation GitHub App provides write token capability even for fork PRs where `GITHUB_TOKEN` is read-only. This app is separate from `camara-release-automation` and is introduced from day one (MVP). -Dispatch runs on whatever branch the user selects in the GitHub UI. The framework derives branch type, release context, and all validation parameters from the checked-out branch content. Dispatch inputs are discussed in section 9.4. - -#### Concurrency - -```yaml -concurrency: - group: ${{ github.ref }}-${{ github.workflow }} - cancel-in-progress: true -``` - -Same model as v0: a new push to a PR branch cancels the previous validation run. Dispatch behaves identically — a second dispatch on the same branch cancels the first. - -### 9.3 Permissions - -The caller workflow declares the maximum permission set. The reusable workflow inherits these as a ceiling — it cannot elevate above what the caller declares. - -```yaml -permissions: - checks: write - pull-requests: write - issues: write - contents: read - statuses: write - id-token: write -``` - -| Permission | Purpose | Fork PR behavior | -|------------|---------|-----------------| -| `checks: write` | Check run annotations (findings inline in PR diff) | Restricted by GitHub; validation app token used instead | -| `pull-requests: write` | PR review interactions | Restricted by GitHub; validation app token used instead | -| `issues: write` | PR comments (PRs use the Issues API for comments) | Restricted by GitHub; validation app token used instead | -| `contents: read` | Repository checkout | Available (read-only) | -| `statuses: write` | Commit status (per-check context in checks list) | Restricted by GitHub; validation app token used instead | -| `id-token: write` | OIDC token for tooling ref resolution (WP-01 pattern) | May not be granted for fork PRs — see section 9.5 | - -For fork PRs, `GITHUB_TOKEN` write permissions are restricted by GitHub regardless of what the caller declares. The validation app token (section 9.1) bypasses this restriction because it is minted from the app's own credentials, independent of `GITHUB_TOKEN`. - -### 9.4 Input Design Principle - -The reusable workflow does not accept inputs that duplicate information derivable from the checked-out branch. This prevents contradictions where an input says one thing but branch content says another. - -**Example of the problem avoided**: If the workflow accepted a `release_type` input, a user could dispatch on `main` with `release_type: public-release` while `release-plan.yaml` on `main` says `target_release_type: pre-release-alpha`. The framework would need reconciliation logic, and the user would get confusing results. - -**The rule**: The framework reads branch type from the branch name, and all release context from `release-plan.yaml` on the checked-out branch. These values are derived at runtime, never accepted as inputs. - -**Forbidden inputs**: `branch_type`, `release_type`, `api_status`, `commonalities_version`, `configurations` — all derivable from branch content or from the central configuration file (section 10.2). - -**Consequence for the caller workflow**: No per-repo inputs exist. All per-repo configuration (linting config subfolder, enabled features, rollout stage) lives in the central config file read by the reusable workflow. The caller workflow is identical across all repositories, with no `with:` block needed in standard operation. This makes it protectable via CODEOWNERS or rulesets — nobody ever needs to edit it. - -### 9.5 Ref Resolution - -#### OIDC-based ref resolution (primary) - -The reusable workflow resolves its own tooling repository and commit SHA via OIDC claims (`job_workflow_sha`), following the pattern established in WP-01 (tooling#121). This ensures all internal checkouts (linting config, shared actions at runtime) use the same tooling version that the caller specified. - -The caller workflow declares `id-token: write` to enable OIDC token generation. - -#### Hardcoded version fallback - -If OIDC token generation fails (e.g., fork PRs where `id-token: write` may not be granted), the reusable workflow falls back to its own hardcoded version tag (e.g., `v1`). This is the same pattern as v0's hardcoded `ref: v0`. - -This fallback is acceptable because: -- **Fork PRs**: Contributors do not need pinned-SHA ref resolution — the release version tag (`v1`) is correct for production validation -- **Feature branch testing**: Done by admins and rule developers who have write access, so OIDC works -- **Release automation**: Always triggered by codeowners with write access, so OIDC works - -The fallback means fork PR validation always uses the published version of the tooling, not a feature branch. This is the expected behavior — only admins test unreleased tooling versions. - -#### Break-glass override +--- -`tooling_ref_override` — a 40-character SHA input to the reusable workflow. Takes precedence over both OIDC and the hardcoded fallback. Documented as pilot/break-glass only. Same mechanism as release automation. +## 9. Workflow Contract -#### Resolution order +### 9.1 Input Design Principle -1. `tooling_ref_override` input (if set) — explicit SHA, highest priority -2. OIDC `job_workflow_sha` claim (if `id-token` available) — exact commit SHA -3. Hardcoded version tag in the reusable workflow (e.g., `v1`) — always available +The reusable workflow does not accept inputs that duplicate information derivable from the checked-out branch. The framework reads branch type from the branch name, and all release context from `release-plan.yaml` on the checked-out branch. These values are derived at runtime, never accepted as inputs. -### 9.6 Reusable Workflow Contract +No per-repo inputs exist. All per-repo configuration lives in the central config file (section 10.2) read by the reusable workflow. The caller workflow is identical across all repositories. -#### Inputs +### 9.2 Reusable Workflow Inputs | Input | Type | Required | Default | Description | |-------|------|----------|---------|-------------| -| `tooling_ref_override` | string | no | *(empty)* | 40-character SHA for non-standard tooling ref. Takes highest priority in ref resolution (section 9.5). Supports testing in contexts where OIDC may not work. Documented as pilot/break-glass only. | -| `profile` | choice | no | *(auto)* | Validation profile: `advisory`, `standard`, `strict`. Default: framework selects based on context — advisory for dispatch, standard for PR, strict for release review PR or release automation. Dispatch users can set explicitly to see what a different profile would flag. | - -No per-repo inputs. All per-repo configuration lives in the central config file (section 10.2). - -#### Caller workflow - -The caller workflow is identical across all repositories: - -```yaml -name: CAMARA Validation - -on: - pull_request: - branches: - - main - - release-snapshot/** - - maintenance/** - workflow_dispatch: +| `tooling_ref_override` | string | no | *(empty)* | 40-character SHA for non-standard tooling ref. Supports testing in contexts where OIDC may not work. Documented as pilot/break-glass only. | +| `profile` | choice | no | *(auto)* | Validation profile: `advisory`, `standard`, `strict`. Default: framework selects based on context. Dispatch users can set explicitly to see what a different profile would flag. | -concurrency: - group: ${{ github.ref }}-${{ github.workflow }} - cancel-in-progress: true +### 9.3 Trigger Summary -permissions: - checks: write - pull-requests: write - issues: write - contents: read - statuses: write - id-token: write +| Trigger | Target branches | Default profile | +|---------|----------------|-----------------| +| `pull_request` | `main`, `release-snapshot/**`, `maintenance/**` | standard (strict for release review PRs) | +| `workflow_dispatch` | Any branch | advisory | +| Release automation call | Base branch (`main` or maintenance) | strict | -jobs: - validation: - uses: camaraproject/tooling/.github/workflows/validation.yml@v1 - secrets: inherit -``` - -No `with:` block in standard operation. The caller is a thin pass-through that provides triggers, permissions, and concurrency. All validation logic, configuration, and surfacing are handled by the reusable workflow. - -#### Version tagging - -The reusable workflow uses a floating version tag (`v1`) analogous to v0's `v0` tag. The tag is moved forward as the framework evolves within the v1 major version. Breaking changes (new required permissions, changed caller contract) require a new major version tag. - -#### Secrets - -The caller passes `secrets: inherit`. The reusable workflow uses: -- `GITHUB_TOKEN` — inherited, for checkout and fallback write surfaces -- Org secrets for validation app token minting (`VALIDATION_APP_PRIVATE_KEY`) — accessed via `secrets` context -- Org variables for app identity (`VALIDATION_APP_ID`) — accessed via `vars` context +--- ## 10. Rollout Strategy ### 10.1 v0/v1 Coexistence -The v0 caller (`pr_validation_caller.yml`) and v1 caller (new file) coexist in each repository during the transition period. Both run on every PR and appear in the PR checks list. - -#### Why separate callers - -- The v0 reusable workflow has a fundamentally different structure (MegaLinter-based, single job, different permissions and output model). A single caller with version switching would require complex conditional logic. -- Separate callers allow independent lifecycle: v0 can be removed per-repo after v1 is proven, without coordinating a simultaneous switch. -- GitHub rulesets can reference the v1 check name independently of v0. - -#### Lifecycle per repository +The v0 caller (`pr_validation_caller.yml`) and v1 caller (new workflow file) coexist in each repository during the transition period. Both run on every PR and appear in the PR checks list. Lifecycle per repository: 1. **v1 deployed**: v1 caller added alongside v0. Both run on every PR. Codeowners see both check results. 2. **v1 validated**: Codeowners confirm v1 findings are correct and consistent with expectations. 3. **v1 required**: GitHub ruleset makes the v1 check blocking for PR merge. 4. **v0 removed**: v0 caller file deleted from the repository. Any v0-specific ruleset removed. -The transition from stage 1 to stage 4 is per-repository. Repositories move through stages independently based on pilot experience and codeowner readiness. - -#### WP-01 relationship - -WP-01 (tooling#121) fixes ref consistency in the existing v0 reusable workflow. It validates the OIDC ref resolution pattern that v1 reuses and adds the `tooling_ref_override` break-glass input. WP-01 does not change the v0 caller — callers still call `@v0`. The v1 reusable workflow reuses the same ref resolution pattern with the hardcoded version fallback (section 9.5). +The transition is per-repository. Repositories move through stages independently based on pilot experience and codeowner readiness. ### 10.2 Central Enable/Disable -#### Mechanism: tooling config file - -A YAML configuration file in the tooling repository maps each API repository to its validation settings. The reusable workflow reads this file from its own checkout (which it already performs for linting configs and shared actions). - -**Rationale for config file over alternatives:** - -- **Org variable with repo list** (rejected): JSON arrays in org variables become unwieldy at 60+ repos and hit variable size limits. Not PR-reviewable. -- **Per-repo variable** (rejected): Requires touching each repository to enable. Violates UC-13 — central administration without per-repo configuration changes. -- **Caller version tag** (rejected): Would require editing the caller workflow per-repo, undermining the identical-caller-across-all-repos design (section 9.6). -- **Tooling config file** (chosen): Version-controlled, PR-reviewable, scalable. Adding a repo is one line in a YAML file. Can hold per-repo settings beyond enable/disable (linting config subfolder, rollout stage). Satisfies UC-13 — no per-repo config changes needed. - -The config file schema is an implementation detail. At minimum it maps repository name to rollout stage and per-repo settings (e.g., linting config subfolder previously handled by the `configurations` input in v0). - -#### Repositories not listed +A YAML configuration file in the tooling repository maps each API repository to its validation settings. The reusable workflow reads this file from its own checkout. This is version-controlled, PR-reviewable, and scalable. Adding a repo is one line in a YAML file. Satisfies UC-13 — no per-repo config changes needed. A repository not listed in the config file is treated as `disabled` — the reusable workflow exits immediately with a notice. This is the safe default for the 60+ repos that have the caller deployed but are not yet onboarded. @@ -960,43 +413,6 @@ The config file controls the validation stage per repository: Stage 3 is the combination of the config file setting (`standard`) and a GitHub ruleset. The config file does not control blocking — rulesets do. This separation keeps the config file focused on validation behavior and rulesets focused on merge policy. -### 10.4 GitHub Rulesets for Blocking - -A new ruleset (org-level or per-repo) requires the v1 validation check to pass before PR merge. The pattern follows the existing `release-snapshot-protection` ruleset. - -- The ruleset references the v1 workflow by check name (workflow name or job name) -- The `camara-release-automation` app can be a bypass actor for automated release PRs that need to merge without validation -- Ruleset management can reuse the existing admin script pattern (`apply-release-rulesets.sh`) - -### 10.5 Rollout Sequence - -1. **Test repo**: `ReleaseTest` — full cycle through stages 0-3, validates all surfacing paths -2. **Template**: `Template_API_Repository` — ensures new repos get v1 caller from creation -3. **Pilot API repos**: 2-3 active repos with engaged codeowners -4. **Batch rollout**: Remaining repos, coordinated with v0 removal (section 10.1) - -### 10.6 Feature Branch Testing - -Admins and rule developers test validation changes on feature branches before merging to main and tagging (UC-15). - -The caller workflow in a test repo is temporarily pointed at the feature branch: - -```yaml -uses: camaraproject/tooling/.github/workflows/validation.yml@feature-branch -``` - -Ref resolution (section 9.5) ensures internal checkouts match — admins have write access, so OIDC resolves the exact SHA. `tooling_ref_override` is available as break-glass for composite action changes not on the workflow branch. - -Rule developers can dispatch validation on existing release branches in a test repo while calling the feature-branch version of the reusable workflow. This validates rule changes against known-good content before merging (UC-10). - -No special framework support is needed — pinned refs are a standard GitHub Actions feature. The framework's only requirement is correct ref resolution (section 9.5). - -### 10.7 Caller Update Strategy - -The v1 caller workflow is deployed by copying from `Template_API_Repository` to each API repo. Since the caller is identical across all repos (section 9.6), deployment is a mechanical copy — no per-repo customization. - -Deployment can be batched using the existing admin tooling pattern (scripted multi-repo operations). The caller can be deployed to all repos at once in stage 0 (dark) — it has no effect until the repo is listed in the config file. - --- ## 11. Release Automation Integration @@ -1010,19 +426,13 @@ Both touchpoints use the strict profile (section 2.1): errors and warnings block ### 11.1 Pre-Snapshot Validation Gate -Release automation invokes validation as part of the `/create-snapshot` command, before creating any branches. The validation runs on the current HEAD of the base branch (`main` or maintenance) — this is exactly the content that will become the snapshot. - -**Timing and lifecycle position**: The release state is PLANNED when `/create-snapshot` is invoked. If validation fails, no snapshot branch is created and the state remains PLANNED. The codeowner fixes the reported issues and re-invokes `/create-snapshot`. - -**Profile**: Strict — both errors and warnings block snapshot creation. +Release automation invokes validation as part of the `/create-snapshot` command, before creating any branches. The validation runs on the current HEAD of the base branch — this is exactly the content that will become the snapshot. -**Rationale for strict**: Once a snapshot is created, mechanical transformations are applied (version replacement, server URLs) and the content becomes immutable. Issues found post-snapshot require discarding and recreating the snapshot. Blocking on warnings at snapshot time avoids this expensive retry cycle. +**Timing**: The release state is PLANNED when `/create-snapshot` is invoked. If validation fails, no snapshot branch is created and the state remains PLANNED. -**Scope**: The validation framework runs all applicable checks — Spectral rules, Python consistency checks, bundling validation, and release-plan semantic checks. This subsumes the existing `validate-release-plan.py` preflight. The framework applies rule applicability conditions (section 5) with the full release context derived from `release-plan.yaml` on the checked-out branch. +**Profile**: Strict — both errors and warnings block snapshot creation. Rationale: once a snapshot is created, content becomes immutable. Issues found post-snapshot require discarding and recreating the snapshot. -**Token**: The `camara-release-automation` app token is passed by the release workflow. This is token priority 1 in the layered resolution (section 9.1). The validation framework does not mint this token; it receives it from the caller. - -**Findings output**: Validation findings are reported in the bot's response comment on the Release Issue. The comment includes a structured findings section with error and warning counts, individual findings with fix hints, and a link to the full workflow run for diagnostic artifacts. +**Scope**: The validation framework runs all applicable checks — Spectral rules, Python consistency checks, bundling validation, and release-plan semantic checks. This subsumes the existing `validate-release-plan.py` preflight. | Aspect | Current release-plan preflight | With validation framework | |--------|-------------------------------|---------------------------| @@ -1031,47 +441,13 @@ Release automation invokes validation as part of the `/create-snapshot` command, | **Findings output** | Bot comment with error list | Bot comment with structured findings + fix hints | | **Token** | camara-release-automation app | Same (passed to framework) | -### 11.2 Bundling and Snapshot Interaction - -This subsection resolves the deferred topic from section 7.7: how bundling and validation interact with release automation's snapshot branch creation. - -**The validation framework produces the bundled API specs as part of its validation pipeline (section 7.2, steps 2-3). These bundled specs are the same artifacts that become the release content on the snapshot branch.** Bundling happens exactly once — during validation. Release automation consumes the bundled output rather than re-bundling independently. - -On the snapshot branch, source API definition files (which contain `$ref` to `code/common/` and `code/modules/`) are **replaced** with the bundled standalone specs produced by the validation framework. This is the "swap strategy" described in the [bundling design document](https://github.com/camaraproject/ReleaseManagement/pull/436): the familiar filename (`api-name.yaml`) is retained, but the content is the fully resolved, consumer-ready artifact. - -#### Handoff model - -Two handoff models are viable: - -**Model A — Validation creates the snapshot branch**: The validation framework creates the snapshot branch with bundled API specs already in place. Release automation takes over the branch for mechanical transformations (version replacement, server URLs, `release-metadata.yaml` generation) and the rest of the release lifecycle. - -**Model B — Artifact handoff**: The validation framework uploads bundled specs as workflow artifacts (section 7.2, step 4). Release automation downloads these artifacts and commits them to the snapshot branch as part of snapshot creation. - -Both models avoid duplicate bundling. The choice between them is an implementation detail that depends on workflow architecture constraints. The requirement is: **bundling runs once, during validation, and the output is consumed by release automation for the snapshot.** - -#### Mechanical transformations after bundling - -Regardless of the handoff model, the mechanical transformer applies version-specific changes on top of the bundled content: - -- `info.version` replacement (`wip` → calculated release version) -- Server URL version updates -- `x-camara-commonalities` version field -- Feature file version updates -- Link replacements - -These transformations are release automation's responsibility. The validation framework validates the source content (including bundling); the mechanical transformer produces the final release-ready content. - -#### Cache sync at snapshot time - -A cache synchronization mismatch (section 7.4) is an error in strict profile, blocking snapshot creation. This ensures that `code/common/` content matches the declared `commonalities_release` version before the bundled output is produced and becomes immutable on the snapshot branch. - -### 11.3 Release Review PR Validation +### 11.2 Release Review PR Validation A release review PR is created by release automation on the `release-review/rX.Y-` branch, targeting the `release-snapshot/rX.Y-` branch. It contains only CHANGELOG and README changes — API specs and other files are immutable on the snapshot branch. -**Detection**: The framework detects a release review PR by its target branch pattern (`release-snapshot/**`). The `is_release_review_pr` context field (section 2.2) is set to `true`, which selects the strict profile. +**Detection**: The framework detects a release review PR by its target branch pattern (`release-snapshot/**`). -**Profile**: Strict — same as the pre-snapshot gate. Errors and warnings block the PR. +**Profile**: Strict — errors and warnings block the PR. **Which checks run**: Only a subset of checks is meaningful on a release review PR: @@ -1085,49 +461,18 @@ A release review PR is created by release automation on the `release-review/rX.Y | Bundling validation | No | Source files are immutable | | Cache sync validation | No | Already validated at snapshot creation time | -**File restriction check**: This is a release-review-specific rule. The framework examines the PR diff and produces an error if any file outside `CHANGELOG.md` (or `CHANGELOG/` directory) and `README.md` is modified. This enforces the separation between immutable snapshot content and reviewable documentation. - -**Token**: Standard PR trigger — the validation app token or `GITHUB_TOKEN` fallback. The `camara-release-automation` app token is not used here because this is a regular `pull_request` event, not a release workflow call. - -### 11.4 Context Model Notes - -The context field `is_release_review_pr` (section 2.2) is documented as "used for profile selection, not rule applicability." The file restriction check (section 11.3) requires `is_release_review_pr` as an applicability condition — the check applies only when this field is `true`. - -**Resolution**: `is_release_review_pr` retains its primary role as profile selector. It is also available as an optional applicability condition in rule metadata for checks that are specific to the release review context. This is the only check currently using it as an applicability condition. - -```yaml -# File restriction check — release review PR only -id: "060" -name: release-review-file-restriction -engine: python -applicability: - is_release_review_pr: true -conditional_level: - default: error -description: "Release review PR may only modify CHANGELOG and README files" -hint: "Only CHANGELOG.md (or CHANGELOG/ directory) and README.md may be modified on the release review branch. API specs and other files are immutable on the snapshot branch." -``` - -### 11.5 Pre-Snapshot Invocation - -The pre-snapshot gate follows the same input design principle as PR and dispatch contexts (section 9.4): the framework derives all release context from `release-plan.yaml` on the checked-out branch. No branch name or release tag inputs are needed. - -Release automation invokes the validation framework on the same branch on which `/create-snapshot` was called. The framework reads `release-plan.yaml` from that branch to derive all context fields (target release type, API statuses, Commonalities version, etc.). +**File restriction check**: The framework examines the PR diff and produces an error if any file outside `CHANGELOG.md` (or `CHANGELOG/` directory) and `README.md` is modified. -The only distinction is the `mode` — the framework needs to know this is a pre-snapshot invocation rather than a dispatch or PR trigger. When `mode` is `pre-snapshot`, the framework: -- Sets `trigger_type` to `release-automation` -- Selects the strict profile -- Produces bundled API specs as output for consumption by release automation (section 11.2) -- Formats findings for inclusion in a Release Issue comment (not a PR comment) +### 11.3 Pre-Snapshot Invocation -The detailed output model (findings format, artifact structure) will be defined consistently across all execution contexts during document consolidation. +The pre-snapshot gate follows the same input design principle as PR and dispatch contexts (section 9.1): the framework derives all release context from `release-plan.yaml` on the checked-out branch. The `mode` parameter distinguishes a pre-snapshot invocation from other triggers. -### 11.6 Post-MVP Extensions +### 11.4 Post-MVP Extensions The following integration enhancements are post-MVP: -- **API-aware change summaries for release notes**: The framework could use oasdiff (section 8.6) to generate semantic change summaries (breaking changes, new endpoints, modified schemas) for inclusion in release notes or the Release Issue. -- **Snapshot transformer validation**: The framework could validate the transformer's configuration before snapshot creation — verifying that version replacement patterns and server URL formats are correct. This goes beyond API content validation into release tooling self-validation. +- **API-aware change summaries for release notes**: Semantic change summaries (breaking changes, new endpoints, modified schemas) for inclusion in release notes or the Release Issue. +- **Snapshot transformer validation**: Validating the transformer's configuration before snapshot creation — verifying that version replacement patterns and server URL formats are correct. --- @@ -1140,7 +485,7 @@ An admin needs to verify that an API repository is correctly configured for the - **Caller workflow**: The v1 caller workflow file exists in `.github/workflows/` and matches the expected template content - **Central config listing**: The repository is listed in the tooling config file (section 10.2) with a valid stage value - **GitHub ruleset** (stage 3 only): The v1 validation check is required in the applicable ruleset -- **Validation app installation**: The validation GitHub App (DEC-005) is installed for the repository +- **Validation app installation**: The validation GitHub App is installed for the repository - **v0 cleanup** (post-transition): The v0 caller file has been removed after v1 is stable at stage 3 ### 12.2 Minimal Change Noise Principle @@ -1162,16 +507,16 @@ UC-16 and UC-17 are **independent work** — the Release Progress Tracker alread **Data collection**: The tracker queries the GitHub API for the most recent completed run of the v1 validation workflow on the default branch (conclusion, URL, timestamp). No changes to the validation framework are needed for this. -**On-demand trigger** (UC-17): The tracker or an admin can dispatch validation on selected repositories via `workflow_dispatch` (section 9.2) to update dashboard status. +**On-demand trigger** (UC-17): The tracker or an admin can dispatch validation on selected repositories via `workflow_dispatch` to update dashboard status. ### 12.4 Cross-System Integration Map | System | Integration point | Data flow | Priority | |--------|------------------|-----------|----------| | Release automation (`/create-snapshot`) | Pre-snapshot gate (11.1) | Validation → release automation: pass/fail + findings + bundled specs | Post-MVP (high) | -| Release automation (release review PR) | PR trigger with strict profile (11.3) | Standard PR validation flow, profile auto-selected | Post-MVP (high) | +| Release automation (release review PR) | PR trigger with strict profile (11.2) | Standard PR validation flow, profile auto-selected | Post-MVP (high) | | Release Progress Tracker | Workflow run query (12.3) | Tracker reads validation run data from GitHub API | Independent | | Release Progress Tracker | Dispatch trigger (12.3) | Tracker dispatches validation via GitHub API | Independent | | Tooling config file | Central enable/disable (10.2) | Validation reads config at runtime | MVP | -| GitHub rulesets | Blocking enforcement (10.4) | Ruleset references validation check name | Stage 3 | -| Validation GitHub App | Token minting (9.1) | App provides write token for findings surfacing | MVP | +| GitHub rulesets | Blocking enforcement (10.3) | Ruleset references validation check name | Stage 3 | +| Validation GitHub App | Token minting (8.3) | App provides write token for findings surfacing | MVP | From bc4bc97680569b35ce7daaec62a7b950382dfde4 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Wed, 18 Mar 2026 20:27:08 +0100 Subject: [PATCH 04/25] docs: add end-to-end processing flow and output pipeline (Session 7) Add sections 8 and 9 to the detailed design document, completing the workflow architecture needed for implementation: - Section 8: job architecture, checkout strategy, context builder, engine orchestration, composite action boundaries, common findings model - Section 9: post-filter processing, profile application, output formatting, truncation strategy, line number mapping, error handling, bundled spec artifact handoff (Model B) Also updates: - Section 6.2: central config schema with fork_owners allowlist - Section 7.1: settle on artifact handoff model (Model B) - Section 7.4: clarify mode input for release automation invocation - Requirements section 9.2: add mode input to workflow inputs table --- ...RA-Validation-Framework-Detailed-Design.md | 667 +++++++++++++++++- ...AMARA-Validation-Framework-Requirements.md | 1 + 2 files changed, 658 insertions(+), 10 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 01b59a9..bfe851a 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -1,7 +1,7 @@ # Validation Framework — Detailed Design **Status**: Work in progress -**Last updated**: 2026-03-17 +**Last updated**: 2026-03-18 > This document supplements the [Validation Framework Requirements](CAMARA-Validation-Framework-Requirements.md) with design and implementation detail for developers and architects. It is expected to migrate to the `tooling` repository alongside the implementation. @@ -519,7 +519,46 @@ Separate callers allow independent lifecycle: v0 can be removed per-repo after v - **Caller version tag** (rejected): Would require editing the caller workflow per-repo, undermining the identical-caller-across-all-repos design (section 5.7). - **Tooling config file** (chosen): Version-controlled, PR-reviewable, scalable. Adding a repo is one line in a YAML file. Can hold per-repo settings beyond enable/disable (linting config subfolder, rollout stage). Satisfies UC-13 — no per-repo config changes needed. -The config file schema is an implementation detail. At minimum it maps repository name to rollout stage and per-repo settings (e.g., linting config subfolder previously handled by the `configurations` input in v0). +#### Central config file schema + +The central config file lives in the tooling repository and maps each API repository to its rollout stage. Spectral ruleset selection is **not** a per-repo config field — it is derived from `commonalities_release` in the repository's own `release-plan.yaml` (section 3.3). + +```yaml +# validation-config.yaml in camaraproject/tooling +version: 1 +defaults: + stage: disabled # default for repositories not listed below +fork_owners: [hdamker, rartych] # GitHub users allowed to test in their forks +repositories: + QualityOnDemand: + stage: standard # stage 2: runs on PRs, standard profile + DeviceLocation: + stage: standard + ReleaseTest: + stage: standard + NetworkSliceBooking: + stage: advisory # stage 1: dispatch only +``` + +| Field | Type | Description | +|-------|------|-------------| +| `version` | integer | Schema version (currently `1`). Allows future schema evolution without breaking existing configs. | +| `defaults.stage` | enum | Default stage for unlisted repositories: `disabled`, `advisory`, `standard`. | +| `fork_owners` | array of strings | GitHub usernames allowed to run validation in their forks. When the workflow runs in a fork owned by a listed user, stage is overridden to `standard` regardless of the repository's upstream stage (section 8.2). | +| `repositories..stage` | enum | Per-repo rollout stage override. Same values as `defaults.stage`. | + +**Stage mapping** (see also Requirements section 10.3): + +| Stage | Config value | Behavior | +|-------|-------------|----------| +| 0 (dark) | `disabled` | Caller deployed but reusable workflow exits immediately | +| 1 (advisory) | `advisory` | Runs on dispatch only, advisory profile, nothing blocks | +| 2 (standard) | `standard` | Runs on PRs and dispatch, standard profile on PRs | +| 3 (blocking) | `standard` + ruleset | Same as stage 2; blocking is enforced by a GitHub ruleset, not the config file | + +**Extensibility**: Additional per-repo fields (e.g., `features`, optional overrides) can be added without a `version` bump — new fields are additive. Future candidates include `spectral_ruleset_override` and `extra_checks`. + +**Self-validation**: The reusable workflow validates the config file against a JSON Schema on every run. An invalid config file is a hard failure with an explicit error message naming the file and the problematic entry. This catches typos, unknown stage values, and schema drift before any validation logic runs. ### 6.3 GitHub Rulesets for Blocking @@ -574,17 +613,18 @@ On the snapshot branch, source API definition files (which contain `$ref` to `co #### Handoff model -Two handoff models are viable: +The validation framework uploads bundled specs as **workflow artifacts** (artifact handoff). Release automation downloads these artifacts and commits them to the snapshot branch as part of snapshot creation. See section 9.7 for the detailed handoff sequence. -**Model A — Validation creates the snapshot branch**: The validation framework creates the snapshot branch with bundled API specs already in place. Release automation takes over the branch for mechanical transformations (version replacement, server URLs, `release-metadata.yaml` generation) and the rest of the release lifecycle. +This model was chosen over the alternative (validation creates the snapshot branch directly) because: +- The validation reusable workflow's checkout is ephemeral — there is no mechanism to hand uncommitted file changes to a different branch without committing and pushing from within the validation job +- Validation would need `contents: write` permission and knowledge of snapshot branch naming conventions, creating tight coupling to release automation internals +- The artifact model keeps validation stateless: it produces files and reports results, release automation owns repository state -**Model B — Artifact handoff**: The validation framework uploads bundled specs as workflow artifacts. Release automation downloads these artifacts and commits them to the snapshot branch as part of snapshot creation. - -Both models avoid duplicate bundling. The choice between them is an implementation detail that depends on workflow architecture constraints. The requirement is: **bundling runs once, during validation, and the output is consumed by release automation for the snapshot.** +Bundling runs once, during validation. Release automation consumes the bundled output without re-bundling. #### Mechanical transformations after bundling -Regardless of the handoff model, the mechanical transformer applies version-specific changes on top of the bundled content: +After release automation downloads the bundled artifacts, the mechanical transformer applies version-specific changes on top of the bundled content: - `info.version` replacement (`wip` → calculated release version) - Server URL version updates @@ -625,10 +665,617 @@ hint: "Only CHANGELOG.md (or CHANGELOG/ directory) and README.md may be modified Release automation invokes the validation framework on the same branch on which `/create-snapshot` was called. The framework reads `release-plan.yaml` from that branch to derive all context fields (target release type, API statuses, Commonalities version, etc.). -The only distinction is the `mode` — the framework needs to know this is a pre-snapshot invocation rather than a dispatch or PR trigger. When `mode` is `pre-snapshot`, the framework: +The only distinction is the `mode` input — a reusable workflow input (alongside `tooling_ref_override` and `profile` from Requirements section 9.2) that tells the framework this is a pre-snapshot invocation rather than a dispatch or PR trigger. Release automation passes `mode: pre-snapshot` when calling the validation workflow. + +When `mode` is `pre-snapshot`, the framework: - Sets `trigger_type` to `release-automation` - Selects the strict profile - Produces bundled API specs as output for consumption by release automation (section 7.1) - Formats findings for inclusion in a Release Issue comment (not a PR comment) -The detailed output model (findings format, artifact structure) will be defined consistently across all execution contexts during implementation. +The detailed output model (findings format, artifact structure) is defined in section 9. + +--- + +## 8. End-to-End Processing Flow + +This section describes the reusable workflow's internal structure and the validation engine's processing pipeline. It covers what happens from the moment the workflow starts to the point where raw findings are collected — output formatting and surfacing are in section 9. + +### 8.1 Job Architecture + +The reusable workflow uses a **single-job design**. All steps run sequentially within one job. + +This differs from release automation's multi-job architecture. Release automation splits into separate jobs because its phases have fundamentally different conditional logic (trigger classification → state derivation → command validation → command execution, where each command is a separate job). The validation workflow has a single linear pipeline — context flows naturally between steps via environment variables and the shared file system within one job. Multi-job would require serializing the context object through job outputs and re-checking out the repository in each job, adding complexity without benefit. + +#### Step sequence + +| # | Step | Composite action | Skip condition | +|---|------|-----------------|----------------| +| 1 | Checkout repository content | — (inline) | Never | +| 2 | Resolve tooling ref and checkout tooling | `resolve-tooling-ref` | Never | +| 3 | Read central config and check stage | — (inline) | Never (exits here if `disabled`) | +| 4 | Build validation context | `build-validation-context` | Never | +| 5 | Pre-bundling validation | `run-pre-bundling-checks` | `is_release_review_pr` (section 8.6) | +| 6 | Bundling | `run-bundling` | No external `$ref` detected; or `is_release_review_pr` | +| 7 | Full validation | `run-full-validation` | `is_release_review_pr` (runs subset instead — section 8.6) | +| 8 | Post-filter and output | `process-findings` | Never (always produces at least a summary) | + +Steps 5–7 are the engine orchestration phase (section 8.4). Step 8 is the output pipeline (section 9). + +### 8.2 Checkout Strategy + +#### Repository content checkout + +```yaml +- uses: actions/checkout@v6 + with: + fetch-depth: 0 +``` + +Full history (`fetch-depth: 0`) is needed for: +- PR diff analysis: identifying which files changed (used by `release_plan_changed` detection and file restriction check) +- Branch type detection: examining `github.base_ref` for PR triggers + +For dispatch triggers, `fetch-depth: 1` would suffice, but a single checkout configuration avoids trigger-specific branching. + +#### Tooling checkout + +The tooling repository is checked out via sparse checkout at the resolved ref (section 5.6): + +```yaml +- uses: actions/checkout@v6 + with: + repository: camaraproject/tooling + ref: ${{ steps.resolve-ref.outputs.tooling_sha }} + sparse-checkout: | + linting/config + validation + shared-actions + path: .tooling +``` + +The checkout lands in `.tooling/` within the workspace. This path is used by all subsequent steps to locate linting configs, validation scripts, shared actions, and the central config file. + +The sparse checkout scope includes: +- `linting/config/` — Spectral rulesets (per Commonalities version), yamllint config, gherkin-lint config +- `validation/` — Python validation scripts, rule metadata, central config file +- `shared-actions/` — composite actions consumed at runtime + +#### Central config gate + +Immediately after the tooling checkout, the workflow reads the central config file (section 6.2) and looks up the current repository: + +1. Validate the config file against its JSON Schema +2. Extract the repository name from `github.repository` **without the owner prefix** (e.g., `camaraproject/QualityOnDemand` → `QualityOnDemand`) +3. Look up `repositories..stage` (fall back to `defaults.stage`) +4. **Fork override**: If the workflow is running in a fork (`github.repository_owner` is not `camaraproject` or `GSMA-Open-Gateway`) and the owner is listed in `fork_owners` → override stage to `standard`. If the owner is not listed → keep the resolved stage (typically `disabled` during early rollout, which exits the workflow) +5. If stage is `disabled`: exit the workflow with a notice in the summary ("Validation is not enabled for this repository") and set the overall result to `skipped` +6. If stage is `advisory` and trigger is `pull_request`: exit similarly ("Validation is in advisory mode — use workflow_dispatch to run") +7. Otherwise: continue, passing the resolved stage to subsequent steps + +The fork override (step 4) enables trusted testers to run full validation in their forks even before the upstream repository has been onboarded. This is especially useful during early rollout when most repos are still at `disabled`. Once upstream repos move to `standard`, the `fork_owners` list becomes less relevant — forks inherit the upstream stage. + +### 8.3 Context Builder + +The context builder assembles a unified context object from multiple sources. It follows the same principle as release automation's `BotContext`: all fields are always present in the output, with missing or inapplicable values defaulting to empty strings or `null`. Downstream consumers never need to handle missing keys. + +#### Branch type derivation + +| Pattern | Branch type | Source | +|---------|-------------|--------| +| `main` | `main` | `github.base_ref` (PR) or `github.ref_name` (dispatch) | +| `release-snapshot/**` | `release` | Same | +| `maintenance/**` | `maintenance` | Same | +| Everything else | `feature` | Same | + +For `pull_request` triggers, branch type is derived from the PR's **target branch** (`github.base_ref`), not the source branch. This determines which validation rules apply to the content being merged. + +For `workflow_dispatch`, it is derived from the checked-out branch (`github.ref_name`). + +#### Trigger type derivation + +| Source | Trigger type | +|--------|-------------| +| `github.event_name == 'pull_request'` | `pr` | +| `github.event_name == 'workflow_dispatch'` | `dispatch` | +| `mode` input == `pre-snapshot` | `release-automation` | + +The `mode` input (section 7.4) is the only way the trigger type `release-automation` is set. In standard caller workflow operation, `mode` is not set and trigger type comes from the GitHub event. + +#### Profile selection + +| Trigger type | Branch type | Condition | Profile | +|-------------|-------------|-----------|---------| +| `dispatch` | any | — | `advisory` | +| `pr` | `release` | `is_release_review_pr` = true | `strict` | +| `pr` | any | — | `standard` | +| `release-automation` | any | — | `strict` | + +If the `profile` input (Requirements section 9.2) is explicitly set, it overrides the auto-selected profile. This allows dispatch users to preview what a different profile would flag. + +#### release-plan.yaml parsing + +The context builder reads `release-plan.yaml` from the checked-out branch and validates it against the release-plan JSON Schema (section 1.6). Parsing extracts: + +- `target_release_type` — determines release context for all rules +- `dependencies.commonalities_release` — drives Spectral ruleset selection (section 3.3) +- `dependencies.identity_consent_management_release` — version constraint (section 3.2) +- Per-API fields: `api_name`, `target_api_version`, `target_api_status` + +If `release-plan.yaml` is absent (valid for repositories not yet using release automation, or feature branches), the context builder produces a minimal context with `target_release_type: null`. Release-plan-specific checks are skipped; Spectral and yamllint still run. If the file is present but invalid, schema violations are reported as findings and processing continues with whatever fields could be parsed. + +#### Derived per-API fields + +For each API declared in `release-plan.yaml`: + +- **`target_api_maturity`**: Derived from `target_api_version` — `initial` if major version is 0, `stable` if >= 1 (section 1.4) +- **`api_pattern`**: Detected from the corresponding OpenAPI spec file content in `code/API_definitions/` (section 1.4). Detection runs during context building because many downstream rules depend on it + +#### Spectral ruleset selection + +The Commonalities version declared in `release-plan.yaml` (`dependencies.commonalities_release`) determines which Spectral ruleset is used (section 3.3). The framework maps the release tag to a version line: + +- `commonalities_release` matching `r3.*` → `.spectral-r3.4.yaml` +- `commonalities_release` matching `r4.*` → `.spectral-r4.yaml` +- Future version lines added as new rulesets are created +- If `commonalities_release` is absent or unresolvable: default to the latest ruleset (currently r4.x) + +This selection is derived from the repository's own `release-plan.yaml`, not from the central config. Different branches of the same repo can target different Commonalities versions. + +#### Detection of PR-specific context + +- **`is_release_review_pr`**: True when the PR targets a `release-snapshot/**` branch (section 7.3) +- **`release_plan_changed`**: True when `release-plan.yaml` is in the PR diff. Detected via the GitHub API's changed files list for the PR. Relevant for the release-plan non-exclusivity check (section 2.2) + +#### Fork scenarios + +The validation workflow can run in forks as well as upstream: + +- **Fork dispatch**: A codeowner dispatches validation on their fork to check work before creating an upstream PR. Org secrets are not available in forks, so the validation app token cannot be minted — token resolution falls back to `GITHUB_TOKEN`, which has full write access within the fork's own context. +- **Fork-internal PRs**: PRs within a fork (feature branch → fork's main) trigger the caller if deployed. Same degraded token situation as fork dispatch. +- **Fork-to-upstream PRs**: The standard contribution path. The workflow runs in the upstream repo context (triggered by `pull_request`), so `github.repository` is the upstream repo. Org secrets are available to the reusable workflow. + +The context builder does not expose fork identity as a context field — no validation rule needs it. Fork vs. upstream is a workflow-level concern handled by token resolution (section 5.1). + +#### Context object structure + +```yaml +# Validation context — all fields always present +repository: "QualityOnDemand" # repo name without owner prefix +branch_type: "main" # main | release | maintenance | feature +trigger_type: "pr" # pr | dispatch | release-automation +profile: "standard" # advisory | standard | strict +stage: "standard" # from central config (disabled | advisory | standard) + +# Release context (from release-plan.yaml; null if absent) +target_release_type: "pre-release-rc" +commonalities_release: "r4.1" +icm_release: ">= r1.0" + +# PR-specific (null for dispatch) +is_release_review_pr: false +release_plan_changed: true +pr_number: 42 + +# Per-API contexts (array; empty if no release-plan.yaml) +apis: + - api_name: "qos-booking" + target_api_version: "1.0.0-rc.1" + target_api_status: "maintained" + target_api_maturity: "stable" # derived + api_pattern: "request-response" # detected from spec + spec_file: "code/API_definitions/qos-booking.yaml" + - api_name: "qos-on-demand" + target_api_version: "0.11.0-alpha.1" + target_api_status: "initial" + target_api_maturity: "initial" + api_pattern: "explicit-subscription" + spec_file: "code/API_definitions/qos-on-demand.yaml" + +# Workflow metadata +workflow_run_url: "https://github.com/camaraproject/QualityOnDemand/actions/runs/12345" +tooling_ref: "abc1234def5678..." +``` + +The context object is serialized as JSON and made available to subsequent steps via an environment variable or step output. + +### 8.4 Engine Orchestration + +Engines run **sequentially** within the single job. Each step captures its findings and passes them to the post-filter. + +Sequential execution is appropriate because: +- GitHub Actions steps within a job are inherently sequential +- Parallel execution would require multi-job with artifact passing — overhead that exceeds the time saved for a 1-3 minute pipeline +- Later steps may depend on earlier results (bundling produces input for Spectral) + +#### Step 5: Pre-bundling validation + +Runs on **source files** before any ref resolution. These checks must work regardless of whether the repo uses `$ref` or copy-paste: + +| Check | Engine | Target files | +|-------|--------|-------------| +| YAML validity | yamllint | `code/API_definitions/*.yaml` | +| `$ref` existence and pattern validation | Python | `code/API_definitions/*.yaml` | +| release-plan.yaml semantic checks | Python | `release-plan.yaml` | +| Cross-file checks not dependent on schema content | Python | Various | + +yamllint uses a configuration from the tooling checkout (`linting/config/.yamllint.yaml`). Invalid YAML produces error-level findings that block subsequent steps — there is no point running Spectral on syntactically invalid files. + +If all API spec files are syntactically valid and no external `$ref` is detected, step 6 (bundling) is skipped and step 7 runs on the source files directly. + +#### Step 6: Bundling + +Conditional — only runs when external `$ref` to `code/common/` or `code/modules/` is detected in at least one API spec file. + +The framework invokes an **external bundling tool** — it does not implement its own OpenAPI bundler. The tool must satisfy two requirements: + +1. **External ref resolution only** (DEC-002): Resolve `$ref` to `code/common/`, `code/modules/`, and other local files. Preserve all internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`). Full dereferencing must not be used. +2. **Source map production**: Produce a mapping from bundled output regions back to source file locations. This is needed for line number translation in the output pipeline (section 9.5). This is a selection criterion for tool evaluation — the chosen tool must either support source maps natively or be wrappable to produce them. + +The specific tool choice (e.g., redocly, swagger-cli, prance, custom wrapper) is deferred to implementation, evaluated against these two requirements. + +**Cache sync validation** runs as part of bundling: the content of `code/common/` files is compared against the expected content for the declared `commonalities_release` version. A mismatch produces a finding — `warning` in standard profile, `error` in strict profile. + +**On bundling failure** (unresolvable `$ref`, missing file, tool error): +- Report bundling error as a finding +- Skip step 7's Spectral run on bundled output (Spectral cannot lint incomplete specs) +- Proceed to step 8 with pre-bundling findings only +- Overall result: `error` (incomplete evaluation) + +#### Step 7: Full validation + +Runs on the **effective input** — bundled specs (from step 6) if bundling ran, otherwise source specs. + +| Check | Engine | Configuration | Output format | +|-------|--------|--------------|---------------| +| Spectral linting | Spectral | Version-selected ruleset (section 3.3) | JSON (`--format json`) | +| Test definition linting | gherkin-lint | `linting/config/.gherkin-lintrc` | Text or JSON | +| Cross-field consistency | Python | Rule metadata (section 1) | Common findings model | +| Version checks | Python | Context + spec content | Common findings model | +| Error response structure | Python | Context + spec content | Common findings model | +| API pattern-specific checks | Python | Context + spec content + `api_pattern` | Common findings model | + +**Spectral** is invoked with `--format json` to capture structured output for programmatic post-processing: + +```bash +spectral lint \ + --ruleset .tooling/linting/config/.spectral-r4.yaml \ + --format json \ + code/API_definitions/*.yaml \ + > spectral-output.json +``` + +The JSON output provides per-finding: `code` (rule name), `path` (file), `message`, `severity` (0=error, 1=warn, 2=info, 3=hint), `range.start.line`, `range.start.character`. + +**Python checks** produce findings directly in the common findings model (section 8.4.1). They receive the full context object and have access to the file system for cross-file analysis. + +**gherkin-lint** validates test definition files in `code/Test_definitions/`. Its findings are normalized into the common model by the framework. + +#### 8.4.1 Common Findings Model + +All engine outputs are normalized into a common findings format before post-filtering: + +```yaml +- rule_id: "042" # framework rule ID (sequential, stable) + engine: spectral # spectral | yamllint | gherkin | python + engine_rule: "camara-parameter-casing-convention" # native engine rule name + level: error # engine-reported level (before post-filter) + message: "Path segment 'qualityOnDemand' should be kebab-case" + path: "code/API_definitions/qos-on-demand.yaml" + line: 47 # line in source file (mapped back if bundled) + column: 5 # column (if available from engine) + api_name: "qos-on-demand" # which API this finding belongs to + hint: "Use kebab-case: /quality-on-demand/{sessionId}" +``` + +| Field | Source — Spectral | Source — yamllint | Source — Python | +|-------|------------------|------------------|-----------------| +| `rule_id` | Looked up from rule metadata by `engine_rule`; auto-assigned if no metadata | Looked up similarly | Set directly by check | +| `engine` | `"spectral"` | `"yamllint"` | `"python"` | +| `engine_rule` | `code` field from JSON | Rule name from output | Check function name | +| `level` | Mapped from severity integer: 0→error, 1→warn, 2→hint, 3→hint | Mapped from yamllint severity | Set directly | +| `message` | `message` field | Error message text | Set directly | +| `path` | `source` field | File path from output | Set directly | +| `line` | `range.start.line` (0-indexed → 1-indexed) | Line number from output | Set directly | +| `column` | `range.start.character` | Column from output | Set directly (or null) | +| `api_name` | Derived from file path | Derived from file path | Set directly | +| `hint` | From rule metadata `hint` field (if present); otherwise engine `message` serves as hint | From rule metadata | Set directly | + +Spectral rules **without** explicit framework metadata entries pass through with identity mapping (section 1.3): `rule_id` is auto-assigned, `hint` defaults to the engine's `message`, and the level maps directly. This means the check inventory does not need to be complete before the framework can run — new Spectral rules work immediately. + +### 8.5 Composite Action Boundaries + +Composite actions encapsulate logic that is independently testable, reusable across contexts, or benefits from encapsulation (e.g., Python scripts with dependencies). Simple shell commands and conditional logic remain as inline workflow steps. + +| Action | Purpose | Key inputs | Key outputs | +|--------|---------|-----------|-------------| +| `resolve-tooling-ref` | OIDC-based ref resolution (section 5.6) | `tooling_ref_override` | `tooling_sha` | +| `build-validation-context` | Context assembly (section 8.3) | Repository checkout path, tooling path, central config | Context JSON, stage, profile | +| `run-pre-bundling-checks` | Pre-bundling validation (section 8.4 step 5) | Context JSON, source files | Findings JSON | +| `run-bundling` | External ref resolution + cache sync (section 8.4 step 6) | Context JSON, source files, cache dir | Bundled specs, source map, findings JSON | +| `run-full-validation` | Spectral + gherkin + Python checks (section 8.4 step 7) | Context JSON, effective input files, ruleset path | Findings JSON | +| `process-findings` | Post-filter + output formatting (section 9) | Context JSON, all findings JSONs, token | Summary, annotations, comment, status | + +The `validate-release-plan` action from release automation is reused (enhanced to produce findings in the common model) rather than reimplemented. Other release automation shared actions (`resolve-tooling-ref`) are consumed directly. + +The boundary between "one large action" and "several small actions" per engine is an implementation choice. The table above shows the logical boundaries; implementation may merge or split actions based on testability and maintenance considerations. + +### 8.6 Release Review PR Short Circuit + +When `is_release_review_pr` is true, the processing flow is shortened: + +- **Skip**: Steps 5 (pre-bundling), 6 (bundling), and the Spectral/gherkin/most-Python portions of step 7 +- **Run**: CHANGELOG format check, README validation, file restriction check (section 7.3) +- **Rationale**: API specs are immutable on the snapshot branch — Spectral checks would be redundant. Release-plan and cache sync were already validated at snapshot creation time (Requirements section 11.2) + +This is implemented as a conditional skip within the engine orchestration steps, not a separate workflow path. The context builder sets `is_release_review_pr` and subsequent steps check it before executing. + +--- + +## 9. Output Pipeline + +This section covers step 8 of the processing flow (section 8.1): post-filter processing, output formatting, truncation, line number mapping, and error handling. It takes the raw findings from all engines and produces the user-visible output. + +### 9.1 Post-Filter Processing + +The post-filter evaluates each raw finding against the rule metadata (section 1) and the current validation context (section 8.3). It produces a filtered, severity-adjusted findings list ready for output formatting. + +**Processing steps per finding:** + +1. **Rule metadata lookup**: Match the finding to its framework rule metadata entry by `engine` + `engine_rule`. Python-produced findings already carry a `rule_id` and skip this step. + +2. **Applicability evaluation**: Apply the rule's `applicability` conditions against the current context (section 1.2). If any condition does not match, the finding is silently removed — it does not apply in this context. + +3. **Conditional level resolution**: Apply `conditional_level` overrides against the context (section 1.2). The first matching override determines the resolved level; if none match, the default level is used. The resolved level replaces the engine-reported level. If the resolved level is `off`, the finding is removed. + +4. **Pass-through**: Spectral rules without explicit framework metadata entries pass through with identity mapping (section 1.3). Their engine-reported severity becomes the resolved level, and the engine's `message` serves as the hint. This is the common case for most Spectral rules — the framework runs without requiring a complete metadata inventory. + +**Per-API evaluation**: Findings associated with a specific API (identified by `api_name`) are evaluated against that API's context fields (`target_api_status`, `target_api_maturity`, `api_pattern`). Repository-level findings (e.g., release-plan.yaml checks) are evaluated against the repository-level context. + +### 9.2 Profile Application and Blocking Decision + +After post-filtering, each finding has a resolved level. The active profile (section 8.3) determines which levels block: + +| Profile | Blocking levels | Typical context | +|---------|----------------|-----------------| +| `advisory` | None — nothing blocks | Dispatch (stage 1) | +| `standard` | `error` | PR validation (stage 2) | +| `strict` | `error` and `warn` | Pre-snapshot, release review PR | + +**Overall result** — one of three values: + +| Result | Meaning | +|--------|---------| +| `pass` | No blocking findings | +| `fail` | At least one finding at a blocking level | +| `error` | An engine failure prevented complete evaluation (section 9.6) — even if no blocking findings were collected, the result is uncertain | + +**Finding grouping for output**: Findings are grouped for display in the following order: by resolved level (error → warn → hint), then by API name, then by file path, then by line number. This ensures the most actionable items appear first. + +### 9.3 Output Formatting + +The framework produces output on multiple surfaces, each with different capabilities and token requirements. All surfaces are generated from the same filtered findings list. + +#### Workflow summary (always available) + +The workflow summary is the primary detailed surface. It requires no write token — it is written via `$GITHUB_STEP_SUMMARY` and is always available, even for fork PRs with read-only tokens. + +Structure: + +```markdown +## CAMARA Validation — {result} + +**Profile**: {profile} | **Branch**: {branch_type} | **Trigger**: {trigger_type} + +### Summary + +| API | Errors | Warnings | Hints | +|-----|--------|----------|-------| +| qos-booking | 0 | 2 | 1 | +| qos-on-demand | 1 | 0 | 3 | + +### Findings + +#### Errors + +| Rule | File | Line | Message | Hint | +|------|------|------|---------|------| +| 042 | qos-on-demand.yaml | 47 | Path segment should be kebab-case | Use: /quality-on-demand | + +#### Warnings +... + +#### Hints +... + +### Engine Status + +| Engine | Status | Findings | +|--------|--------|----------| +| Spectral | Completed | 4 | +| yamllint | Completed | 0 | +| Python | Completed | 3 | + +--- +Commit: abc1234 | Tooling: def5678 | [Full workflow run]({workflow_run_url}) +``` + +#### Check run annotations (write token required) + +One annotation per finding, mapped to the source file and line number: + +- Annotation level: `failure` for error, `warning` for warn, `notice` for hint +- Annotation message: includes the rule ID, engine rule name, and fix hint +- Annotation title: rule name from metadata (or engine rule name if no metadata) + +GitHub limits annotations to **50 per step**. If more findings exist: +- Prioritize: errors first, then warnings, then hints +- Note the truncation in the workflow summary: "Showing 50 of N findings as annotations. See workflow summary for the complete list." + +#### PR comment (write token required) + +A concise summary comment on the PR — not a full findings list. Links to the workflow summary for details. + +```markdown +### CAMARA Validation — {result} + +{errors} errors, {warnings} warnings, {hints} hints | Profile: {profile} + +[View full results]({workflow_run_url}) +``` + +The comment uses a create-or-update pattern with a marker (``) to avoid duplicate comments on subsequent pushes. Each new push updates the existing comment rather than creating a new one. + +**Pre-snapshot context**: When invoked by release automation (`mode: pre-snapshot`), findings are formatted for inclusion in the bot's Release Issue comment (section 7.2) rather than as a standalone PR comment. The framework returns the formatted findings section to the calling workflow. + +#### Commit status (write token required) + +Commit statuses provide at-a-glance results in the PR's checks list: + +- **Overall status**: Context `CAMARA Validation`, state = `success` / `failure` / `error` +- **Per-engine statuses** (optional, for diagnostics): `CAMARA Validation / Spectral`, `CAMARA Validation / yamllint`, etc. — each shows the engine's individual pass/fail state + +The overall status is the one referenced by GitHub rulesets for blocking (stage 3). + +### 9.4 Truncation Strategy + +GitHub limits workflow step summaries to **1 MB per step** and **1 MB total per job**. The framework must stay within this limit. + +**Approach**: Estimate the rendered size during summary generation. If the cumulative size approaches 900 KB: + +1. Show all **errors** first — never truncated +2. Show **warnings** up to the remaining budget +3. If budget exhausted before all warnings are shown: display a count of remaining findings and link to the full report +4. **Hints** are shown only if budget permits after errors and warnings + +```markdown +> Showing 85 of 214 findings. Full Spectral output available in +> [workflow artifacts]({artifact_url}). +``` + +The full Spectral JSON output and the complete findings list (all engines) are always uploaded as **workflow artifacts** regardless of summary truncation. These artifacts are the authoritative complete record. + +### 9.5 Line Number Mapping + +When Spectral runs on bundled output (section 8.4, step 7), finding line numbers reference the bundled file, not the original source file. Annotations and summary tables must show source file locations to be actionable for developers. + +**Requirement on the bundling tool**: The external bundling tool (section 8.4, step 6) must produce — or be augmented to produce — a source map that records which regions of the bundled file originated from which source file and line range. This is a selection criterion for bundling tool evaluation, alongside external-ref-only resolution (DEC-002). + +**Design choice**: Content pulled in from external refs (e.g., schemas from `CAMARA_common.yaml`) maps back to the **`$ref` line in the source file**, not to the external file itself. The `$ref` declaration is the actionable location — it is the line the developer controls. If Spectral reports an issue at line 247 of the bundled file, and that region was pulled from an external ref declared at line 15 of the source file, the finding is reported at source line 15. + +**Scope**: Line number mapping is only needed for repositories that use `$ref` to `code/common/` or `code/modules/`. Copy-paste repositories have identity mapping — source and effective input are the same file. + +The source map format is an implementation detail. A line-offset table per external ref insertion point is sufficient for MVP. + +### 9.6 Error Handling + +**Design principle**: Always surface what succeeded; never silently skip. If an engine fails, the failure is reported explicitly and remaining engines continue. + +#### Engine failure + +When an engine crashes (Spectral error, Python exception, missing dependency): + +1. Catch the failure in the step (non-zero exit code or exception) +2. Record an engine-level error finding: + ```yaml + - rule_id: "engine-failure" + engine: spectral # the engine that failed + engine_rule: null + level: error + message: "Spectral exited with code 2: Cannot read ruleset file" + path: null + line: null + api_name: null + hint: "Check the workflow log for details" + ``` +3. Continue with remaining engines +4. Set overall result to `error` (distinct from `fail` — signals incomplete evaluation) +5. Workflow summary explicitly lists which engines succeeded and which failed (engine status table in section 9.3) + +Following the release automation pattern: error messages are shown in code blocks, immediately visible, not collapsed. The workflow run URL links to full logs. + +#### Config file missing or invalid + +- Config file **missing** from the tooling checkout: hard failure with an explicit error in the summary. This is a tooling repository issue, not a per-repo issue. +- Config file **invalid** (bad YAML, unknown stage value, schema violation): hard failure naming the config file and the problematic entry. The framework does not proceed with potentially incorrect configuration. + +#### release-plan.yaml missing or invalid + +- **Missing**: The context builder produces a minimal context with `target_release_type: null`. Release-plan-specific checks (version consistency, non-exclusivity, dependency validation) are skipped. Spectral and yamllint still run on source files. This is not an error — repos without release-plan.yaml (or feature branches that haven't added one) are valid. +- **Invalid** (present but fails schema validation): Schema violations are reported as findings. The context builder extracts whatever fields it can and continues with a partial context. Remaining checks run with the available context. + +#### Bundling failure + +When bundling fails (unresolvable `$ref`, missing file in `code/common/`, tool crash): + +1. Report the bundling error as a finding (error level) +2. Skip Spectral on bundled output (step 7 Spectral) — cannot lint incomplete specs +3. Proceed to post-filter with pre-bundling findings (step 5 results) and the bundling error +4. Set overall result to `error` if the active profile would have required checks on bundled output + +Pre-bundling findings are still valuable — yamllint and ref pattern checks provide feedback even when bundling fails. + +#### Token minting failure + +When the validation app token cannot be minted (app not installed, secret missing, API error): + +1. Follow the layered token resolution (section 5.1) — fall back to `GITHUB_TOKEN`, then to read-only +2. Log the degraded state in the workflow summary header: "Write surfaces unavailable — showing findings in workflow summary only" +3. **Never** fail the workflow because of a token issue — validation results are always produced, only the surfacing degrades +4. Check run annotations, PR comments, and commit statuses are silently skipped when no write token is available + +#### Failure mode summary + +| Failure | Behavior | Overall result | +|---------|----------|---------------| +| Engine crash | Record engine-failure finding, continue with remaining engines | `error` | +| Config file missing/invalid | Hard failure, explicit error in summary | `error` (workflow exits) | +| release-plan.yaml missing | Minimal context, skip release-plan checks, run Spectral/yamllint | Normal (`pass`/`fail`) | +| release-plan.yaml invalid | Report violations as findings, partial context, continue | Normal (`pass`/`fail`) | +| Bundling failure | Report error, skip bundled-output checks, show pre-bundling findings | `error` | +| Token minting failure | Degrade surfacing, never fail the workflow | Normal (`pass`/`fail`) | + +### 9.7 Bundled Spec Artifacts + +Bundled API specs produced in step 6 (section 8.4) are a distinct output category from findings — they are build artifacts, not validation results. This section covers how they are persisted and consumed. + +#### Artifact upload + +After bundling completes (step 6), the bundled spec files are uploaded as **GitHub workflow artifacts**, regardless of whether validation passes or fails. Bundled specs are useful for review even when findings exist. + +Artifact naming follows a convention that identifies the content: + +``` +validation-bundled-specs-{commit-sha-short} +``` + +The artifact contains one bundled YAML file per API (retaining the original filename, e.g., `qos-booking.yaml`) plus the source map file produced by the bundling tool (section 9.5). + +Artifacts use the GitHub default retention period (90 days). They are available for download from the workflow run's artifact list. + +#### User surfacing + +The workflow summary (section 9.3) includes a link to the bundled spec artifacts when bundling ran: + +```markdown +### Bundled Specs + +Bundled standalone API specs are available as [workflow artifacts]({artifact_url}). +``` + +This gives PR reviewers visibility into the bundled output — they can download and inspect the fully resolved specs to verify that `$ref` resolution produced the expected result. For copy-paste repositories (no bundling), this section is omitted. + +#### Release automation handoff (Model B) + +When release automation invokes validation with `mode: pre-snapshot`, it consumes the bundled spec artifacts to populate the snapshot branch. The handoff uses the **artifact model** (section 7.1, Model B): + +1. Release automation calls the validation reusable workflow with `mode: pre-snapshot` +2. Validation runs the full pipeline (context → pre-bundling → bundling → full validation → output) +3. Validation uploads bundled specs as workflow artifacts +4. Validation returns its overall result (`pass` / `fail` / `error`) to the caller +5. If result is `pass`: release automation downloads the bundled spec artifacts and commits them to the snapshot branch, replacing the source files that contained `$ref` +6. If result is `fail` or `error`: release automation does not create the snapshot — findings are reported in the Release Issue comment + +The validation framework does not need `contents: write` permission and has no knowledge of snapshot branch naming. It produces files and reports results; release automation decides what to do with them. This keeps a clean separation: validation is stateless, release automation owns the repository state. + +**For non-pre-snapshot runs** (PR and dispatch triggers): bundled specs are uploaded as artifacts for reviewer inspection only. No branch creation or file replacement occurs — the artifacts are informational. diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md index 9c861ff..6b968c5 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md @@ -370,6 +370,7 @@ No per-repo inputs exist. All per-repo configuration lives in the central config |-------|------|----------|---------|-------------| | `tooling_ref_override` | string | no | *(empty)* | 40-character SHA for non-standard tooling ref. Supports testing in contexts where OIDC may not work. Documented as pilot/break-glass only. | | `profile` | choice | no | *(auto)* | Validation profile: `advisory`, `standard`, `strict`. Default: framework selects based on context. Dispatch users can set explicitly to see what a different profile would flag. | +| `mode` | string | no | *(empty)* | Execution mode. Set to `pre-snapshot` by release automation to invoke validation as a pre-snapshot gate. Affects trigger type derivation, profile selection, bundled spec handoff, and findings output target. | ### 9.3 Trigger Summary From 5ff2df3c2848350b499d509574e5e17c155ddec6 Mon Sep 17 00:00:00 2001 From: Herbert Damker Date: Thu, 19 Mar 2026 08:14:21 +0100 Subject: [PATCH 05/25] Update documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md Co-authored-by: Tanja de Groot <87864067+tanjadegroot@users.noreply.github.com> --- .../CAMARA-Validation-Framework-Detailed-Design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index bfe851a..587c9c8 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -160,7 +160,7 @@ Summary of check areas identified so far, pending the Commonalities audit for co - Security: no secrets in path/query parameters - HTTP method validity, no request body on GET/DELETE - Unused components detection -- *New*: info.version format (wip/alpha.n/rc.n/public), XCorrelator pattern, phone number format, device object structure +- *New*: info.version format (wip/alpha.n/rc.n/public), XCorrelator pattern, phone number format, device object structure, no info.contact field, check externalDocs format **Python (new):** - Server URL version consistency with info.version (cross-field) From 31bc02816fb6c98e693f1105553d9032fb528910 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Thu, 19 Mar 2026 08:28:33 +0100 Subject: [PATCH 06/25] docs: expand naming conventions examples in check areas summary --- .../CAMARA-Validation-Framework-Detailed-Design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 587c9c8..e0d3b42 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -154,7 +154,7 @@ Summary of check areas identified so far, pending the Commonalities audit for co **Spectral (existing + new):** - OpenAPI version enforcement (3.0.3) -- Naming conventions (paths: kebab-case, schemas: PascalCase, operationId: camelCase) +- Naming conventions (e.g. paths: kebab-case, schemas: PascalCase, operationId: camelCase, enums: UPPER_SNAKE_CASE, headers: kebab-case) - Required descriptions (operations, parameters, responses, properties) - Reserved words detection - Security: no secrets in path/query parameters From 3af1df02f46d00505a15bb8f0d69b6dee8b1d5a5 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Thu, 19 Mar 2026 09:15:21 +0100 Subject: [PATCH 07/25] docs: add design guide audit and update check inventory (Session 8) - Add CAMARA-Validation-Framework-Design-Guide-Audit.md: comprehensive audit of 106 machine-checkable rules from both design guides, cross- referenced against existing Spectral rules, OWASP rules (tooling#95), Linting-rules.md, and api_review_validator_v0_6.py - Update section 2.2 with complete check areas by engine - Add Appendix A: naming conventions (full list from design guide) - Clean internal tracking identifiers from document --- ...Validation-Framework-Design-Guide-Audit.md | 410 ++++++++++++++++++ ...RA-Validation-Framework-Detailed-Design.md | 96 +++- 2 files changed, 482 insertions(+), 24 deletions(-) create mode 100644 documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md new file mode 100644 index 0000000..0af3704 --- /dev/null +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md @@ -0,0 +1,410 @@ +# Commonalities Design Guide Audit + +**Date**: 2026-03-19 +**Scope**: Machine-checkable rules from CAMARA-API-Design-Guide.md and CAMARA-API-Event-Subscription-and-Notification-Guide.md +**Versions compared**: r3.4 (Commonalities 0.5) vs r4.1 (Commonalities 0.6) + +## Methodology + +1. Walked both design guides section by section, extracting every machine-checkable requirement +2. Cross-referenced against: Linting-rules.md, current .spectral.yaml (17 CAMARA rules + core OAS), tooling#95 OWASP rules, api_review_validator_v0_6.py (80 checks) +3. Compared r3.4 tag against current r4.1 for version sensitivity + +## Legend + +**Coverage column**: +- `spectral: ` — rule exists in current .spectral.yaml +- `owasp: ` — rule exists in tooling#95 (not merged) +- `linting-rules.md: ` — listed in Linting-rules.md but NOT in .spectral.yaml +- `v0_6: V6-NNN` — covered by api_review_validator_v0_6.py +- `core-oas: ` — spectral:oas built-in rule +- `gap` — no current implementation + +**v1 Engine**: `spectral` / `python` / `manual` / `obsolete` + +**Version**: `both` / `r4.x-only` / `changed` / `r3.4-only` + +--- + +## Section 2: Data Types + +### 2.2 Data Type Constraints + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-001 | String: maxLength or enum MUST be used | owasp: api4:2023-string-limit (warn) | spectral | r4.x-only | New MUST in r4.1. Target severity: error | +| DG-002 | String: format/pattern/enum/const SHOULD be used | owasp: api4:2023-string-restricted (warn) | spectral | both | Advisory | +| DG-003 | date-time description MUST state RFC 3339 with timezone | gap | spectral | both | Regex on description field | +| DG-004 | duration description MUST state RFC 3339 | gap | spectral | both | Regex on description field | +| DG-005 | Array: maxItems MUST be specified | owasp: api4:2023-array-limit (warn) | spectral | r4.x-only | New MUST in r4.1. Target severity: error | +| DG-006 | Integer: format (int32/int64) MUST be specified | owasp: api4:2023-integer-format (warn) | spectral | r4.x-only | Was informal in r3.4. Target: error | +| DG-007 | Integer: minimum and maximum MUST be specified | owasp: api4:2023-integer-limit-legacy (warn) | spectral | r4.x-only | Was informal in r3.4. Target: error | +| DG-008 | Object: required properties MUST be listed | gap | spectral | both | Check for `required` array on object schemas | +| DG-009 | type attribute MUST be present on all properties | spectral: camara-schema-type-check (error) | spectral | both | | + +### 2.2.1 Discriminator + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-010 | oneOf/anyOf SHOULD include discriminator | spectral: camara-discriminator-use (hint) | spectral | changed | Deprecated in r4.1 (was warn in r3.4) | + +--- + +## Section 3: Errors and Responses + +### 3.1 Business-level Outcomes (new in r4.1) + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-011 | contextCode values SHOULD follow API_NAME.SPECIFIC_CODE in SCREAMING_SNAKE_CASE | gap | python | r4.x-only | New section. Check if contextCode field is used | + +### 3.2 Error Responses + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-012 | Error response MUST have status, code, message fields | v0_6: V6-033 | python | both | ErrorInfo schema structure. Becomes spectral with bundling (ref to CAMARA_common.yaml) | +| DG-013 | Error code MUST NOT be numeric | gap | spectral | both | Regex on error code enum values | +| DG-014 | Error code MUST be SCREAMING_SNAKE_CASE | gap | spectral | r4.x-only | New explicit requirement in r4.1 | +| DG-015 | API-specific error codes MUST follow API_NAME.SPECIFIC_CODE | gap | spectral | both | Pattern: `^[A-Z][A-Z0-9_]*\.[A-Z][A-Z0-9_]*$`. Clarified in r4.1 | +| DG-016 | All APIs MUST document 401 response | owasp: api8:2023-define-error-responses-401 (error) | spectral | both | | +| DG-017 | All APIs MUST document 403 response | gap | spectral | both | OWASP only covers 401, not 403 | +| DG-018 | CONFLICT error code is DEPRECATED | gap | python | r4.x-only | Warn if API uses CONFLICT | +| DG-019 | Error response content-type MUST be application/json | v0_6: V6-028 | spectral | both | | +| DG-020 | Error responses MUST reference ErrorInfo or allOf containing ErrorInfo | v0_6: V6-029 | python | both | Needs $ref resolution | + +--- + +## Section 5: API Definition + +### 5.1 Reserved Words + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-021 | No reserved words in paths, params, operationIds, components, security schemes | spectral: camara-reserved-words (warn) | spectral | both | | +| DG-022 | Resource names MUST NOT contain HTTP method names | linting-rules.md: camara-resource-reserved-words (warn) | spectral | both | **DISCREPANCY**: listed in Linting-rules.md but NOT in .spectral.yaml | + +### 5.2 OpenAPI Version + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-023 | OpenAPI version MUST be 3.0.3 | spectral: camara-oas-version (error) | spectral | both | | + +### 5.3 Info Object + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-024 | info.title MUST NOT include "API" | v0_6: V6-003; linting-rules.md: camara-info-title (tbd) | spectral | both | **DISCREPANCY**: Linting-rules.md marks "tbd", not in .spectral.yaml | +| DG-025 | info.version MUST follow semver/wip/alpha/rc | v0_6: V6-004; linting-rules.md: camara-info-version-format (tbd) | spectral | both | **DISCREPANCY**: Linting-rules.md marks "tbd", not in .spectral.yaml | +| DG-026 | info.license.name MUST be "Apache 2.0" | v0_6: V6-005 | spectral | both | | +| DG-027 | info.license.url MUST be Apache URL | v0_6: V6-006; core-oas: license-url | spectral | both | | +| DG-028 | x-camara-commonalities MUST specify version | v0_6: V6-007 | spectral | both | | +| DG-029 | info.description MUST contain "Authorization and authentication" section | v0_6: V6-010/V6-011 | python | both | Normalized text matching | +| DG-030 | info.description MUST contain "Additional CAMARA error responses" section | v0_6: V6-012/V6-013 | python | r4.x-only | New in v0.6 | +| DG-031 | info.termsOfService MUST be absent | v0_6: V6-008 | spectral | both | | +| DG-032 | info.contact MUST be absent | gap | spectral | both | core-oas has contact-properties disabled | + +### 5.4 External Docs + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-033 | externalDocs MUST be present | v0_6: V6-014 | spectral | both | | +| DG-034 | externalDocs.description MUST match expected text | v0_6: V6-015 | spectral | both | "Product documentation at CAMARA" | +| DG-035 | externalDocs.url MUST be https://github.com/camaraproject/{repo} | v0_6: V6-016/V6-017 | spectral | both | | + +### 5.5 Servers + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-036 | At least one server MUST be defined | core-oas: oas3-api-servers | spectral | both | | +| DG-037 | Server URL MUST follow {apiRoot}/api-name/api-version | v0_6: V6-020/V6-057 | python | both | Cross-field: URL pattern vs info.version | +| DG-038 | All servers MUST have same api-name and api-version | v0_6: partial | python | both | | + +### 5.6 Tags + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-039 | Each operation SHOULD use exactly one tag | core-oas: operation-singular-tag (warn) | spectral | both | | +| DG-040 | Operation tags MUST be defined in global tags | core-oas: operation-tag-defined (warn) | spectral | both | | +| DG-041 | Tag names SHOULD use Title Case (with spaces) | gap | spectral | both | No casing rule for tags exists | + +### 5.7.1 Paths + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-042 | Paths MUST use kebab-case | spectral: camara-parameter-casing-convention (error) | spectral | both | | +| DG-043 | Path parameters MUST NOT be generic {id} | spectral: camara-path-param-id (warn) | spectral | both | | +| DG-044 | Path parameter morphology SHOULD be consistent ({entityId}) | linting-rules.md: camara-path-param-id-morphology (warn) | spectral | both | **DISCREPANCY**: in Linting-rules.md but NOT in .spectral.yaml | +| DG-045 | No trailing slash on paths | core-oas: path-keys-no-trailing-slash | spectral | both | | +| DG-046 | No query string in path | core-oas: path-not-include-query | spectral | both | | + +### 5.7.2 Operations + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-047 | Operations MUST have operationId | core-oas: operation-operationId | spectral | both | | +| DG-048 | operationId MUST use camelCase | spectral: camara-operationid-casing-convention (hint) | spectral | both | **SEVERITY**: .spectral.yaml=hint, Linting-rules.md=error | +| DG-049 | Operations MUST have summary | spectral: camara-operation-summary (warn) | spectral | both | | +| DG-050 | Operations MUST have description | spectral: camara-routes-description (warn) | spectral | both | | +| DG-051 | operationId MUST be unique | core-oas: operation-operationId-unique (error) | spectral | both | | +| DG-052 | Valid HTTP methods only (GET/PUT/POST/PATCH/DELETE/OPTIONS) | spectral: camara-http-methods (error) | spectral | both | | +| DG-053 | Operations MUST have at least one 2xx/3xx response | core-oas: operation-success-response | spectral | both | | + +### 5.7.4 Parameters + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-054 | All parameters MUST have description | spectral: camara-parameters-descriptions (warn) | spectral | both | | +| DG-055 | Property names MUST use lowerCamelCase | linting-rules.md: camara-property-casing-convention (error) | spectral | both | **DISCREPANCY**: in Linting-rules.md but NOT in .spectral.yaml | + +### 5.7.5 Request Bodies + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-056 | GET/DELETE MUST NOT have requestBody | spectral: camara-get-no-request-body (error) | spectral | both | | + +### 5.7.6 Responses + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-057 | All responses MUST have description | spectral: camara-response-descriptions (warn) | spectral | both | | +| DG-058 | Array response items MUST have description | gap | spectral | r4.x-only | New requirement in r4.1 | + +### 5.8.1 Schemas + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-059 | Schema names MUST use PascalCase | spectral: camara-schema-casing-convention (warn) | spectral | both | | +| DG-060 | All schema properties MUST have description | spectral: camara-properties-descriptions (warn) | spectral | both | | + +### 5.8.5 Headers + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-061 | x-correlator header MUST be included | v0_6: V6-032 | spectral | both | Check parameter/header presence | +| DG-062 | x-correlator pattern MUST match expected regex | v0_6: V6-062 | spectral | both | Pattern: `^[a-zA-Z0-9-_:;.\/<>{}]{0,256}$` | + +### 5.8.6 Security Schemes + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-063 | MUST use openIdConnect scheme named 'openId' | v0_6: V6-036/V6-041 | spectral | both | | +| DG-064 | openIdConnectUrl MUST use HTTPS | v0_6: V6-039 | spectral | both | | +| DG-065 | openIdConnectUrl MUST point to .well-known/openid-configuration | v0_6: V6-040 | spectral | both | | +| DG-066 | No oauth2 scheme type (must use openIdConnect) | v0_6: V6-042 | spectral | both | | + +--- + +## Section 6: Security + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-067 | HTTPS must always be used (no HTTP) | owasp: api8:2023-no-scheme-http + no-server-http (error) | spectral | both | | +| DG-068 | Sensitive data (MSISDN/IMSI/phoneNumber) MUST NOT be in path/query params | spectral: camara-security-no-secrets-in-path-or-query-parameters (warn) | spectral | both | | +| DG-069 | Write operations MUST have security scheme | owasp: api2:2023-write-restricted (error) | spectral | r4.x-only | | +| DG-070 | Read operations SHOULD have security scheme | owasp: api2:2023-read-restricted (warn) | spectral | r4.x-only | | +| DG-071 | Short-lived access tokens with refresh | owasp: api2:2023-short-lived-access-tokens (error) | spectral | r4.x-only | | +| DG-072 | No credentials in URL parameters | owasp: api2:2023-no-credentials-in-url (error) | spectral | r4.x-only | | +| DG-073 | No numeric IDs (use UUIDs or random) | owasp: api1:2023-no-numeric-ids (error) | spectral | r4.x-only | | +| DG-074 | Admin endpoints: unique security scheme | owasp: api5:2023-admin-security-unique (error) | spectral | r4.x-only | | +| DG-075 | No unconstrained additionalProperties | owasp: api3:2023-no-additionalProperties (warn) | spectral | r4.x-only | | +| DG-076 | Constrained additionalProperties | owasp: api3:2023-constrained-additionalProperties (warn) | spectral | r4.x-only | | +| DG-077 | 401 response MUST be defined on all operations | owasp: api8:2023-define-error-responses-401 (error) | spectral | both | | +| DG-078 | Error validation response (400/422/4XX) SHOULD be defined | owasp: api8:2023-define-error-validation (warn) | spectral | r4.x-only | | + +### 6.6 Scope Naming + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-079 | Scopes MUST follow api-name:[resource:]action pattern | v0_6: V6-050 | python | both | Context-dependent (subscription vs regular) | +| DG-080 | Subscription POST scopes: api-name:event-type:create | v0_6: V6-051 | python | both | | +| DG-081 | Subscription GET scopes: api-name:read | v0_6: V6-052 | python | both | | +| DG-082 | Subscription DELETE scopes: api-name:delete | v0_6: V6-053 | python | both | | + +--- + +## Section 7: Versioning (context-dependent) + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-083 | info.version MUST be wip on main branch | v0_6: V6-054 | python | both | Requires branch context | +| DG-084 | info.version MUST NOT be wip on release branches | v0_6: V6-056 | python | both | Requires branch context | +| DG-085 | Server URL version MUST match info.version | v0_6: V6-057 | python | both | Cross-field validation | + +--- + +## Naming Conventions (complete list) + +Consolidated from design guide sections. This is the full set for the detailed design appendix. + +| Element | Convention | Spectral Name | DG Section | Coverage | +|---------|-----------|--------------|------------|----------| +| Paths (URLs) | kebab-case | camara-parameter-casing-convention | 5.7.1 | Implemented (error) | +| Path parameters | lowerCamelCase (entityId form) | camara-path-param-id, camara-path-param-id-morphology | 5.7.1 | Partial (morphology not in .spectral.yaml) | +| Schemas | PascalCase | camara-schema-casing-convention | 5.8.1 | Implemented (warn) | +| operationId | camelCase | camara-operationid-casing-convention | 5.7.2 | Implemented (hint; should be error per Linting-rules.md) | +| Properties | lowerCamelCase | camara-property-casing-convention | 5.7.4 | **Not in .spectral.yaml** (Linting-rules.md: error) | +| Enum values | UPPER_SNAKE_CASE (macro) | camara-enum-casing-convention | 3.2 | **Not in .spectral.yaml** (Linting-rules.md: info, tbd) | +| Tags | Title Case (with spaces) | — | 5.7.3 | **Gap** — no rule exists | +| Headers | kebab-case | — | 5.8.5 | **Gap** — implied by x-correlator convention | +| Scope names | kebab-case with : separators | — | 6.6 | v0_6 only (V6-050) | +| API name | kebab-case | — | 1.2 | v0_6 only (V6-058/V6-059) | +| Event type | org.camaraproject.\.\.\ | — | Event Guide 3.1 | **Gap** | +| Error codes | SCREAMING_SNAKE_CASE | — | 3.2 | **Gap** (r4.x-only explicit) | +| API-specific error codes | API_NAME.SPECIFIC_CODE | — | 3.2.1 | **Gap** | + +--- + +## Event Subscription and Notification Guide + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-086 | Event type MUST follow org.camaraproject.\.\.\ | gap (v0_6 partial) | python | both | | +| DG-087 | specversion MUST be "1.0" enum | gap | spectral | both | | +| DG-088 | Subscription API filename MUST append "subscriptions" | gap | python | both | Filesystem check | +| DG-089 | Explicit subscriptions: 4 operations (POST/GET/GET{id}/DELETE) | v0_6: V6-065/V6-066 | python | both | | +| DG-090 | protocol attribute MUST be "HTTP" | gap | spectral | both | Enum constraint | +| DG-091 | sink MUST use HTTPS URI | gap | spectral | both | | +| DG-092 | sinkCredential MUST NOT appear in responses | gap | python | both | | +| DG-093 | notificationsBearerAuth scheme for callbacks | v0_6: V6-037/V6-044–046/V6-048 | python | both | | +| DG-094 | Notification content-type: application/cloudevents+json | gap | spectral | both | | +| DG-095 | Event version independent of API version | gap | python | both | Complex versioning logic | +| DG-096 | Implicit subscriptions: callbacks defined in operations | v0_6: V6-067 | python | both | | + +--- + +## Cross-File Checks + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-097 | Common schemas identical across API files | v0_6: V6-068 | python/obsolete | both | Becomes partially obsolete with bundling | +| DG-098 | License identical across API files | v0_6: V6-069 | python | both | | +| DG-099 | x-camara-commonalities identical across API files | v0_6: V6-070 | python | both | | +| DG-100 | Test directory exists | v0_6: V6-071 | python | both | | +| DG-101 | Test files exist for each API | v0_6: V6-072 | python | both | | +| DG-102 | Test file version alignment | v0_6: V6-073 | python | both | | +| DG-103 | Filename uses kebab-case | v0_6: V6-058 | python | both | | +| DG-104 | Filename matches api-name in server URL | v0_6: V6-059 | python | both | | + +--- + +## Telco Language and Spelling + +| ID | Rule | Coverage | v1 Engine | Version | Notes | +|----|------|----------|-----------|---------|-------| +| DG-105 | Avoid telco-specific terminology | spectral: camara-language-avoid-telco (hint) | spectral | both | | +| DG-106 | Spelling check on descriptions | linting-rules.md: camara-language-spelling (No/warn) | spectral | both | Listed but explicitly not enabled | + +--- + +## Gap Summary + +### Gaps requiring new Spectral rules (r4.x ruleset) + +| ID | Rule | Priority | +|----|------|----------| +| DG-003 | date-time description MUST state RFC 3339 | Low (advisory) | +| DG-004 | duration description MUST state RFC 3339 | Low (advisory) | +| DG-008 | Object: required properties MUST be listed | Medium | +| DG-013 | Error code MUST NOT be numeric | Medium | +| DG-014 | Error code MUST be SCREAMING_SNAKE_CASE | Medium (r4.x-only) | +| DG-015 | API-specific error codes: API_NAME.SPECIFIC_CODE | Medium | +| DG-017 | All APIs MUST document 403 response | Medium | +| DG-032 | info.contact MUST be absent | Low | +| DG-041 | Tag names SHOULD use Title Case | Low | +| DG-058 | Array items MUST have description | Medium (r4.x-only) | +| DG-087 | specversion MUST be "1.0" | Low (subscription APIs only) | +| DG-090 | protocol MUST be "HTTP" | Low (subscription APIs only) | +| DG-091 | sink MUST use HTTPS | Low (subscription APIs only) | +| DG-094 | Notification content-type: cloudevents+json | Low (subscription APIs only) | + +### Gaps requiring Python checks + +| ID | Rule | Priority | +|----|------|----------| +| DG-011 | contextCode SCREAMING_SNAKE_CASE format | Low (r4.x-only) | +| DG-018 | CONFLICT error code deprecated warning | Low (r4.x-only) | +| DG-086 | Event type format validation | Medium | +| DG-088 | Subscription API filename convention | Medium | +| DG-092 | sinkCredential not in responses | Medium | +| DG-095 | Event version independence from API version | Low | + +### Rules already covered by v0_6 but not by Spectral (need Python in v1) + +Most v0_6 checks that need Python (V6-010, V6-012, V6-029, V6-050–053, V6-054–057, V6-068–080) will continue to need Python in v1 due to cross-field, cross-file, or context-dependent logic. + +--- + +## Linting-rules.md Discrepancies + +Rules listed in Linting-rules.md but **NOT implemented** in current .spectral.yaml: + +| Linting-rules.md Rule | Severity | Status | Notes | +|------------------------|----------|--------|-------| +| camara-resource-reserved-words | warn | Missing | Resource names must not contain HTTP method names | +| camara-path-param-id-morphology | warn | Missing | Consistent {entityId} morphology | +| camara-property-casing-convention | error | Missing | Property names lowerCamelCase | +| camara-enum-casing-convention | info | Missing (tbd) | Enum values UPPER_SNAKE_CASE | +| camara-info-title | warn | Missing (tbd) | Title must not contain "API" | +| camara-info-version-format | warn | Missing (tbd) | Version format x.y.z/wip/alpha/rc | +| camara-language-spelling | warn | Missing (No) | Spell checking on descriptions | + +Rules in .spectral.yaml but **NOT listed** in Linting-rules.md: +- None found — all current .spectral.yaml rules are listed + +## Severity Alignment Issues + +| Rule | .spectral.yaml | Linting-rules.md | Design Guide Intent | +|------|----------------|-------------------|---------------------| +| camara-operationid-casing-convention | hint | error | MUST (error appropriate) | +| camara-schema-casing-convention | warn | warn | MUST in DG text (error may be more appropriate) | +| camara-path-param-id | warn | warn | Aligned | +| camara-operation-summary | warn | warn | Aligned | + +--- + +## Version-Sensitive Rules Summary + +### New in r4.x (not applicable to r3.4 repos) + +| ID | Rule | OWASP? | +|----|------|--------| +| DG-001 | String maxLength/enum MUST | Yes | +| DG-005 | Array maxItems MUST | Yes | +| DG-006 | Integer format MUST | Yes | +| DG-007 | Integer min/max MUST | Yes | +| DG-011 | contextCode naming | No | +| DG-014 | Error code SCREAMING_SNAKE_CASE | No | +| DG-018 | CONFLICT deprecated | No | +| DG-030 | info.description error responses section | No | +| DG-058 | Array items description | No | +| DG-069–078 | All OWASP rules | Yes | + +### Changed between versions + +| ID | Rule | r3.4 | r4.x | +|----|------|------|------| +| DG-010 | Discriminator on oneOf/anyOf | warn (recommended) | hint (deprecated) | +| DG-048 | operationId casing | hint | error (per Linting-rules.md) | + +### Implications for per-version Spectral rulesets + +- The r3.4 Spectral ruleset is effectively the current .spectral.yaml (no OWASP, no new MUSTs) +- The r4.x ruleset adds: OWASP rules (from tooling#95) + new CAMARA rules for DG-014, DG-058, and the missing Linting-rules.md rules +- OWASP api4 rules have current (warn) and target (error) severities — target is for 2027 releases + +--- + +## Statistics + +| Category | Count | +|----------|-------| +| Total rules extracted | 106 | +| Covered by existing Spectral | 26 | +| Covered by OWASP (tooling#95) | 17 | +| Covered by v0_6 validator only | 28 | +| Gaps (no implementation) | 20 | +| Multiple coverage | 15 (overlap) | +| r4.x-only rules | 19 | +| Changed between versions | 2 | +| Linting-rules.md discrepancies | 7 | +| Severity alignment issues | 2 | diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index e0d3b42..f2e15b0 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -137,43 +137,69 @@ The framework must accept exactly the values defined in these schemas. Any chang ### 2.1 Inventory Status -The per-rule inventory is **not yet complete**. The following work is required: +The per-rule inventory is based on a Commonalities audit that examined `CAMARA-API-Design-Guide.md` and `CAMARA-API-Event-Subscription-and-Notification-Guide.md` at both r3.4 and r4.1 versions, cross-referenced against the existing Spectral rules (17 CAMARA custom + core OAS), the OWASP rules from [tooling#95](https://github.com/camaraproject/tooling/pull/95), the `Linting-rules.md` maintained in Commonalities, and the deprecated `api_review_validator_v0_6.py` (80 checks). -1. **Commonalities audit** (dependency): Examine `CAMARA-API-Design-Guide.md` and `CAMARA-API-Event-Subscription-and-Notification-Guide.md` at both r3.4 and r4.1/r4.2 versions to: - - Identify checks not yet covered by any engine - - Validate existing Spectral rules against the current design guide - - Identify rules that changed between Commonalities releases +The audit identified 106 machine-checkable rules total: 26 already covered by existing Spectral, 17 by OWASP rules (tooling#95, not yet merged), 28 by the v0_6 validator only, and 20 gaps with no current implementation. Of the 106 rules, 19 are r4.x-only (not applicable to r3.4 repositories), 2 changed between versions, and 7 rules listed in `Linting-rules.md` are not yet implemented in the `.spectral.yaml` configuration. -2. **Existing rule classification**: Map each current Spectral rule to the framework metadata model (applicability, conditional level, hints). The existing Spectral severity levels are assumed valid for now; detailed severity review is deferred. +Remaining inventory work: +- **Existing rule classification**: Map each current Spectral rule to the framework metadata model (applicability, conditional level, hints). The existing Spectral severity levels are assumed valid for now; detailed severity review is deferred. -3. **api-review v0.6 coverage**: The deprecated `api_review_validator_v0_6.py` (~43 checks) was a monolithic implementation used manually by release reviewers for the Fall25 meta-release. Its checks serve as input for identifying Python-needed validations not covered by Spectral. +### 2.2 Check Areas by Engine -### 2.2 Known Check Areas by Engine - -Summary of check areas identified so far, pending the Commonalities audit for completeness: - -**Spectral (existing + new):** +**Spectral (existing rules):** - OpenAPI version enforcement (3.0.3) -- Naming conventions (e.g. paths: kebab-case, schemas: PascalCase, operationId: camelCase, enums: UPPER_SNAKE_CASE, headers: kebab-case) +- Naming conventions (see Appendix A for full list: paths, schemas, operationId, plus gaps for properties, enums, tags) - Required descriptions (operations, parameters, responses, properties) -- Reserved words detection +- Reserved words detection (language-specific + HTTP method names in resource paths) - Security: no secrets in path/query parameters - HTTP method validity, no request body on GET/DELETE - Unused components detection -- *New*: info.version format (wip/alpha.n/rc.n/public), XCorrelator pattern, phone number format, device object structure, no info.contact field, check externalDocs format - -**Python (new):** +- Discriminator on oneOf/anyOf (deprecated in r4.x, now hint) +- Schema type attribute presence + +**Spectral (new rules needed):** +- info.version format (wip/alpha.n/rc.n/semver) +- info.title must not contain "API" +- info.contact and info.termsOfService must be absent +- externalDocs presence and format +- x-correlator header presence and pattern +- Error code format: not numeric, SCREAMING_SNAKE_CASE (r4.x), API_NAME.SPECIFIC_CODE pattern +- 403 response required on all operations +- Array items must have description (r4.x) +- Tag names: Title Case convention +- Property names: lowerCamelCase (listed in Linting-rules.md, not yet implemented) +- Enum values: UPPER_SNAKE_CASE (listed in Linting-rules.md, not yet implemented) +- Subscription API schemas: specversion enum, protocol enum, sink HTTPS, notification content-type + +**OWASP Spectral rules (from [tooling#95](https://github.com/camaraproject/tooling/pull/95), r4.x-only):** +- String limits: maxLength/enum/const (warn, target error) +- Array limits: maxItems (warn, target error) +- Integer limits: format + minimum/maximum (warn, target error) +- String restriction: format/pattern/enum/const (warn) +- Security: no credentials in URL, no HTTP scheme, write-restricted, read-restricted, short-lived access tokens, no numeric IDs, admin security unique +- Error responses: 401 required (error), error validation response (warn) +- Additional properties: constrained or disabled (warn) + +**Python (cross-field, cross-file, and context-dependent):** - Server URL version consistency with info.version (cross-field) - Version must be wip on main, must not be wip on release branches (context-dependent) - release-plan.yaml non-exclusivity check (PR diff analysis) - release-plan.yaml schema and semantic validation (existing, to be integrated) -- Error response structure and code compliance (cross-schema) +- Error response structure: ErrorInfo schema compliance, $ref resolution (cross-schema) +- info.description: authorization and error response template sections (normalized text matching) +- Security scheme validation: openIdConnect named 'openId', notificationsBearerAuth for callbacks +- Scope naming: api-name:[resource:]action pattern, subscription-specific scopes +- Event type format: org.camaraproject.\.\.\ (subscription APIs) +- Subscription API structure: required operations, sinkCredential not in responses - Test file existence and version alignment (cross-file) - CHANGELOG format and link tag-locking (file content analysis) -- API pattern-specific checks: subscription endpoints, CloudEvents format, event type naming (structural + semantic) +- Common schema consistency across API files (cross-file; partially obsolete with bundling) +- License and x-camara-commonalities consistency across API files (cross-file) +- Filename conventions: kebab-case, matches api-name (filesystem) +- CONFLICT error code deprecated warning (r4.x) **Manual + prompt:** -- Data minimization compliance (GEN-009, GEN-010) +- Data minimization compliance - Meaningful description quality (beyond presence checks) - User story adequacy - Breaking change justification @@ -597,9 +623,9 @@ The v1 caller workflow is deployed by copying from `Template_API_Repository` to Deployment can be batched using the existing admin tooling pattern (scripted multi-repo operations). The caller can be deployed to all repos at once in stage 0 (dark) — it has no effect until the repo is listed in the config file. -### 6.7 WP-01 Relationship +### 6.7 Relationship to tooling#121 -WP-01 (tooling#121) fixes ref consistency in the existing v0 reusable workflow. It validates the OIDC ref resolution pattern that v1 reuses and adds the `tooling_ref_override` break-glass input. WP-01 does not change the v0 caller — callers still call `@v0`. The v1 reusable workflow reuses the same ref resolution pattern with the hardcoded version fallback (section 5.6). +[tooling#121](https://github.com/camaraproject/tooling/pull/121) fixes ref consistency in the existing v0 reusable workflow. It validates the OIDC ref resolution pattern that v1 reuses and adds the `tooling_ref_override` break-glass input. tooling#121 does not change the v0 caller — callers still call `@v0`. The v1 reusable workflow reuses the same ref resolution pattern with the hardcoded version fallback (section 5.6). --- @@ -909,7 +935,7 @@ Conditional — only runs when external `$ref` to `code/common/` or `code/module The framework invokes an **external bundling tool** — it does not implement its own OpenAPI bundler. The tool must satisfy two requirements: -1. **External ref resolution only** (DEC-002): Resolve `$ref` to `code/common/`, `code/modules/`, and other local files. Preserve all internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`). Full dereferencing must not be used. +1. **External ref resolution only** (see section 3.1): Resolve `$ref` to `code/common/`, `code/modules/`, and other local files. Preserve all internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`). Full dereferencing must not be used. 2. **Source map production**: Produce a mapping from bundled output regions back to source file locations. This is needed for line number translation in the output pipeline (section 9.5). This is a selection criterion for tool evaluation — the chosen tool must either support source maps natively or be wrappable to produce them. The specific tool choice (e.g., redocly, swagger-cli, prance, custom wrapper) is deferred to implementation, evaluated against these two requirements. @@ -1159,7 +1185,7 @@ The full Spectral JSON output and the complete findings list (all engines) are a When Spectral runs on bundled output (section 8.4, step 7), finding line numbers reference the bundled file, not the original source file. Annotations and summary tables must show source file locations to be actionable for developers. -**Requirement on the bundling tool**: The external bundling tool (section 8.4, step 6) must produce — or be augmented to produce — a source map that records which regions of the bundled file originated from which source file and line range. This is a selection criterion for bundling tool evaluation, alongside external-ref-only resolution (DEC-002). +**Requirement on the bundling tool**: The external bundling tool (section 8.4, step 6) must produce — or be augmented to produce — a source map that records which regions of the bundled file originated from which source file and line range. This is a selection criterion for bundling tool evaluation, alongside external-ref-only resolution (see section 3.1). **Design choice**: Content pulled in from external refs (e.g., schemas from `CAMARA_common.yaml`) maps back to the **`$ref` line in the source file**, not to the external file itself. The `$ref` declaration is the actionable location — it is the line the developer controls. If Spectral reports an issue at line 247 of the bundled file, and that region was pulled from an external ref declared at line 15 of the source file, the finding is reported at source line 15. @@ -1279,3 +1305,25 @@ When release automation invokes validation with `mode: pre-snapshot`, it consume The validation framework does not need `contents: write` permission and has no knowledge of snapshot branch naming. It produces files and reports results; release automation decides what to do with them. This keeps a clean separation: validation is stateless, release automation owns the repository state. **For non-pre-snapshot runs** (PR and dispatch triggers): bundled specs are uploaded as artifacts for reviewer inspection only. No branch creation or file replacement occurs — the artifacts are informational. + +--- + +## Appendix A: Naming Conventions + +Complete naming convention rules from CAMARA-API-Design-Guide.md and CAMARA-API-Event-Subscription-and-Notification-Guide.md, with current Spectral rule coverage. + +| Element | Convention | Example | DG Section | Spectral Rule | Status | +|---------|-----------|---------|------------|---------------|--------| +| Paths (URLs) | kebab-case | `/customer-segments` | 5.7.1 | camara-parameter-casing-convention (error) | Implemented | +| Path parameters | {entityId} form | `{userId}`, `{accountId}` | 5.7.1 | camara-path-param-id (warn) | Implemented (morphology rule missing) | +| Schemas | PascalCase | `ErrorInfo`, `DeviceResponse` | 5.8.1 | camara-schema-casing-convention (warn) | Implemented | +| operationId | camelCase | `helloWorld`, `retrieveLocation` | 5.7.2 | camara-operationid-casing-convention | Implemented (severity: hint vs error in Linting-rules.md) | +| Properties | lowerCamelCase | `sessionId`, `phoneNumber` | 5.7.4 | camara-property-casing-convention (error) | Listed in Linting-rules.md, not in .spectral.yaml | +| Enum values | UPPER_SNAKE_CASE | `INVALID_ARGUMENT`, `PERMISSION_DENIED` | 3.2 | camara-enum-casing-convention (info) | Listed in Linting-rules.md, not in .spectral.yaml | +| Error codes | SCREAMING_SNAKE_CASE | `UNAUTHENTICATED`, `NOT_FOUND` | 3.2 | — | Gap (r4.x explicit requirement) | +| API-specific error codes | API_NAME.SPECIFIC_CODE | `CARRIER_BILLING.PAYMENT_DENIED` | 3.2.1 | — | Gap | +| Tags | Title Case with spaces | `Quality On Demand` | 5.7.3 | — | Gap | +| Headers | kebab-case | `x-correlator` | 5.8.5 | — | Implied by convention, no rule | +| API name | kebab-case | `location-verification` | 1.2 | — | v0_6 validator only | +| Scope names | kebab-case with : separators | `qod:sessions:create` | 6.6 | — | v0_6 validator only | +| Event type | org.camaraproject.\.\.\ | `org.camaraproject.device-roaming-subscriptions.v1.roaming-status` | Event Guide 3.1 | — | Gap | From 21d8949259bb93ad8210ef498fbbae4320420d53 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Thu, 19 Mar 2026 09:33:16 +0100 Subject: [PATCH 08/25] docs: add testing guidelines audit Audit of 65 machine-checkable rules from API-Testing-Guidelines.md, cross-referenced against gherkin-lintrc (25 rules) and api_review_validator_v0_6.py test alignment checks. --- ...tion-Framework-Testing-Guidelines-Audit.md | 220 ++++++++++++++++++ 1 file changed, 220 insertions(+) create mode 100644 documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md new file mode 100644 index 0000000..bfd86b9 --- /dev/null +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md @@ -0,0 +1,220 @@ +# Testing Guidelines Audit + +**Date**: 2026-03-19 +**Scope**: Machine-checkable rules from API-Testing-Guidelines.md +**Sources**: API-Testing-Guidelines.md, .gherkin-lintrc (25 rules), api_review_validator_v0_6.py (test checks V6-071–V6-080) + +## Methodology + +1. Walked API-Testing-Guidelines.md section by section, extracting every machine-checkable requirement +2. Cross-referenced against: .gherkin-lintrc rules, v0_6 validator test alignment checks, pr_validation.yml MegaLinter integration + +## Legend + +**Coverage column**: +- `gherkin: ` — rule exists in .gherkin-lintrc +- `v0_6: V6-NNN` — covered by api_review_validator_v0_6.py +- `gap` — no current implementation + +**v1 Engine**: `gherkin-lint` / `python` / `manual` + +--- + +## File Structure and Location + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-001 | Test files MUST be in `code/Test_definitions/` | v0_6: V6-071 | python | Directory existence check | +| TG-002 | Each API MUST have at least one `.feature` file | v0_6: V6-072 | python | Matched by api-name prefix | +| TG-003 | Single-feature file: filename = api-name (kebab-case) | v0_6: V6-072 | python | e.g. `location-verification.feature` | +| TG-004 | Multi-feature files split by operation: `{api-name}-{operationId}.feature` | v0_6: V6-076 | python | operationId must exist in spec | +| TG-005 | Multi-feature files other grouping: `{api-name}-{description}.feature` | gap | python | No validation of description part | +| TG-006 | Filenames follow `{api-name}` or `{api-name}-{operationId}` convention | v0_6: V6-072/V6-076 | python | gherkin-lint `file-name` (kebab-case) cannot be used: convention mixes kebab-case api-name with camelCase operationId | +| TG-007 | `.feature` files must not be empty | gherkin: no-empty-file | gherkin-lint | | +| TG-008 | `.feature` files must contain scenarios | gherkin: no-files-without-scenarios | gherkin-lint | | + +--- + +## Feature Description + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-009 | Feature MUST have a name | gherkin: no-unnamed-features | gherkin-lint | | +| TG-010 | Feature name MUST include API name | gap | python | Not checked by gherkin-lint | +| TG-011 | Feature name MUST include API version | v0_6: V6-073 | python | Checks lines 1-2 for version pattern | +| TG-012 | Feature name max 250 characters | gherkin: name-length (Feature: 250) | gherkin-lint | | +| TG-013 | Feature names MUST be globally unique | gherkin: no-dupe-feature-names | gherkin-lint | | + +--- + +## Feature Context (RECOMMENDED) + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-014 | Feature context SHOULD reference API spec file location | gap | python | Pattern: `in {apiname}.yaml` | +| TG-015 | Feature context SHOULD include "Implementation indications" section | gap | manual | Recommended, not mandatory | +| TG-016 | Feature context SHOULD include "Testing assets" section | gap | manual | Recommended, not mandatory | +| TG-017 | Feature context SHOULD include "References to OAS spec schemas" | gap | manual | Recommended, not mandatory | + +--- + +## Background Section + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-018 | Background MUST NOT be empty | gherkin: no-empty-background | gherkin-lint | | +| TG-019 | Background MUST NOT exist without scenarios | gherkin: no-background-only-scenario | gherkin-lint | | +| TG-020 | Background SHOULD include environment setup (apiRoot) | gap | manual | Recommended pattern, not strictly checkable | +| TG-021 | Background SHOULD include Content-Type header setup | gap | manual | Recommended pattern | +| TG-022 | Background SHOULD include Authorization header setup | gap | manual | Recommended pattern | +| TG-023 | Background SHOULD include x-correlator header setup | gap | manual | Recommended pattern | + +--- + +## Scenario Structure + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-024 | Each scenario MUST have a name | gherkin: no-unnamed-scenarios | gherkin-lint | | +| TG-025 | Scenario names MUST be unique within feature | gherkin: no-dupe-scenario-names (in-feature) | gherkin-lint | | +| TG-026 | Scenario name max 250 characters | gherkin: name-length (Scenario: 250) | gherkin-lint | | +| TG-027 | Step name max 250 characters | gherkin: name-length (Step: 250) | gherkin-lint | | +| TG-028 | Steps MUST follow Given → When → Then order | gherkin: keywords-in-logical-order | gherkin-lint | | +| TG-029 | Repeated step keywords MUST use `And` | gherkin: use-and | gherkin-lint | | +| TG-030 | Scenario Outline MUST have Examples section | gherkin: no-scenario-outlines-without-examples | gherkin-lint | | +| TG-031 | Max 50 scenarios per file | gherkin: max-scenarios-per-file (50) | gherkin-lint | | +| TG-032 | All scenario variables MUST be defined | gherkin: no-unused-variables | gherkin-lint | | + +--- + +## Scenario Tagging + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-033 | Scenario tag format: `@{feature_identifier}_{number}_{optional_detail}` | gap | python | Pattern check on tag naming | +| TG-034 | Tag identifiers: lowercase with underscores | gap | python | Could be gherkin-lint allowed-tags pattern | +| TG-035 | Every scenario MUST have at least one tag | gherkin: required-tags (pattern: `^@.*$`) | gherkin-lint | | +| TG-036 | Tags `@watch` and `@wip` are forbidden | gherkin: no-restricted-tags | gherkin-lint | | +| TG-037 | No duplicate tags on same scenario | gherkin: no-duplicate-tags | gherkin-lint | | +| TG-038 | No superfluous tags | gherkin: no-superfluous-tags | gherkin-lint | | +| TG-039 | Single space between tags | gherkin: one-space-between-tags | gherkin-lint | | +| TG-040 | No inline comments on tag lines | gherkin: no-partially-commented-tag-lines | gherkin-lint | | + +--- + +## Request Step Syntax + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-041 | operationId in `When the request "{operationId}" is sent` MUST exist in API spec | v0_6: V6-075 | python | Cross-file: test ↔ API spec | +| TG-042 | Path parameters: `the path parameter "{name}" is set as "{value}"` | gap | manual | Standardized syntax, hard to enforce | +| TG-043 | Query parameters: `the query parameter "{name}" is set as "{value}"` | gap | manual | Standardized syntax | +| TG-044 | Headers: `the header "{name}" is set as "{value}"` | gap | manual | Standardized syntax | +| TG-045 | Request body: `the request body property "{json_path}" is set as "{value}"` | gap | manual | JSON path notation | + +--- + +## Response Validation Syntax + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-046 | Status code validation: `the response status code is {code}` | gap | manual | Standardized syntax | +| TG-047 | Schema compliance: `the response body complies with the OAS schema {ref}` | gap | manual | Standardized syntax | +| TG-048 | Error code: `the response property "$.code" is "{ERROR_CODE}"` | gap | manual | Standardized syntax | + +--- + +## URL and Version Alignment (cross-file) + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-049 | Test file URLs MUST contain correct api-name matching API spec | v0_6: V6-078 | python | Regex extraction + comparison | +| TG-050 | Test file URL version suffix MUST match API version | v0_6: V6-079 | python | Version mapping rules (wip→vwip, 1.0.0→v1, 0.3.0→v0.3) | +| TG-051 | Test file URLs SHOULD have leading slash | v0_6: V6-080 | python | Style convention (LOW severity) | +| TG-052 | Operation-specific test filenames MUST reference valid operationIds | v0_6: V6-076 | python | Cross-file: filename ↔ API spec | +| TG-053 | Orphan test files (not matching any API name) MUST be flagged | v0_6: V6-077 | python | Multi-file name matching | + +--- + +## Coverage Requirements + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-054 | RC: minimum basic test plan with sunny-day scenarios | gap | manual | Semantic — requires human judgment | +| TG-055 | Public release: full test plan with sunny and rainy day scenarios | gap | manual | Semantic | +| TG-056 | All HTTP statuses documented in spec MUST have test scenarios | gap | python | Cross-file: response codes in spec → scenario coverage. Feasible but complex | +| TG-057 | 3-legged token responses: MUST NOT include device identifier | gap | manual | Response content validation at runtime | +| TG-058 | 2-legged token responses: SHOULD include device identifier | gap | manual | Response content validation at runtime | + +--- + +## Indentation and Formatting + +| ID | Rule | Coverage | v1 Engine | Notes | +|----|------|----------|-----------|-------| +| TG-059 | Feature: 0 spaces indentation | gherkin: indentation | gherkin-lint | | +| TG-060 | Background/Scenario/Step: 2 spaces | gherkin: indentation | gherkin-lint | | +| TG-061 | Examples header: 4 spaces | gherkin: indentation | gherkin-lint | | +| TG-062 | Example rows: 6 spaces | gherkin: indentation | gherkin-lint | | +| TG-063 | Tags: 2 spaces indentation | gherkin: indentation | gherkin-lint | | +| TG-064 | No trailing spaces | gherkin: no-trailing-spaces | gherkin-lint | | +| TG-065 | No multiple empty lines | gherkin: no-multiple-empty-lines | gherkin-lint | | + +--- + +## Gherkin-lint Rules Not Mapped to Testing Guidelines + +These rules exist in `.gherkin-lintrc` but do not correspond to a specific testing guidelines requirement: + +| Gherkin Rule | Status | Notes | +|-------------|--------|-------| +| no-homogenous-tags | ON | Quality heuristic, not in guidelines | +| allowed-tags | ON | Patterns: `^@watch$`, `^@wip$`, `^@.*$` — enforces tagging | +| new-line-at-eof | OFF | Not mentioned in guidelines | +| scenario-size (15 steps) | OFF | Not enforced; guidelines allow complex scenarios | +| only-one-when | OFF | Guidelines explicitly allow multiple When blocks | +| no-restricted-patterns | OFF | Debugging step patterns configured but disabled | + +--- + +## Gap Summary + +### Gaps addressable by gherkin-lint configuration changes + +| ID | Rule | Suggested Change | +|----|------|-----------------| +| ~~TG-006~~ | ~~Filenames use kebab-case~~ | Not a gap: covered by v0_6 Python checks; gherkin-lint `file-name` cannot be used due to mixed casing (`{api-name}-{operationId}`) | +| TG-033/034 | Tag naming convention `@{feature}_{number}_{detail}` | Could use `allowed-tags` pattern: `^@[a-z][a-z0-9]*(_[a-z0-9]+)*$` | + +### Gaps requiring Python checks + +| ID | Rule | Priority | +|----|------|----------| +| TG-010 | Feature name includes API name | Medium | +| TG-014 | Feature context references API spec file | Low (RECOMMENDED) | +| TG-056 | All documented HTTP status codes have test scenarios | Medium (cross-file, complex) | + +### Gaps that remain manual + +| ID | Rule | Reason | +|----|------|--------| +| TG-015–017 | Feature context sections | Semantic content, RECOMMENDED | +| TG-020–023 | Background setup patterns | Recommended patterns, not strict syntax | +| TG-042–048 | Standardized step syntax | Natural language with alternatives allowed | +| TG-054–055 | Test coverage completeness | Requires human judgment on adequacy | +| TG-057–058 | Device identifier in responses | Runtime validation, not static | + +--- + +## Statistics + +| Category | Count | +|----------|-------| +| Total rules extracted | 65 | +| Covered by gherkin-lint | 25 | +| Covered by v0_6 validator | 10 | +| Gaps (no implementation) | 30 | +| Of which: addressable by config change | 1 | +| Of which: addressable by Python | 3 | +| Of which: remain manual | 25 | +| Gherkin rules not mapped to guidelines | 6 | From b2a1480304af621d0f3e9020f47a33e989d1f4ec Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Thu, 19 Mar 2026 10:01:06 +0100 Subject: [PATCH 09/25] docs: add prior discussions section to design guide audit Cross-reference identified discrepancies, gaps, and disabled rules against prior Commonalities and tooling issues/PRs to document known rationale and decisions. --- ...Validation-Framework-Design-Guide-Audit.md | 60 +++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md index 0af3704..24b1756 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md @@ -394,6 +394,66 @@ Rules in .spectral.yaml but **NOT listed** in Linting-rules.md: --- +## Prior Discussions + +Research of Commonalities and tooling issues/PRs (open and closed) for rationale behind identified discrepancies, gaps, and disabled rules. + +### Linting-rules.md discrepancies + +| Rule | Prior Discussion | Rationale | +|------|-----------------|-----------| +| camara-resource-reserved-words | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) (topic 13) | Documented as planned in initial linting ruleset PR (Oct 2023), confirmed as "to be developed in a later iteration." Never implemented. Gap from incremental rollout. | +| camara-path-param-id-morphology | [Commonalities#16](https://github.com/camaraproject/Commonalities/issues/16) (naming discussion) | Listed in Linting-rules.md but not in summary table. Same incremental rollout gap. No dedicated discussion about the missing implementation. | +| camara-property-casing-convention | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) (topic 6) | Documented with an explicit question: "Should it be lowerCamelCase in DG?" — indicating uncertainty about the design guide requirement. In [PR#466](https://github.com/camaraproject/Commonalities/pull/466) the summary table was updated to `Yes/Yes`, but the rule was still never added to .spectral.yml. | +| camara-enum-casing-convention | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) | Explicitly marked "tbd" since Dec 2023. Linting-rules.md notes "No clear requirement" in the design guide. Deliberate deferral — r4.1 has since added explicit SCREAMING_SNAKE_CASE for error codes (section 3.2), which partially resolves this for error enums but not for general enums. | +| camara-info-title | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) (topic 3); [#201](https://github.com/camaraproject/Commonalities/issues/201) / [#214](https://github.com/camaraproject/Commonalities/pull/214) | Raised as "Do we need rules for info-title; info-version?" in PR#74 review. #214 added DG guidelines (title without "API") but no linting rule followed. "tbd" since Dec 2023, unchanged. | +| camara-info-version-format | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) (topic 4) | Same as info-title. Acknowledged as potential new rule. Complexity noted: format allows wip, x.y.z-alpha.n, x.y.z-rc.n. "tbd" since Dec 2023, unchanged. | +| camara-language-spelling | None | Marked "No/No" from the very first version (Dec 2023). Never intended for implementation — would produce excessive false positives with domain-specific terminology. [#545](https://github.com/camaraproject/Commonalities/issues/545) proposes spelling checks for markdown docs, not API YAML descriptions. | + +### Severity alignment issues + +| Rule | Prior Discussion | Rationale | +|------|-----------------|-----------| +| camara-operationid-casing-convention (hint vs error) | [Commonalities#76](https://github.com/camaraproject/Commonalities/issues/76) (severity discussion) | Internal inconsistency within Linting-rules.md itself: section 3 says `error`, section 4 summary says `Hint`. Implementation matches `hint`. Likely a documentation error in section 3 that was never caught. | +| camara-schema-casing-convention (warn vs MUST) | [Commonalities#76](https://github.com/camaraproject/Commonalities/issues/76) | General pattern: linting rules use lower severities than DG language suggests. MegaLinter only blocks on errors, so `warn` means "flagged but not blocking." Conservative choice during rollout. | + +### OWASP disabled rules + +All OWASP enablement decisions were individually reviewed in the master tracking issue [Commonalities#539](https://github.com/camaraproject/Commonalities/issues/539) with sub-issues [#548](https://github.com/camaraproject/Commonalities/issues/548) (api2), [#549](https://github.com/camaraproject/Commonalities/issues/549) (api3), [#551](https://github.com/camaraproject/Commonalities/issues/551) (api4), [#552](https://github.com/camaraproject/Commonalities/issues/552) (api8). Documentation landed via [PR#582](https://github.com/camaraproject/Commonalities/pull/582). + +| Category | Disabled Rules | Rationale | +|----------|---------------|-----------| +| Authentication (api2) | auth-insecure-schemes, jwt-best-practices, no-http-basic, no-api-keys-in-url | CAMARA uses only OpenID Connect (defined by ICM). These check for auth schemes never present in CAMARA specs — would never trigger. | +| Rate limiting (api4) | rate-limit, rate-limit-retry-after, rate-limit-responses-429 | Rate limiting is implementation/API Gateway specific. No IETF standard for rate limit headers yet. Not mandated by CAMARA. | +| CORS (api8) | define-cors-origin | CORS is implementation/API Gateway specific, not in API specs. | +| Error 500 (api8) | define-error-responses-500 | CAMARA guidelines do NOT recommend defining 500 responses — including them could leak implementation details. | +| Inventory (api9) | inventory-access, inventory-environment | Require vendor extensions (x-internal) and environment terms (staging, production) that CAMARA does not use. | +| Integer limit (api4) | integer-limit | For OAS 3.1 only — CAMARA uses OAS 3.0.x. The equivalent `integer-limit-legacy` IS enabled. | + +### OWASP severity downgrades (api4 rules at warn) + +Discussed in [Commonalities#539](https://github.com/camaraproject/Commonalities/issues/539) and [PR#582](https://github.com/camaraproject/Commonalities/pull/582). Rationale: these rules introduce new requirements that existing CAMARA APIs do not yet satisfy (many APIs lack maxLength on strings, format on integers, maxItems on arrays). Making them errors immediately would break all existing API linting runs. The `warn` severity provides a transition window. + +- The "Target CAMARA severity" column in Linting-rules.md explicitly targets `error` for 2027 meta-releases +- [tooling#95](https://github.com/camaraproject/tooling/pull/95) implements both current (`.spectral-owasp.yaml`) and target (`.spectral-owasp-target.yaml`) configurations +- Commonalities' own templates were fixed in [PR#590](https://github.com/camaraproject/Commonalities/pull/590) (CAMARA_common.yaml) and [PR#591](https://github.com/camaraproject/Commonalities/pull/591) (event-subscription-template.yaml) +- Active debate: [Commonalities#596](https://github.com/camaraproject/Commonalities/issues/596) challenges whether api4 constraints should apply to output properties at all (arguing OWASP api4 is about input validation only). Unresolved. + +### Other items + +| Topic | Prior Discussion | Finding | +|-------|-----------------|---------| +| Tag naming convention (Title Case) | [Commonalities#80](https://github.com/camaraproject/Commonalities/issues/80) | #80 discussed operation tag usage but not Title Case enforcement. No discussion found about linting tag casing. | +| 403 response required | None | OWASP covers 401 only. No discussion found about requiring 403 via linting. | +| Linting-rules.md as source of truth | [Commonalities#482](https://github.com/camaraproject/Commonalities/issues/482) (open) | Acknowledges dual location of linting config. Proposes removing Commonalities copy, keeping tooling as single source. | +| Rule alignment umbrella | [Commonalities#532](https://github.com/camaraproject/Commonalities/issues/532) (open, Spring26) | "Review and correct artifacts to be conforming with linting rules" — focuses on template compliance, not the tbd rules. | + +### Cross-cutting finding + +The incremental rollout approach was established in [PR#74](https://github.com/camaraproject/Commonalities/pull/74) (Oct 2023), where rules were documented before implementation. Several rules from that era remain documented but not yet implemented. No existing issue specifically tracks the gap between documented rules in Linting-rules.md and implemented rules in .spectral.yml. + +--- + ## Statistics | Category | Count | From 8d90eb0e1f836f4e6635faa8ddd3781a7f0aa745 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Thu, 19 Mar 2026 10:07:59 +0100 Subject: [PATCH 10/25] docs: reclassify spelling rule as deliberate exclusion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit camara-language-spelling was never intended for Spectral — marked "No/No" from day one. Spelling checks are better suited to AI-based review tools. Move from discrepancies to other items section. --- .../CAMARA-Validation-Framework-Design-Guide-Audit.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md index 24b1756..901fbee 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md @@ -346,7 +346,8 @@ Rules listed in Linting-rules.md but **NOT implemented** in current .spectral.ya | camara-enum-casing-convention | info | Missing (tbd) | Enum values UPPER_SNAKE_CASE | | camara-info-title | warn | Missing (tbd) | Title must not contain "API" | | camara-info-version-format | warn | Missing (tbd) | Version format x.y.z/wip/alpha/rc | -| camara-language-spelling | warn | Missing (No) | Spell checking on descriptions | + +Note: `camara-language-spelling` is listed in Linting-rules.md as "No/No" — this is not a discrepancy but a deliberate exclusion. Spelling and description quality checks are better suited to AI-based review tools than pattern-based linting. Rules in .spectral.yaml but **NOT listed** in Linting-rules.md: - None found — all current .spectral.yaml rules are listed @@ -408,7 +409,7 @@ Research of Commonalities and tooling issues/PRs (open and closed) for rationale | camara-enum-casing-convention | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) | Explicitly marked "tbd" since Dec 2023. Linting-rules.md notes "No clear requirement" in the design guide. Deliberate deferral — r4.1 has since added explicit SCREAMING_SNAKE_CASE for error codes (section 3.2), which partially resolves this for error enums but not for general enums. | | camara-info-title | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) (topic 3); [#201](https://github.com/camaraproject/Commonalities/issues/201) / [#214](https://github.com/camaraproject/Commonalities/pull/214) | Raised as "Do we need rules for info-title; info-version?" in PR#74 review. #214 added DG guidelines (title without "API") but no linting rule followed. "tbd" since Dec 2023, unchanged. | | camara-info-version-format | [Commonalities#74](https://github.com/camaraproject/Commonalities/pull/74) (topic 4) | Same as info-title. Acknowledged as potential new rule. Complexity noted: format allows wip, x.y.z-alpha.n, x.y.z-rc.n. "tbd" since Dec 2023, unchanged. | -| camara-language-spelling | None | Marked "No/No" from the very first version (Dec 2023). Never intended for implementation — would produce excessive false positives with domain-specific terminology. [#545](https://github.com/camaraproject/Commonalities/issues/545) proposes spelling checks for markdown docs, not API YAML descriptions. | +| ~~camara-language-spelling~~ | — | Not a discrepancy — see "Other items" below | ### Severity alignment issues @@ -443,6 +444,7 @@ Discussed in [Commonalities#539](https://github.com/camaraproject/Commonalities/ | Topic | Prior Discussion | Finding | |-------|-----------------|---------| +| camara-language-spelling | None | Marked "No/No" from day one (Dec 2023). Not a discrepancy — never intended for Spectral. Spelling and description quality checks are better handled by AI-based review tools, not pattern-based linting. [#545](https://github.com/camaraproject/Commonalities/issues/545) proposes spelling checks for markdown docs, not API YAML descriptions. | | Tag naming convention (Title Case) | [Commonalities#80](https://github.com/camaraproject/Commonalities/issues/80) | #80 discussed operation tag usage but not Title Case enforcement. No discussion found about linting tag casing. | | 403 response required | None | OWASP covers 401 only. No discussion found about requiring 403 via linting. | | Linting-rules.md as source of truth | [Commonalities#482](https://github.com/camaraproject/Commonalities/issues/482) (open) | Acknowledges dual location of linting config. Proposes removing Commonalities copy, keeping tooling as single source. | @@ -466,5 +468,5 @@ The incremental rollout approach was established in [PR#74](https://github.com/c | Multiple coverage | 15 (overlap) | | r4.x-only rules | 19 | | Changed between versions | 2 | -| Linting-rules.md discrepancies | 7 | +| Linting-rules.md discrepancies | 6 | | Severity alignment issues | 2 | From 95ccb200e077f2e8bbec51374ed3e0343f6836b4 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Thu, 19 Mar 2026 10:26:04 +0100 Subject: [PATCH 11/25] docs: add gherkin-lint to check areas and findings normalization table Section 2.2: add gherkin-lint engine category listing test definition file checks (structural rules, step ordering, tagging, formatting). Section 8.4: add gherkin-lint column to findings normalization source mapping table. --- ...RA-Validation-Framework-Detailed-Design.md | 31 ++++++++++++------- 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index f2e15b0..51509e0 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -198,6 +198,13 @@ Remaining inventory work: - Filename conventions: kebab-case, matches api-name (filesystem) - CONFLICT error code deprecated warning (r4.x) +**Gherkin-lint (test definition files):** +- Structural rules: named features and scenarios, unique names, non-empty backgrounds, scenarios with examples +- Step ordering: Given → When → Then, use `And` for repeated keywords +- Tagging: required tags, no restricted tags (@watch, @wip), no duplicates +- Formatting: indentation, no trailing spaces, no multiple empty lines +- Limits: max 50 scenarios per file, max 250 character names + **Manual + prompt:** - Data minimization compliance - Meaningful description quality (beyond presence checks) @@ -994,18 +1001,18 @@ All engine outputs are normalized into a common findings format before post-filt hint: "Use kebab-case: /quality-on-demand/{sessionId}" ``` -| Field | Source — Spectral | Source — yamllint | Source — Python | -|-------|------------------|------------------|-----------------| -| `rule_id` | Looked up from rule metadata by `engine_rule`; auto-assigned if no metadata | Looked up similarly | Set directly by check | -| `engine` | `"spectral"` | `"yamllint"` | `"python"` | -| `engine_rule` | `code` field from JSON | Rule name from output | Check function name | -| `level` | Mapped from severity integer: 0→error, 1→warn, 2→hint, 3→hint | Mapped from yamllint severity | Set directly | -| `message` | `message` field | Error message text | Set directly | -| `path` | `source` field | File path from output | Set directly | -| `line` | `range.start.line` (0-indexed → 1-indexed) | Line number from output | Set directly | -| `column` | `range.start.character` | Column from output | Set directly (or null) | -| `api_name` | Derived from file path | Derived from file path | Set directly | -| `hint` | From rule metadata `hint` field (if present); otherwise engine `message` serves as hint | From rule metadata | Set directly | +| Field | Source — Spectral | Source — yamllint | Source — gherkin-lint | Source — Python | +|-------|------------------|------------------|----------------------|-----------------| +| `rule_id` | Looked up from rule metadata by `engine_rule`; auto-assigned if no metadata | Looked up similarly | Looked up similarly | Set directly by check | +| `engine` | `"spectral"` | `"yamllint"` | `"gherkin"` | `"python"` | +| `engine_rule` | `code` field from JSON | Rule name from output | Rule name from output | Check function name | +| `level` | Mapped from severity integer: 0→error, 1→warn, 2→hint, 3→hint | Mapped from yamllint severity | Mapped from gherkin-lint severity | Set directly | +| `message` | `message` field | Error message text | Error message text | Set directly | +| `path` | `source` field | File path from output | File path from output | Set directly | +| `line` | `range.start.line` (0-indexed → 1-indexed) | Line number from output | Line number from output | Set directly | +| `column` | `range.start.character` | Column from output | Column from output (or null) | Set directly (or null) | +| `api_name` | Derived from file path | Derived from file path | Derived from file path | Set directly | +| `hint` | From rule metadata `hint` field (if present); otherwise engine `message` serves as hint | From rule metadata | From rule metadata | Set directly | Spectral rules **without** explicit framework metadata entries pass through with identity mapping (section 1.3): `rule_id` is auto-assigned, `hint` defaults to the engine's `message`, and the level maps directly. This means the check inventory does not need to be complete before the framework can run — new Spectral rules work immediately. From 77b728a7847fc626b22d6c0a983600f391415e5b Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Thu, 19 Mar 2026 10:37:17 +0100 Subject: [PATCH 12/25] docs: add user story file existence check to Python engine Conditional check for stable public APIs per API Readiness Checklist. File existence is automated; content adequacy remains manual. --- .../CAMARA-Validation-Framework-Detailed-Design.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 51509e0..84b99b5 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -197,6 +197,7 @@ Remaining inventory work: - License and x-camara-commonalities consistency across API files (cross-file) - Filename conventions: kebab-case, matches api-name (filesystem) - CONFLICT error code deprecated warning (r4.x) +- User story file existence in `documentation/API_documentation/` (conditional: mandatory for stable public APIs per API Readiness Checklist) **Gherkin-lint (test definition files):** - Structural rules: named features and scenarios, unique names, non-empty backgrounds, scenarios with examples @@ -208,7 +209,7 @@ Remaining inventory work: **Manual + prompt:** - Data minimization compliance - Meaningful description quality (beyond presence checks) -- User story adequacy +- User story adequacy (content quality; file existence is checked automatically for stable public APIs) - Breaking change justification **Obsolete (handled by release automation):** From 6937f24575b667e8cb7c6e36247ed3940dacd02f Mon Sep 17 00:00:00 2001 From: Herbert Damker Date: Mon, 23 Mar 2026 13:52:27 +0100 Subject: [PATCH 13/25] Update documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md Co-authored-by: Tanja de Groot <87864067+tanjadegroot@users.noreply.github.com> --- .../CAMARA-Validation-Framework-Design-Guide-Audit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md index 901fbee..61ae6b8 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md @@ -260,7 +260,7 @@ Consolidated from design guide sections. This is the full set for the detailed d |----|------|----------|-----------|---------|-------| | DG-086 | Event type MUST follow org.camaraproject.\.\.\ | gap (v0_6 partial) | python | both | | | DG-087 | specversion MUST be "1.0" enum | gap | spectral | both | | -| DG-088 | Subscription API filename MUST append "subscriptions" | gap | python | both | Filesystem check | +| DG-088 | Subscription API filename MUST append "-subscriptions" | gap | python | both | Filesystem check | | DG-089 | Explicit subscriptions: 4 operations (POST/GET/GET{id}/DELETE) | v0_6: V6-065/V6-066 | python | both | | | DG-090 | protocol attribute MUST be "HTTP" | gap | spectral | both | Enum constraint | | DG-091 | sink MUST use HTTPS URI | gap | spectral | both | | From 4caae2ef93aa2ec95279ec2dcfffb7dd5086e770 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Mon, 23 Mar 2026 14:12:43 +0100 Subject: [PATCH 14/25] docs: remove obsolete Model B label from handoff section The Model A/B labels were removed from section 7.1 during earlier editing but a reference remained in section 9.7. --- .../CAMARA-Validation-Framework-Detailed-Design.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 84b99b5..2ca3b18 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -1299,9 +1299,9 @@ Bundled standalone API specs are available as [workflow artifacts]({artifact_url This gives PR reviewers visibility into the bundled output — they can download and inspect the fully resolved specs to verify that `$ref` resolution produced the expected result. For copy-paste repositories (no bundling), this section is omitted. -#### Release automation handoff (Model B) +#### Release automation handoff -When release automation invokes validation with `mode: pre-snapshot`, it consumes the bundled spec artifacts to populate the snapshot branch. The handoff uses the **artifact model** (section 7.1, Model B): +When release automation invokes validation with `mode: pre-snapshot`, it consumes the bundled spec artifacts to populate the snapshot branch. The handoff uses the **artifact model** (section 7.1): 1. Release automation calls the validation reusable workflow with `mode: pre-snapshot` 2. Validation runs the full pipeline (context → pre-bundling → bundling → full validation → output) From 2753f1b1ac2763a80e2c1ec9a6ade5ce45733eea Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Mon, 23 Mar 2026 14:28:19 +0100 Subject: [PATCH 15/25] docs: address review comments on naming conventions and test completeness - Standardize on SCREAMING_SNAKE_CASE (matching Commonalities design guide) - Add Examples (named) to naming convention tables in audit and appendix - Add TG-066: operationId completeness check for operation-specific test files - Add DG-101 cross-reference to testing guidelines audit --- .../CAMARA-Validation-Framework-Design-Guide-Audit.md | 7 ++++--- .../CAMARA-Validation-Framework-Detailed-Design.md | 5 +++-- ...CAMARA-Validation-Framework-Testing-Guidelines-Audit.md | 1 + 3 files changed, 8 insertions(+), 5 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md index 61ae6b8..76a3e69 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md @@ -243,7 +243,7 @@ Consolidated from design guide sections. This is the full set for the detailed d | Schemas | PascalCase | camara-schema-casing-convention | 5.8.1 | Implemented (warn) | | operationId | camelCase | camara-operationid-casing-convention | 5.7.2 | Implemented (hint; should be error per Linting-rules.md) | | Properties | lowerCamelCase | camara-property-casing-convention | 5.7.4 | **Not in .spectral.yaml** (Linting-rules.md: error) | -| Enum values | UPPER_SNAKE_CASE (macro) | camara-enum-casing-convention | 3.2 | **Not in .spectral.yaml** (Linting-rules.md: info, tbd) | +| Enum values | SCREAMING_SNAKE_CASE | camara-enum-casing-convention | 3.2 | **Not in .spectral.yaml** (Linting-rules.md: info, tbd) | | Tags | Title Case (with spaces) | — | 5.7.3 | **Gap** — no rule exists | | Headers | kebab-case | — | 5.8.5 | **Gap** — implied by x-correlator convention | | Scope names | kebab-case with : separators | — | 6.6 | v0_6 only (V6-050) | @@ -251,6 +251,7 @@ Consolidated from design guide sections. This is the full set for the detailed d | Event type | org.camaraproject.\.\.\ | — | Event Guide 3.1 | **Gap** | | Error codes | SCREAMING_SNAKE_CASE | — | 3.2 | **Gap** (r4.x-only explicit) | | API-specific error codes | API_NAME.SPECIFIC_CODE | — | 3.2.1 | **Gap** | +| Examples (named) | SCREAMING_SNAKE_CASE | — | OAS 3.0.3 | **Gap** | --- @@ -280,7 +281,7 @@ Consolidated from design guide sections. This is the full set for the detailed d | DG-098 | License identical across API files | v0_6: V6-069 | python | both | | | DG-099 | x-camara-commonalities identical across API files | v0_6: V6-070 | python | both | | | DG-100 | Test directory exists | v0_6: V6-071 | python | both | | -| DG-101 | Test files exist for each API | v0_6: V6-072 | python | both | | +| DG-101 | Test files exist for each API | v0_6: V6-072 | python | both | See testing guidelines audit TG-004, TG-052, TG-066 for operation-specific file naming and completeness | | DG-102 | Test file version alignment | v0_6: V6-073 | python | both | | | DG-103 | Filename uses kebab-case | v0_6: V6-058 | python | both | | | DG-104 | Filename matches api-name in server URL | v0_6: V6-059 | python | both | | @@ -343,7 +344,7 @@ Rules listed in Linting-rules.md but **NOT implemented** in current .spectral.ya | camara-resource-reserved-words | warn | Missing | Resource names must not contain HTTP method names | | camara-path-param-id-morphology | warn | Missing | Consistent {entityId} morphology | | camara-property-casing-convention | error | Missing | Property names lowerCamelCase | -| camara-enum-casing-convention | info | Missing (tbd) | Enum values UPPER_SNAKE_CASE | +| camara-enum-casing-convention | info | Missing (tbd) | Enum values SCREAMING_SNAKE_CASE | | camara-info-title | warn | Missing (tbd) | Title must not contain "API" | | camara-info-version-format | warn | Missing (tbd) | Version format x.y.z/wip/alpha/rc | diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 2ca3b18..1eaafc1 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -168,7 +168,7 @@ Remaining inventory work: - Array items must have description (r4.x) - Tag names: Title Case convention - Property names: lowerCamelCase (listed in Linting-rules.md, not yet implemented) -- Enum values: UPPER_SNAKE_CASE (listed in Linting-rules.md, not yet implemented) +- Enum values: SCREAMING_SNAKE_CASE (listed in Linting-rules.md, not yet implemented) - Subscription API schemas: specversion enum, protocol enum, sink HTTPS, notification content-type **OWASP Spectral rules (from [tooling#95](https://github.com/camaraproject/tooling/pull/95), r4.x-only):** @@ -1327,7 +1327,7 @@ Complete naming convention rules from CAMARA-API-Design-Guide.md and CAMARA-API- | Schemas | PascalCase | `ErrorInfo`, `DeviceResponse` | 5.8.1 | camara-schema-casing-convention (warn) | Implemented | | operationId | camelCase | `helloWorld`, `retrieveLocation` | 5.7.2 | camara-operationid-casing-convention | Implemented (severity: hint vs error in Linting-rules.md) | | Properties | lowerCamelCase | `sessionId`, `phoneNumber` | 5.7.4 | camara-property-casing-convention (error) | Listed in Linting-rules.md, not in .spectral.yaml | -| Enum values | UPPER_SNAKE_CASE | `INVALID_ARGUMENT`, `PERMISSION_DENIED` | 3.2 | camara-enum-casing-convention (info) | Listed in Linting-rules.md, not in .spectral.yaml | +| Enum values | SCREAMING_SNAKE_CASE | `INVALID_ARGUMENT`, `PERMISSION_DENIED` | 3.2 | camara-enum-casing-convention (info) | Listed in Linting-rules.md, not in .spectral.yaml | | Error codes | SCREAMING_SNAKE_CASE | `UNAUTHENTICATED`, `NOT_FOUND` | 3.2 | — | Gap (r4.x explicit requirement) | | API-specific error codes | API_NAME.SPECIFIC_CODE | `CARRIER_BILLING.PAYMENT_DENIED` | 3.2.1 | — | Gap | | Tags | Title Case with spaces | `Quality On Demand` | 5.7.3 | — | Gap | @@ -1335,3 +1335,4 @@ Complete naming convention rules from CAMARA-API-Design-Guide.md and CAMARA-API- | API name | kebab-case | `location-verification` | 1.2 | — | v0_6 validator only | | Scope names | kebab-case with : separators | `qod:sessions:create` | 6.6 | — | v0_6 validator only | | Event type | org.camaraproject.\.\.\ | `org.camaraproject.device-roaming-subscriptions.v1.roaming-status` | Event Guide 3.1 | — | Gap | +| Examples (named) | SCREAMING_SNAKE_CASE | `SESSION_CREATION_EXAMPLE_WITH_DEVICE_RESPONSE` | OAS 3.0.3 | — | Gap | diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md index bfd86b9..99d769d 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md @@ -133,6 +133,7 @@ | TG-051 | Test file URLs SHOULD have leading slash | v0_6: V6-080 | python | Style convention (LOW severity) | | TG-052 | Operation-specific test filenames MUST reference valid operationIds | v0_6: V6-076 | python | Cross-file: filename ↔ API spec | | TG-053 | Orphan test files (not matching any API name) MUST be flagged | v0_6: V6-077 | python | Multi-file name matching | +| TG-066 | If operation-specific test files are used, all operationIds MUST be covered | gap | python | Completeness check: `{api-name}-{operationId}.feature` for every operationId (RC and later) | --- From 4f6e3ab313e3096a39a9f03e7910bcebf52c37b1 Mon Sep 17 00:00:00 2001 From: Herbert Damker Date: Mon, 23 Mar 2026 16:45:28 +0100 Subject: [PATCH 16/25] Update documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md Co-authored-by: Kevin Smith --- .../CAMARA-Validation-Framework-Design-Guide-Audit.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md index 76a3e69..e0b7a1a 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Design-Guide-Audit.md @@ -3,7 +3,8 @@ **Date**: 2026-03-19 **Scope**: Machine-checkable rules from CAMARA-API-Design-Guide.md and CAMARA-API-Event-Subscription-and-Notification-Guide.md **Versions compared**: r3.4 (Commonalities 0.5) vs r4.1 (Commonalities 0.6) - +**Conventions**: +The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. ## Methodology 1. Walked both design guides section by section, extracting every machine-checkable requirement From df625b4debc3406ac0c745c1ca748336452c1a5e Mon Sep 17 00:00:00 2001 From: Herbert Damker Date: Mon, 23 Mar 2026 16:45:53 +0100 Subject: [PATCH 17/25] Update documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md Co-authored-by: Kevin Smith --- .../CAMARA-Validation-Framework-Testing-Guidelines-Audit.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md index 99d769d..5e8fdbc 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Testing-Guidelines-Audit.md @@ -3,7 +3,8 @@ **Date**: 2026-03-19 **Scope**: Machine-checkable rules from API-Testing-Guidelines.md **Sources**: API-Testing-Guidelines.md, .gherkin-lintrc (25 rules), api_review_validator_v0_6.py (test checks V6-071–V6-080) - +**Conventions**: +The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. ## Methodology 1. Walked API-Testing-Guidelines.md section by section, extracting every machine-checkable requirement From 9c9abc39bc7c1661c94306372e82d8017e23011c Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 24 Mar 2026 15:31:34 +0100 Subject: [PATCH 18/25] docs: clarify bundling mechanism and add tool requirements for component-based inclusion Clarify in section 3.1 that bundling converts external $ref to internal $ref with component inclusion in the appropriate components/ subsection, covering all component types (schemas, securitySchemes, headers, parameters, responses, examples) with deduplication. Add explicit bundling tool requirements in section 8.4 step 6: component-based inclusion (not raw JSON pointer inlining) and deduplication of multiply-referenced components. --- .../CAMARA-Validation-Framework-Detailed-Design.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 1eaafc1..5baf3d9 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -229,7 +229,7 @@ Remaining inventory work: The framework uses **bundling** (external ref resolution only), not full dereferencing: -- **Bundling**: Resolves external `$ref` — pulls content from `code/common/`, `code/modules/`, and other local files into the document. Internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`) are preserved. +- **Bundling**: Resolves each external `$ref` by placing the referenced component into the appropriate `components/` subsection of the output document and replacing the external `$ref` with an internal `$ref` (e.g., `$ref: '../common/CAMARA_common.yaml#/components/schemas/ErrorInfo'` becomes `$ref: '#/components/schemas/ErrorInfo'`). This applies to all component types — schemas, securitySchemes, headers, parameters, responses, examples — not just schemas. When the same external component is referenced multiple times, it is included once and all references point to the single internal definition (deduplication). Internal `$ref` already present in the source are preserved unchanged. - **Full dereferencing**: Resolves all `$ref` including internal ones, producing a flat document with zero `$ref` and massive duplication. The framework must **not** use full dereferencing. Preserving internal `$ref` ensures that: @@ -941,12 +941,14 @@ If all API spec files are syntactically valid and no external `$ref` is detected Conditional — only runs when external `$ref` to `code/common/` or `code/modules/` is detected in at least one API spec file. -The framework invokes an **external bundling tool** — it does not implement its own OpenAPI bundler. The tool must satisfy two requirements: +The framework invokes an **external bundling tool** — it does not implement its own OpenAPI bundler. The tool must satisfy the following requirements: 1. **External ref resolution only** (see section 3.1): Resolve `$ref` to `code/common/`, `code/modules/`, and other local files. Preserve all internal `$ref` (`#/components/schemas/...`, `#/components/responses/...`). Full dereferencing must not be used. -2. **Source map production**: Produce a mapping from bundled output regions back to source file locations. This is needed for line number translation in the output pipeline (section 9.5). This is a selection criterion for tool evaluation — the chosen tool must either support source maps natively or be wrappable to produce them. +2. **Component-based inclusion**: Each resolved external `$ref` must be placed into the appropriate `components/` subsection (schemas, securitySchemes, headers, parameters, responses, examples) and replaced with an internal `$ref` pointing to that component. Raw JSON pointer inlining (where the first occurrence is expanded in place and subsequent references point to that arbitrary location) is not acceptable — it breaks Spectral rules that expect components in their standard locations. +3. **Deduplication**: When the same external component is referenced from multiple locations, the tool must include it once in `components/` and replace all occurrences with internal `$ref` to that single definition. +4. **Source map production**: Produce a mapping from bundled output regions back to source file locations. This is needed for line number translation in the output pipeline (section 9.5). This is a selection criterion for tool evaluation — the chosen tool must either support source maps natively or be wrappable to produce them. -The specific tool choice (e.g., redocly, swagger-cli, prance, custom wrapper) is deferred to implementation, evaluated against these two requirements. +The specific tool choice (e.g., Redocly CLI, vacuum, custom wrapper) is deferred to implementation, evaluated against these requirements. **Cache sync validation** runs as part of bundling: the content of `code/common/` files is compared against the expected content for the declared `commonalities_release` version. A mismatch produces a finding — `warning` in standard profile, `error` in strict profile. From 85c6b22d2aeedfb2e03c8e786dea0d22d48e37be Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 24 Mar 2026 16:03:11 +0100 Subject: [PATCH 19/25] docs: decouple common file list from current Commonalities structure Replace hardcoded file list (CAMARA_common.yaml, notification-as-cloud-event.yaml) with reference to Commonalities#603 restructuring. The tooling must map commonalities_release versions to the correct source files without assuming a fixed file list. --- .../CAMARA-Validation-Framework-Detailed-Design.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 5baf3d9..132ef4b 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -260,11 +260,9 @@ In the MVP, some parts may still be manual — providing the correct copy in `co Three categories of shared schema dependencies exist, each with different characteristics: **Commonalities** (well-known, hardcoded in tooling): -The Commonalities repository provides shared schemas that are well-known to automation tooling. Currently two files are relevant: -- `CAMARA_common.yaml` — common data types, error responses, headers -- `notification-as-cloud-event.yaml` — CloudEvents notification schema +The Commonalities repository provides shared schemas that are well-known to automation tooling. The primary file is `CAMARA_common.yaml` (common data types, error responses, headers, security schemes). The exact set of common files and their directory structure within the Commonalities repository is subject to the ongoing restructuring ([Commonalities#603](https://github.com/camaraproject/Commonalities/issues/603)) and may evolve — the tooling must not assume a fixed file list but should be configurable per Commonalities version. -These files, their source location in the Commonalities repository, and the mapping from `release-plan.yaml.dependencies.commonalities_release` to the correct version are built into the tooling. No per-repository configuration is needed. +The mapping from `release-plan.yaml.dependencies.commonalities_release` to the correct source files and their locations is built into the tooling. No per-repository configuration is needed. **ICM** (version compatibility constraint): Identity and Consent Management schemas are currently contained within Commonalities files — there are no separate ICM files to cache. The `dependencies.identity_consent_management_release` in `release-plan.yaml` is a version compatibility constraint (potentially `>= x.y.z`) rather than a file-caching relationship. The exact nature of this dependency requires further discussion. From 4e9eac3093ec29c488ca316ac074b59656439743 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 24 Mar 2026 16:33:59 +0100 Subject: [PATCH 20/25] docs: reframe sub-project commons as possible future extension Replace concrete dependency declaration design with a neutral statement that the framework should not preclude cross-repository schema sharing, without committing to a specific organizational model. The how/where is an open question outside the scope of this design. --- .../CAMARA-Validation-Framework-Detailed-Design.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 132ef4b..a71d5ee 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -267,13 +267,8 @@ The mapping from `release-plan.yaml.dependencies.commonalities_release` to the c **ICM** (version compatibility constraint): Identity and Consent Management schemas are currently contained within Commonalities files — there are no separate ICM files to cache. The `dependencies.identity_consent_management_release` in `release-plan.yaml` is a version compatibility constraint (potentially `>= x.y.z`) rather than a file-caching relationship. The exact nature of this dependency requires further discussion. -**Sub-project commons** (extensible, declared per repository): -Sub-projects may define common schemas shared across their API repositories (e.g., a device API family sharing common device type definitions). These dependencies must be declarable without requiring changes to the automation tooling. Each sub-project dependency requires: -- Source repository -- Release tag or version -- Array of files to consume - -This extensible model requires a dependency declaration format, either within `release-plan.yaml` or as a separate manifest. The schema design is a follow-up topic for the bundling design document. +**Cross-repository commons** (possible future extension): +In the future, groups of related API repositories may share common schemas beyond those provided by Commonalities (e.g., QoS-related type definitions shared between quality-on-demand and qos-profiles). How such cross-repository schemas would be organized — in a dedicated common repository, within API repositories themselves, or as additional files in the Commonalities repository — is an open question outside the scope of this design. The framework architecture should not preclude this extension, but no implementation is needed until a concrete use case is agreed. #### File caching strategy From cfcc77a8dbfbd2b9fed6c8f4528a6908cdf010ec Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 24 Mar 2026 16:46:28 +0100 Subject: [PATCH 21/25] docs: clarify step reference points to validation pipeline (section 6.2) --- .../CAMARA-Validation-Framework-Detailed-Design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index a71d5ee..7471704 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -319,7 +319,7 @@ Placeholder replacement with defined values could be introduced together with bu Bundling integrates into the rule architecture (Requirements section 5) without requiring changes to the context model or rule metadata: -- **Step assignment**: Each rule runs in either step 1 (pre-bundling validation) or step 3 (full validation). Assignment is an implementation detail — the framework knows which checks belong to which step. +- **Pipeline step assignment**: Each rule runs in either pipeline step 1 (pre-bundling validation) or step 3 (full validation) as defined in Requirements section 6.2. Assignment is an implementation detail — the framework knows which checks belong to which step. - **No new context fields**: The context model from Requirements section 2.2 is sufficient. Whether external refs existed and were resolved is an implementation concern, not a rule applicability condition. - **Cache sync is a check, not context**: The cache synchronization validation (section 3.2) produces findings (warning or error depending on profile). It is not a context field consumed by other rules. - **Spectral ruleset selection**: The `commonalities_release` field (already in the context model) drives Spectral ruleset pre-selection (section 3.3). No additional metadata is needed. From d817b231f5ec594d654a150f88e92108d5f0556f Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 24 Mar 2026 16:50:29 +0100 Subject: [PATCH 22/25] =?UTF-8?q?docs:=20fix=20API=20name=20in=20examples?= =?UTF-8?q?=20(qos-on-demand=20=E2=86=92=20quality-on-demand)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../CAMARA-Validation-Framework-Detailed-Design.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 7471704..1501d9b 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -892,12 +892,12 @@ apis: target_api_maturity: "stable" # derived api_pattern: "request-response" # detected from spec spec_file: "code/API_definitions/qos-booking.yaml" - - api_name: "qos-on-demand" + - api_name: "quality-on-demand" target_api_version: "0.11.0-alpha.1" target_api_status: "initial" target_api_maturity: "initial" api_pattern: "explicit-subscription" - spec_file: "code/API_definitions/qos-on-demand.yaml" + spec_file: "code/API_definitions/quality-on-demand.yaml" # Workflow metadata workflow_run_url: "https://github.com/camaraproject/QualityOnDemand/actions/runs/12345" @@ -990,10 +990,10 @@ All engine outputs are normalized into a common findings format before post-filt engine_rule: "camara-parameter-casing-convention" # native engine rule name level: error # engine-reported level (before post-filter) message: "Path segment 'qualityOnDemand' should be kebab-case" - path: "code/API_definitions/qos-on-demand.yaml" + path: "code/API_definitions/quality-on-demand.yaml" line: 47 # line in source file (mapped back if bundled) column: 5 # column (if available from engine) - api_name: "qos-on-demand" # which API this finding belongs to + api_name: "quality-on-demand" # which API this finding belongs to hint: "Use kebab-case: /quality-on-demand/{sessionId}" ``` @@ -1101,7 +1101,7 @@ Structure: | API | Errors | Warnings | Hints | |-----|--------|----------|-------| | qos-booking | 0 | 2 | 1 | -| qos-on-demand | 1 | 0 | 3 | +| quality-on-demand | 1 | 0 | 3 | ### Findings @@ -1109,7 +1109,7 @@ Structure: | Rule | File | Line | Message | Hint | |------|------|------|---------|------| -| 042 | qos-on-demand.yaml | 47 | Path segment should be kebab-case | Use: /quality-on-demand | +| 042 | quality-on-demand.yaml | 47 | Path segment should be kebab-case | Use: /quality-on-demand | #### Warnings ... From 1f9707dbbabbfc828e5a0572dd12e127a44d16dc Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 24 Mar 2026 23:12:51 +0100 Subject: [PATCH 23/25] docs: add testing guidelines and common schemas to authoritative sources Add API-Testing-Guidelines.md and artifacts/CAMARA_common.yaml as authoritative sources for validation rules. Note that further Commonalities artifacts depend on restructuring (Commonalities#603). --- .../CAMARA-Validation-Framework-Requirements.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md index 6b968c5..a034490 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md @@ -198,6 +198,10 @@ The validation rules are derived from the following upstream documents. Rules mu **Commonalities** (version-dependent — r3.4 = v0.6.x, r4.1/r4.2 = v0.7.x): - `CAMARA-API-Design-Guide.md` — API design rules, versioning, error handling, naming conventions - `CAMARA-API-Event-Subscription-and-Notification-Guide.md` — subscription and notification patterns +- `API-Testing-Guidelines.md` — test file structure, naming, and content requirements +- `artifacts/CAMARA_common.yaml` — canonical common schemas (bundling reference) + +Further Commonalities artifacts (API template, subscription template) can be added after the Commonalities restructuring ([Commonalities#603](https://github.com/camaraproject/Commonalities/issues/603)) has landed and files are stable. **Release Management:** - `release-plan-schema.yaml` — field definitions and allowed values for release-plan.yaml From 87b09b69d9b4121b2bbaf9ee2105f541ed38fe96 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Tue, 24 Mar 2026 23:25:26 +0100 Subject: [PATCH 24/25] docs: add release readiness checks to release review PR table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Clarify that artifact presence checks (per target API status) do not run on the release review PR — already validated at snapshot creation time and artifacts are immutable on the snapshot branch. --- .../CAMARA-Validation-Framework-Requirements.md | 1 + 1 file changed, 1 insertion(+) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md index a034490..95937ae 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md @@ -465,6 +465,7 @@ A release review PR is created by release automation on the `release-review/rX.Y | release-plan.yaml checks | No | Already validated at snapshot creation time | | Bundling validation | No | Source files are immutable | | Cache sync validation | No | Already validated at snapshot creation time | +| Release readiness checks (artifact presence by target API status) | No | Already validated at snapshot creation time; artifacts cannot be added or removed on the snapshot branch | **File restriction check**: The framework examines the PR diff and produces an error if any file outside `CHANGELOG.md` (or `CHANGELOG/` directory) and `README.md` is modified. From ef6dc5601a2d49721644391c5939a99661736c92 Mon Sep 17 00:00:00 2001 From: Herbert Damker <52109189+hdamker@users.noreply.github.com> Date: Wed, 25 Mar 2026 00:42:18 +0100 Subject: [PATCH 25/25] docs: add Appendix B for local validation considerations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add implementation guardrails for UC-03 (local validation) to the detailed design. Classifies processing steps by environment portability, adds `local` trigger type to the rule metadata vocabulary, and defines five constraints to keep the framework implementation reusable for local tooling. Move UC-03 from MVP to post-MVP — the design guardrails are MVP scope, but the local entry point and tooling are not. --- ...RA-Validation-Framework-Detailed-Design.md | 88 ++++++++++++++++++- ...AMARA-Validation-Framework-Requirements.md | 7 +- 2 files changed, 90 insertions(+), 5 deletions(-) diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md index 1501d9b..ee8a94b 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Detailed-Design.md @@ -22,7 +22,7 @@ hint: "Use kebab-case for all path segments: /my-resource/{resourceId}" applicability: # only list fields that constrain; omitted = no constraint branch_types: [main, release] - trigger_types: [pr, dispatch] + trigger_types: [pr, dispatch, local] # ... further conditions as needed conditional_level: @@ -870,7 +870,7 @@ The context builder does not expose fork identity as a context field — no vali # Validation context — all fields always present repository: "QualityOnDemand" # repo name without owner prefix branch_type: "main" # main | release | maintenance | feature -trigger_type: "pr" # pr | dispatch | release-automation +trigger_type: "pr" # pr | dispatch | release-automation | local profile: "standard" # advisory | standard | strict stage: "standard" # from central config (disabled | advisory | standard) @@ -1331,3 +1331,87 @@ Complete naming convention rules from CAMARA-API-Design-Guide.md and CAMARA-API- | Scope names | kebab-case with : separators | `qod:sessions:create` | 6.6 | — | v0_6 validator only | | Event type | org.camaraproject.\.\.\ | `org.camaraproject.device-roaming-subscriptions.v1.roaming-status` | Event Guide 3.1 | — | Gap | | Examples (named) | SCREAMING_SNAKE_CASE | `SESSION_CREATION_EXAMPLE_WITH_DEVICE_RESPONSE` | OAS 3.0.3 | — | Gap | + +--- + +## Appendix B: Local Validation Considerations + +This appendix addresses UC-03 ("Run most validation rules locally, on local clones, via scripts"). Its purpose is not to design the local CLI — that is implementation work — but to identify where the framework implementation must stay modular to avoid duplicated effort when local tooling is built. + +### B.1 Scope + +**In scope:** Implementation guardrails for the framework that keep local reuse viable. + +**Out of scope:** Entry point design (script, CLI, Makefile), tooling distribution and versioning for local use, local cache management for Commonalities artifacts, IDE integration, performance targets. + +### B.2 Environment Boundary + +The 8 processing steps (section 8) fall into three categories: + +**Environment-neutral** — reusable locally without modification: + +- Spectral engine invocation and output parsing +- Python check execution +- Bundling pipeline (external ref resolution, cache sync validation) +- Rule metadata lookup and condition evaluation +- Post-filter: applicability matching, conditional level resolution, profile application +- Findings list construction + +**Workflow-only** — require GitHub infrastructure: + +- Tooling ref resolution (OIDC `job_workflow_sha`, version tag fallback) +- Central config lookup (tooling checkout, repository stage) +- PR metadata detection (`is_release_review_pr` from target branch, `release_plan_changed` from PR diff API) +- Output surfacing: check run annotations, PR comments, commit status + +**Adaptable** — work locally with alternative input: + +- Context building: branch type is derivable from `git branch --show-current` using the same pattern logic. Trigger type and PR-specific fields need sensible defaults (see B.4) +- Repository checkout: a local clone replaces the workflow checkout step + +### B.3 `local` Trigger Type + +The `trigger_types` vocabulary (section 1.1) includes `local` alongside `pr`, `dispatch`, and `release-automation`. + +Semantics: +- Profile is always `advisory` (nothing blocks) +- No GitHub event context, no PR metadata +- `is_release_review_pr` defaults to `false` +- `release_plan_changed` defaults to `null` (unknown — rules requiring it are skipped) + +Most rules have no `trigger_types` constraint and apply in all contexts including `local`. Rules that explicitly list only `[pr]` are automatically excluded. + +### B.4 Local Context Derivation + +How context fields map when running locally (no GitHub event): + +| Field | Workflow source | Local source | Notes | +|-------|-----------|-------------|-------| +| `branch_type` | `github.base_ref` / `github.ref_name` | `git branch --show-current` | Same pattern-matching logic | +| `trigger_type` | GitHub event name / `mode` input | Fixed: `local` | | +| `profile` | Derived from trigger type | Fixed: `advisory` | | +| `stage` | Central config lookup | Not applicable | Local runs are not gated by rollout stage | +| `target_release_type` | release-plan.yaml | release-plan.yaml | Identical | +| `commonalities_release` | release-plan.yaml | release-plan.yaml | Identical — drives Spectral ruleset selection | +| `icm_release` | release-plan.yaml | release-plan.yaml | Identical | +| `target_api_status` | release-plan.yaml (per-API) | release-plan.yaml (per-API) | Identical | +| `target_api_maturity` | Derived from version | Derived from version | Identical | +| `api_pattern` | Detected from spec content | Detected from spec content | Identical | +| `is_release_review_pr` | PR target branch | Fixed: `false` | | +| `release_plan_changed` | PR diff via API | Fixed: `null` | Rules requiring this are skipped | + +The majority of context fields (8 of 12) are file-derived and work identically in both environments. The workflow-only fields (`stage`, `is_release_review_pr`, `release_plan_changed`) degrade to safe defaults that skip inapplicable rules rather than producing incorrect results. + +### B.5 Implementation Guardrails + +These constraints apply to the framework implementation to preserve local reusability: + +1. **Engine invocation must be callable without GitHub context.** Spectral and Python check runners accept file paths and a context object as inputs. They must not read `GITHUB_EVENT_PATH`, `GITHUB_OUTPUT`, or other Actions environment variables directly. Workflow-level steps pass these values in; engines do not reach for them. + +2. **Context construction must be separable from GitHub event parsing.** The context builder must support being fed from either a GitHub event payload or from local defaults combined with git-derived values. A clean boundary: the workflow caller resolves GitHub-specific inputs and hands them to the context builder; the builder itself is environment-neutral. + +3. **Output formatting must support terminal output.** The post-filter produces a structured findings list. Rendering to workflow summary markdown and rendering to terminal are separate formatters behind a common findings-list interface. The terminal formatter is the natural output for local runs. + +4. **Bundling must not depend on the workflow artifacts API.** The bundling pipeline reads files and writes files. Artifact upload is a separate workflow step, not embedded in the bundler. Locally, bundled output is written to a directory. + +5. **Rule metadata and Spectral rulesets must be loadable from a local path.** The reusable workflow resolves tooling via OIDC ref or version tag and checks out into `.tooling/`. Locally, the user either has a tooling clone or a downloaded copy. The engine invocation code must accept a configurable base path for rule metadata and rulesets, not hardcode the `.tooling/` checkout location. diff --git a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md index 95937ae..8dad0ac 100644 --- a/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md +++ b/documentation/SupportingDocuments/CAMARA-Validation-Framework-Requirements.md @@ -98,7 +98,7 @@ A rule is skipped silently if its conditions don't match the current context. Mu | Field | Type | Values | Source | |-------|------|--------|--------| | `branch_types` | array | `main`, `release`, `maintenance`, `feature` | Target branch (PR) or checked branch (dispatch) | -| `trigger_types` | array | `pr`, `dispatch`, `release-automation` | How validation was invoked | +| `trigger_types` | array | `pr`, `dispatch`, `release-automation`, `local` | How validation was invoked | | `target_release_type` | array | `none`, `pre-release-alpha`, `pre-release-rc`, `public-release`, `maintenance-release` | `repository.target_release_type` in release-plan.yaml | | `target_api_status` | array | `draft`, `alpha`, `rc`, `public` | `apis[].target_api_status` in release-plan.yaml (per-API) | | `target_api_maturity` | array | `initial` (0.x.y), `stable` (x.y.z, x>=1) | Derived from `apis[].target_api_version` | @@ -157,11 +157,11 @@ The MVP replaces `pr_validation` v0 and delivers the minimum useful validation o ### MVP includes -- UC-01 through UC-07 (contributor and codeowner use cases) +- UC-01, UC-02, UC-04 through UC-07 (contributor and codeowner use cases) - UC-13 (incremental rollout with central config) - UC-15 (feature branch testing with pinned refs) - Profiles: advisory and standard -- Execution contexts: PR (fork-to-upstream), PR (upstream branch), dispatch (upstream repo), dispatch (fork repo) +- Execution contexts: PR (fork-to-upstream), PR (upstream branch), dispatch (upstream repo), dispatch (fork repo), local (design guardrails only — see Detailed Design Appendix B) - Existing Spectral rules and YAML linting - Understandable output with fix hints - Caller workflow with all triggers from day one (PR, dispatch), deployed alongside `pr_validation` v0 @@ -170,6 +170,7 @@ The MVP replaces `pr_validation` v0 and delivers the minimum useful validation o ### Post-MVP priorities - UC-08, UC-09 (release automation strict gates, section 11) — high priority, requires strict profile; depends on PR integration being operational first +- UC-03 (local validation) — framework design supports it (Detailed Design Appendix B), entry point and tooling are post-MVP - UC-10 (regression testing against release branches) — rule developer tooling - UC-14 (repository configuration validation) — admin tooling - Automated cache synchronization for `code/common/` and strict version enforcement — bundling itself (ref resolution via `$ref`) is MVP scope (section 6)