CI-02: wire Xenon complexity gate into CI using installed Radon baseline

[longieirl]

Summary

Wire a complexity gate into CI using Xenon (which wraps the already-installed Radon). Radon is in requirements/ci.txt but no CI job enforces any complexity threshold. Three functions are already above grade C — these need to be documented as known exemptions before the gate is turned on.

Background

Radon is installed (radon==6.0.1 in requirements/ci.txt) but is not invoked in any CI job. Xenon provides a CI-friendly exit-code wrapper around Radon's cyclomatic complexity scores, making it trivial to add as a blocking gate.

Current complexity baseline

Running radon cc packages/parser-core/src -s --min B today against the full codebase:

Average complexity: A (3.67) across 595 analysed blocks — a healthy baseline.

Outliers above grade C (the proposed gate threshold):

Location	Grade	Score	Decision
analysis/table_detector.py:149 TableDetector._detect_text_based_table	E	32	Temporary scoped exemption; dedicated refactor issue required (see Q9 decision)
commands/init.py:12 init_directories	D	26	Investigate before accepting — see Q10 decision
templates/template_detector.py:446 TemplateDetector.get_detection_explanation	D	22	Review during implementation — explanation/diagnostic logic may be justified
templates/template_detector.py:227 TemplateDetector.detect_template	C	18	At gate boundary
templates/template_registry.py:197 TemplateRegistry.from_multiple_directories	C	18	At gate boundary

Grade C functions (12 total — at threshold, would need review): spread across column_analyzer.py, iban_spatial_filter.py, template_model.py, template_detector.py, template_registry.py, loan_reference_detector.py, iban_detector.py.

Decisions made

Q9 — _detect_text_based_table (grade E, score 32)

Decision: Temporary scoped exemption — NOT || true advisory mode for the whole gate.

• Keep Xenon blocking for the entire repo from day one (gate has credibility)
• Add one explicit, temporary per-file exemption for table_detector.py via --exclude or config
• Create a dedicated refactor issue in the same PR with:
  • A clear exit condition: "exemption removed once _detect_text_based_table is refactored"
  • A concrete target date
  • A prerequisite: characterisation tests must be written before any structural changes
• Exemption must be commented explicitly: "temporary — remove once _detect_text_based_table is refactored (see issue #XXX)"

Why not || true: advisory gates lose credibility and tend to stay advisory permanently.
Why not refactor now: PDF parser hotspot without targeted characterisation tests — wrong sequencing, high regression risk.
Why not a naked exemption: poor governance without a tracked refactor plan.

Q10 — init_directories (grade D, score 26)

Decision: Investigate before deciding. Do not auto-exempt.

Sequence:

1. Read commands/init.py:12 — inspect the branches
2. Determine whether the branches are setup orchestration (justified) or misplaced decision-heavy logic (extractable)
3. Exempt only if the complexity is justified and stable — document inline why it is acceptable
4. Refactor only if extractable logic is found — do not force speculative cleanup

Rationale: Grade D in a commands/init file is suspicious enough to inspect. Auto-exempting without reading undermines gate credibility; speculative refactoring of stable init logic introduces unnecessary risk.

Use Xenon with the following thresholds:

xenon --max-average A --max-modules B --max-absolute C packages/parser-core/src

Flag	Threshold	Meaning
--max-average	A	Average complexity across all blocks must stay ≤ A (current: 3.67 — comfortable)
--max-modules	B	No module-level average may exceed B
--max-absolute	C	No single function/method may exceed C (blocks D/E/F new additions)

This configuration would currently fail due to the D/E outliers. The gate is introduced with a scoped exemption for table_detector.py only — not || true on the whole step.

Proposed Changes

1. requirements/ci.txt

+xenon>=0.9.0

Xenon depends on Radon, which is already pinned. No version conflict.

2. .github/workflows/ci.yml — complexity gate step

- name: Complexity gate (Xenon)
  run: |
    xenon --max-average A --max-modules B --max-absolute C \
      --exclude packages/parser-core/src/bankstatements_core/analysis/table_detector.py \
      packages/parser-core/src
    # table_detector.py temporarily excluded — _detect_text_based_table grade E (score 32)
    # Remove exclusion once refactored. Tracked: https://github.com/longieirl/bankstatementprocessor/issues/XXX

Gate is blocking from day one. No || true.

3. Refactor issue (created in same PR)

A new issue must be created covering:

• Write characterisation tests for _detect_text_based_table first
• Then decompose the function
• Exit condition: remove the --exclude flag from the Xenon CI step
• Target date: defined at issue creation

4. Document remaining D outliers inline

Before removing || true, each D/E outlier should carry a comment:

# complexity: grade E (score 32) — heuristic table boundary detection;
# decomposition blocked by pdfplumber word-coordinate API coupling.
# Tracked: https://github.com/longieirl/bankstatementprocessor/issues/XXX
def _detect_text_based_table(self, ...):

Why not Radon directly

Radon's cc command has no built-in exit-code threshold. Xenon adds exactly that — one flag, one exit code, zero additional config. It is a thin wrapper and does not introduce new tooling philosophy.

Why mutmut is NOT included here

Mutation testing (mutmut) was evaluated as part of this gate review. It is not suitable as a CI PR gate for this codebase because:

• Runtime on the 1,420-test suite would be 30–90 minutes per run
• Flaky tests (if any) make mutation scores non-deterministic
• The value is in targeted pre-release audits of domain logic, not continuous gating

Recommended use: run mutmut run --paths-to-mutate packages/parser-core/src/bankstatements_core/domain/ locally before merging significant domain changes. No CI job needed.

Acceptance Criteria

• xenon>=0.9.0 added to requirements/ci.txt
• Xenon step added to CI as a blocking gate (no || true)
• table_detector.py excluded via --exclude with an explicit inline comment referencing the refactor issue
• Dedicated refactor issue created in the same PR: characterisation tests → decompose → remove exclusion; includes target date and exit condition
• init_directories (D/26) investigated — read function, then either exempt with inline justification or refactor if extractable logic found
• CI passes with scoped exclusion in place