Framework detection architecture: spec-driven horizontal layer (like tree-sitter)

[peteromallet]

Context

PR #414 adds Next.js framework detection — the first framework-specific detector in the codebase. The scanners and detection logic are solid, but before we establish the pattern that future frameworks will follow, we should think about what scales to 20 languages × 20 frameworks.

Current plugin architecture

The codebase has two plugin depth levels and several horizontal layers:

                        Shallow                              Deep
                        (generic_lang)                       (LangConfig class)
                        ─────────────────────────────────────────────────
Language depth:         JS, Nim, Zig, Bash                   TS, Go, Rust, C++

Horizontal layers (work at any depth):
  Tree-sitter:          ✓ (via treesitter_spec=)             ✓ (via *all_treesitter_phases())
  Tool specs:           ✓ (via tools=[...])                  ✓ (via make_tool_phase())

Tree-sitter is the key precedent: it's spec-driven, generates DetectorPhase objects, and works with both shallow and deep plugins. Adding a new language's tree-sitter support = adding a spec, not modifying infrastructure.

The question: what are frameworks?

Most framework-specific issues fall into four categories:

1. File convention checks — "error.tsx needs use client", "Django views belong in views.py". File pattern + content pattern. Mechanical.
2. API misuse — "use server in a client module", "useEffect only to sync state". String or AST pattern matches.
3. External tool output — next lint, django check. Tool running + output parsing.
4. Architectural violations — "mixing pages/ and app/ router", "circular model imports". Needs dep graph / cross-file analysis.

Categories 1-3 are shallow — pattern matching and tool running. Category 4 needs the language plugin's deep infrastructure (dep graph, zone map), but doesn't need deep framework-specific wiring — it just consumes what the language plugin already provides.

Frameworks aren't a new depth level. They're a horizontal layer, like tree-sitter.

Proposed architecture

_framework/
  frameworks/
    types.py              # FrameworkSpec, ScannerRule, DetectionConfig
    registry.py           # registered framework specs
    detection.py          # generic "is this framework present?"
                          #   reads package.json, requirements.txt, Cargo.toml, go.mod
                          #   keyed by ecosystem
    phases.py             # framework_phases(lang_name) → list[DetectorPhase]
                          #   analogous to all_treesitter_phases()
    specs/
      nextjs.py           # Next.js: detection config + scanner rules + issue templates
      django.py           # Django
      rails.py            # Rails
      spring.py           # Spring
      gin.py              # etc.

A framework spec would be data-driven, like a tree-sitter spec:

FrameworkSpec(
    id="nextjs",
    ecosystem="node",
    detection=DetectionConfig(
        dependencies=["next"],
        config_files=["next.config.js", "next.config.mjs"],
        markers=["app/", "pages/"],
    ),
    scanners=[
        ScannerRule(
            id="use_server_in_client",
            extensions=[".tsx", ".jsx", ".ts", ".js"],
            check=check_use_server_in_client,
            tier=2,
            confidence="high",
            summary_template="'use server' directive in client module {file}",
        ),
        # ...more scanner rules
    ],
    tool_integration=ToolConfig(
        cmd="npx next lint --format json",
        parser=parse_next_lint,
        slow=True,    # respects --skip-slow
    ),
)

Language plugins opt in with one line:

# Deep plugin (typescript/__init__.py)
phases = [
    ...existing phases...,
    *framework_phases("typescript"),
]

# Shallow plugin (javascript/__init__.py)
generic_lang(
    name="javascript",
    ...,
    frameworks=True,
)

Why this scales

• Adding a framework = adding a spec file. No language plugin modifications needed.
• Cross-language coverage is automatic. Scanner rules declare which extensions they apply to, so TS and JS both get Next.js checks from the same spec. No cross-plugin imports, no duplication.
• Detection is ecosystem-based. Node reads package.json, Python reads requirements.txt/pyproject.toml, Go reads go.mod. Centralized and consistent.
• The copy-paste problem disappears. The spec→issue mapping is generic (like how generic_support maps tool specs to issues), not 25 hand-coded blocks.
• Specs are testable in isolation. Test the spec, not the wiring.
• Works at any plugin depth. Shallow plugins get framework support via one flag. Deep plugins get it via one line. Scanners can optionally consume the dep graph if available.

The updated spectrum

                        Shallow                              Deep
                        (generic_lang)                       (LangConfig class)
                        ─────────────────────────────────────────────────
Language depth:         JS, Nim, Zig, Bash                   TS, Go, Rust, C++

Horizontal layers (work at any depth):
  Tree-sitter:          ✓ (via treesitter_spec=)             ✓ (via *all_treesitter_phases())
  Frameworks:           ✓ (via frameworks=True)              ✓ (via *framework_phases())
  Tool specs:           ✓ (via tools=[...])                  ✓ (via make_tool_phase())

Migration path

1. MacHatter1's Next.js scanners (PR feat(frameworks): add shared Next.js detector pipeline + enforced next lint (TS/JS) #414) become a nextjs.py framework spec
2. Existing React detectors in typescript/detectors/react/ eventually become a react.py spec
3. Both TS and JS get framework coverage from the same specs
4. The hand-coded phases_smells.py React wiring gets replaced by *framework_phases("typescript")

Open questions

• Should the check function in scanner rules receive the full LangRuntimeContract (for dep graph access), or just file content + path?
• How do we handle framework-specific fixers? (e.g., auto-adding 'use client' to error boundaries)
• Should framework specs support "requires deep plugin" scanners that gracefully skip for shallow plugins?

Feedback welcome — especially from @MacHatter1 who has already done the hard work of building the first framework detector and has the best sense of what the scanner authoring experience should feel like.

[MacHatter1]

@peteromallet this is a really good write-up. Coming straight off building the first framework detector in #414, I’m +1 on the core framing: frameworks should be a horizontal layer (tree-sitter/tool-spec style), spec-driven, and not something that forces JS↔TS plugin imports.

A few things I learned the hard way implementing the Next.js pass that probably shape the spec/API:

• A bunch of “framework” checks are truly shallow (file convention + content patterns) and should be shareable across .js/.jsx/.ts/.tsx without duplicating logic. In feat(frameworks): add shared Next.js detector pipeline + enforced next lint (TS/JS) #414 I initially missed .js/.jsx for error boundaries + middleware and it immediately showed why this needs to be spec-driven.
• Some checks need a tiny bit of framework context derived from detection evidence (for Next.js: app_roots vs pages_roots). That context isn’t language-specific, but it’s also not just “package.json says next is installed”. I’d want the spec to have an optional “derive context” hook so scanners don’t re-derive this ad hoc.
• We also need to be careful about false positives on “directive-like” strings. The 'use server' stuff is subtle (inline server actions are valid; comments/string literals should not match). Having shared helpers at the framework layer (strip comments / mask strings / prologue parsing) will pay for itself quickly.

On the open questions:

• What should check receive? I’d avoid handing every rule a raw LangRuntimeContract, but I do want to let deep rules opt into dep graph / zone map when available. I think a small context object + explicit requirements is the right compromise:

  ScannerRule(..., requires=("dep_graph",), check=...)

  If requirements aren’t met (shallow plugin), skip the rule but surface it as a coverage/capability warning or confidence drop — not a silent skip.

• Fixers: +1 to supporting them eventually, but I’d start with “obviously safe, local, deterministic” things (e.g. add 'use client' to error.* / global-error.* when missing). Anything migration-y (router moves, exports changes) should stay as findings.

• Tool integrations: next lint is basically a slow tool spec with a parser. I’d rather framework specs reuse the existing tool plumbing (or share a common ToolSpec type) so we don’t end up with two parallel “how to run tools” systems. slow=True needs to map to DetectorPhase.slow and respect --skip-slow uniformly (that bit mattered in practice).

On detection: I like your spec model, but I’d lean toward “present frameworks” rather than “primary framework” as the only shape. “Primary” is useful for some UI/logging and maybe to gate very expensive stuff, but in real repos multiple specs can be true at once. Maybe the detection layer returns all candidates + scores, and specs can optionally declare an exclusivity group/priority if something truly must be single-winner.

On the React migration note: +1 to moving the current TS React checks out of hand-wired phases_smells.py over time, but I’d treat React as “library rules” rather than a framework per se (since it shows up in lots of stacks). If the spec layer can represent both “framework” and “library” cleanly, I think React becomes a great second proof point after Next.js:

• Some React checks are genuinely cross-language (JS/TS) and should come “for free” once they’re specs.
• Some might stay deep-only (or at least optional) if they depend on language-specific infra — which is another good test of the requires=(...)/capability story.

If we want a low-risk proof of concept, I’d suggest:

1. land minimal _framework/frameworks/ infra (types/registry + framework_phases(...)),
2. convert Next.js into the first spec by reusing the scanners we already have,
3. wire JS shallow + TS deep via the one-liners you sketched, and only then
4. migrate anything else (React, Django, etc).

Happy to take a first pass at turning the Next.js work into a specs/nextjs.py once we align on (a) detection output shape (present vs primary) and (b) the scanner context/requirements mechanism.

[peteromallet]

@MacHatter1 Thanks for the thorough reply — really useful having your perspective from building the first framework detector hands-on.

We're largely aligned. A few notes on the design points:

On requires= and what check receives: I agree rules should declare their capabilities explicitly, and missing capabilities should surface as coverage/confidence degradation rather than silent skips. My one nuance is on the runtime interface — I'd avoid inventing a second parallel context object. The existing architecture already standardizes phases around DetectorPhase.run(path, lang) with LangRuntimeContract, so framework check functions should receive the same (Path, LangRuntimeContract) signature. The requires= field would gate whether the rule runs, not reshape what it receives. That keeps the framework layer as a thin spec-to-DetectorPhase adapter (like tree-sitter), not a separate subsystem.

On detection shape: Agreed on "present frameworks" over "primary framework." I'd keep it even simpler than scores though — binary present/absent with evidence strings for debugging. Framework detection is mostly deterministic (does package.json list next?), not probabilistic. For mutual exclusion cases (CRA vs Next.js), an excludes= field on the spec handles it without needing priority groups.

On fixers and tool integration: Agreed on both — reuse FixerConfig for file-local fixes, reuse ToolSpec/make_tool_phase() for framework tools. No parallel systems.

On React: I don't think we need to distinguish "library" vs "framework" in the type system. A FrameworkSpec is just "scanner rules gated on a detection condition" — React fits that shape fine. Optional kind label for display if we want it, but it shouldn't affect behavior.

If you want to take a first pass at turning the Next.js work into a specs/nextjs.py under this architecture, that would be hugely appreciated. Let me know if you have any questions on the above.