tooling: replace black/isort/flake8 with ruff in CI and dev dependencies

[longieirl]

Background

Ruff is a Rust-based Python linter and formatter positioned as a drop-in replacement for black, isort, and flake8 (plus many plugins). It is already installed as a dev dependency in packages/parser-core but is not wired into CI and has no configuration — it is currently dead weight.

This issue tracks migrating the lint toolchain to consolidate on Ruff while keeping mypy for static type checking.

---

Current State

Dev dependencies (packages/parser-core/pyproject.toml [dev])

Tool	Version	Purpose
black	>=23.0.0,<27.0.0	Formatter
isort	>=5.12.0,<9.0.0	Import sorter
flake8	>=6.0.0,<8.0.0	Linter (pycodestyle + pyflakes)
mypy	>=1.8.0,<2.0.0	Static type checker
pyright	>=1.1.350	Static type checker (duplicate of mypy, not in CI)
ruff	>=0.0.265,<1.0.0	Installed but unused — no config, not in CI

packages/parser-free has no lint dev dependencies at all — they are installed inline during the CI step (pip install black isort flake8).

CI — lint-core job (.github/workflows/ci.yml)

black --check --diff src tests
isort --check-only --diff src tests
flake8 src tests --max-line-length=88 --extend-ignore=E203,W503,E501,W504,D,C420
mypy src --ignore-missing-imports
stubtest pdfplumber --allowlist stubtest-allowlist.txt

CI — lint-free job

pip install black isort flake8   ← installed ad-hoc, no pinned versions
black --check --diff src tests
isort --check-only --diff src tests
flake8 src tests --max-line-length=88 --extend-ignore=E203,W503,E501,W504,D,C420

Existing config

• [tool.black] — line-length = 88, target-version = ['py311', 'py312', 'py313']
• [tool.isort] — profile = "black", known_first_party = [...]
• [tool.mypy] — fully configured with strict settings (warn_return_any, disallow_untyped_defs, etc.)
• [tool.ruff] — does not exist

---

What Will Be Replaced

Removed	Replaced by	Notes
black	ruff format	Output is intentionally Black-compatible — no formatting changes
isort	ruff check --select I	Reimplementation, Black-compatible by default; known_first_party migrates to [tool.ruff.lint.isort]
flake8	ruff check	Strict superset; existing --extend-ignore flags map directly to Ruff rule codes
pyright	(removed, not replaced)	Redundant with mypy already wired in CI under strict config

Kept unchanged:

• mypy — static type checker, already configured strictly, already in CI
• bandit / safety — security scanning, separate concern
• stubtest — pdfplumber stub validation, no Ruff equivalent

---

What Will Be Added / Changed

1. [tool.ruff] config in both pyproject.toml files

Start with a rule set equivalent to the current flake8 + isort baseline:

[tool.ruff]
line-length = 88
target-version = "py311"

[tool.ruff.lint]
select = ["E", "W", "F", "I"]   # pycodestyle + pyflakes + isort
ignore = ["E203", "W503", "E501", "W504"]

[tool.ruff.lint.isort]
known-first-party = ["bankstatements_core"]   # or bankstatements_free

[tool.ruff.format]
# Black-compatible defaults — no additional config needed

2. CI steps simplified

Before (3 tools, ~30 s):

- run: black --check --diff src tests
- run: isort --check-only --diff src tests
- run: flake8 src tests --max-line-length=88 --extend-ignore=E203,W503,E501,W504,D,C420

After (1 tool, ~2 s):

- run: ruff format --check src tests
- run: ruff check src tests

3. Dev dependency cleanup (packages/parser-core)

Remove: black, isort, flake8, pyright
Keep: ruff (bump to current stable), mypy

4. lint-free gets proper dev dependencies

Add ruff and mypy to packages/parser-free/pyproject.toml under a [dev] group instead of ad-hoc pip install in CI.

5. Add mypy to lint-free CI job

Currently lint-free has no static type checking. Add mypy src --ignore-missing-imports after the ruff steps.

---

Impact Assessment

Area	Impact
Formatting output	None — ruff format is Black-compatible
Import ordering	None — ruff isort is Black-compatible
Lint rules caught	Same or more — ruff covers all current flake8 rules
CI speed	Faster — ruff is 10–100x faster than the three tools combined
Developer workflow	make format and make lint targets need updating in Makefile
Pre-commit hooks	Update .pre-commit-config.yaml if it references black/isort/flake8

Risk: rule divergence on first run

Ruff may flag code that the current flake8 version was silently ignoring. Expect a small number of one-off fixups — run ruff check --fix src tests to auto-resolve the majority.

---

Files to Change

File	Change
packages/parser-core/pyproject.toml	Add [tool.ruff] config; remove black/isort/flake8/pyright from dev deps; bump ruff version
packages/parser-free/pyproject.toml	Add [tool.ruff] config; add ruff + mypy to new [dev] group
.github/workflows/ci.yml	Replace black/isort/flake8 steps with ruff in both lint jobs; add mypy to lint-free
Makefile	Update lint, format, ci-lint targets
.pre-commit-config.yaml	Replace black/isort/flake8 hooks with ruff hook (if file exists)
MEMORY.md	Update "Black Formatting" note to reference ruff format
README.md / CONTRIBUTING.md	Update tooling references and developer setup instructions

---

Test Plan

• Run ruff format --check src tests in both packages — confirm zero formatting diffs (validates Black-compatibility)
• Run ruff check src tests in both packages — fix any new findings, document any intentional suppressions with # noqa: <code>
• Run mypy src --ignore-missing-imports in parser-free — fix any type errors surfaced for the first time
• Run full test suite in both packages (pytest) — confirm all 1420+ tests still pass after any fixups
• Run make ci-all locally — confirm the simulated CI passes end-to-end
• Push to a PR branch — confirm the updated CI lint-core and lint-free jobs go green
• Confirm ci-pass required status check still passes

---

Out of Scope

• Enabling additional Ruff rule sets beyond E/W/F/I (e.g. B, UP, SIM) — follow-on issue once baseline is stable
• Changing mypy strictness settings — already well-configured
• Adding mypy to the private longieirl/bankstatements repo — separate scope