Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# changes — detect which paths changed (skips heavy jobs on workflow-only PRs)
# lint-core — black, isort, ruff, mypy on packages/parser-core
# lint-free — black, isort, ruff on packages/parser-free
# security — bandit + pip-audit on both packages
# security — bandit + pip-audit + xenon complexity gate
# test-core — pytest with 91% coverage gate (Python matrix), needs lint-core
# test-free — pytest on packages/parser-free, needs lint-free

Expand Down Expand Up @@ -207,6 +207,19 @@ jobs:
- name: pip-audit — dependency vulnerability scan
run: pip-audit -r requirements/base.txt --progress-spinner off -f json -o pip-audit-report.json

- name: Complexity gate (Xenon)
run: |
# table_detector.py: _detect_text_based_table grade E (score 32) — PDF heuristic hotspot,
# pdfplumber word-coordinate coupling blocks safe decomposition without characterisation tests.
# Temporary exclusion — remove once refactored. Tracked: https://github.com/longieirl/bankstatementprocessor/issues/98
# template_detector.py: get_detection_explanation grade D (score 22) — diagnostic method;
# exhaustive path coverage of all scoring outcomes is inherent to its purpose. Stable exemption.
# commands/init.py: init_directories grade D (score 26) — CLI entrypoint orchestration;
# complexity is setup sequencing (mkdir + sample file creation), not extractable decision logic. Stable exemption.
xenon --max-average A --max-modules B --max-absolute C \
--exclude "packages/parser-core/src/bankstatements_core/analysis/table_detector.py,packages/parser-core/src/bankstatements_core/templates/template_detector.py,packages/parser-core/src/bankstatements_core/commands/init.py" \
packages/parser-core/src

- name: Upload security reports
uses: actions/upload-artifact@v7
if: always()
Expand Down
50 changes: 33 additions & 17 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ pip install -e "packages/parser-free[test]"

```bash
# Core tests with coverage
pytest packages/parser-core/tests/ --cov=bankstatements_core --cov-fail-under=91
pytest packages/parser-core/tests/ --cov=bankstatements_core --cov-fail-under=92

# Free CLI smoke test
bankstatements --version
Expand All @@ -63,15 +63,14 @@ bankstatements --init

1. Fork the repository and create a branch
2. Make changes following the code standards below
3. **Write tests** — 91% coverage on `bankstatements-core` is enforced by CI
3. **Write tests** — 92% coverage on `bankstatements-core` is enforced by CI
4. Run quality checks:
```bash
# From packages/parser-core/
black src tests
isort src tests
flake8 src tests --max-line-length=88 --extend-ignore=E203,W503
ruff format src tests
ruff check src tests
mypy src --ignore-missing-imports
pytest tests/ --cov=bankstatements_core --cov-fail-under=91
pytest tests/ --cov=bankstatements_core --cov-fail-under=92
```
5. Submit a PR — CI must pass (lint, boundary check, tests)

Expand All @@ -81,8 +80,8 @@ bankstatements --init

### Formatting

- **Line length:** 88 characters (Black default)
- **Imports:** organised by isort (stdlib → third-party → local)
- **Line length:** 88 characters (ruff default)
- **Imports:** organised by ruff (stdlib → third-party → local)

### Type Hints

Expand Down Expand Up @@ -110,13 +109,13 @@ def process_transaction(amount: Decimal, date: datetime) -> Transaction:

### Coverage Requirement

**91% minimum on `bankstatements-core`** — enforced by CI. PRs that drop below this threshold will not be merged.
**92% minimum on `bankstatements-core`** — enforced by CI. PRs that drop below this threshold will not be merged.

### Running Tests

```bash
# All tests with coverage
pytest packages/parser-core/tests/ --cov=bankstatements_core --cov-fail-under=91 -v
# All tests with coverage (integration tests excluded by default)
pytest packages/parser-core/tests/ --cov=bankstatements_core --cov-fail-under=92 -v

# Specific file or test
pytest packages/parser-core/tests/services/test_my_service.py -v
Expand All @@ -129,12 +128,29 @@ pytest packages/parser-core/tests/ -n auto
pytest packages/parser-free/tests/ -v
```

### Integration Tests

Integration tests run the full processing pipeline against real PDFs on your local machine. They are excluded from the standard test run and never run in CI.

```bash
# First run — create your local snapshot baseline (requires PDFs in input/)
cd packages/parser-core
pytest tests/integration/ -m integration --snapshot-update -v

# Subsequent runs — validate output against your baseline
pytest tests/integration/ -m integration -v
```

The snapshot file (`tests/integration/snapshots/output_snapshot.json`) is gitignored — it is personal to your machine and input files.

### Test Organisation

```
packages/parser-core/tests/
├── test_processor.py
├── test_app.py
├── integration/ # local-only, requires real PDFs
│ └── test_output_snapshot.py
├── services/
├── extraction/
├── domain/
Expand All @@ -158,19 +174,19 @@ Quick summary:

### Before Submitting

- [ ] All tests pass with 91%+ coverage
- [ ] Code is formatted (Black + isort)
- [ ] Linting passes (Flake8, MyPy)
- [ ] All tests pass with 92%+ coverage
- [ ] Code is formatted (`ruff format`)
- [ ] Linting passes (`ruff check`, MyPy)
- [ ] New functionality has tests
- [ ] No regressions in existing tests

### CI Checks

Your PR must pass:

- **lint-core** — Black, isort, Flake8, MyPy on `parser-core`
- **lint-free** — Black, isort, Flake8 on `parser-free`
- **test-core** — pytest with 91%+ coverage on `bankstatements-core`
- **lint-core** — ruff, MyPy on `parser-core`
- **lint-free** — ruff on `parser-free`
- **test-core** — pytest with 92%+ coverage on `bankstatements-core`
- **test-free** — pytest on `parser-free`
- **boundary-check** — CI fails if `parser-free` imports `bankstatements_premium` or `src.licensing`
- **security** — Bandit on both packages
Expand Down
3 changes: 1 addition & 2 deletions requirements/ci.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,4 @@ pip-licenses>=5.0.0
PyYAML>=6.0

# Complexity analysis
radon==6.0.1
mccabe==0.7.0
xenon>=0.9.0