SEC-01: replace safety with pip-audit for dependency vulnerability scanning

[longieirl]

Summary

Replace safety with pip-audit in the CI security job as a hard gate scoped to production runtime dependencies. pip-audit is the PyPA-maintained standard for Python package CVE scanning — no API key required, no paid tier risk, non-zero exit on findings.

Background

The project already had two CVE-driven pin bumps in early 2026:

• pypdf>=6.7.5 (security patches 2026-03-02)
• cryptography>=46.0.5 (security patches 2026-03-02)

These were caught manually. pip-audit would surface these automatically, earlier, and as a CI gate.

Problem with safety

safety==3.7.0 is currently in requirements/ci.txt and runs in the CI security job with || true — meaning it never fails CI under any circumstances. Since Safety v3, full database access requires a paid API key; without one the result set is silently degraded. The combination of || true and a degraded scan means this step is currently security theatre.

What pip-audit provides

Property	Detail
Maintained by	Python Packaging Authority (PyPA)
Database	PyPI Advisory Database + OSV (open, no key)
CVE aliases	Reports CVE IDs, GHSA IDs, PYSEC IDs
Fix versions	Shows lowest safe version per vulnerability
Input modes	--requirement, --local, pyproject.toml
Output	JSON, human-readable
CI exit code	Non-zero on any vulnerability — real gate
Cost	Zero — open source, no API key

Scope Decision

Scan requirements/base.txt only — production runtime dependencies.

Do NOT scan the full installed environment. Dev/test tools (pytest, locust, factory-boy, bandit etc.) carry their own CVE surface that is irrelevant to the production runtime and would generate noise unrelated to the shipped image. Scoping to requirements/base.txt keeps the signal clean and the gate meaningful.

Rollout: Advisory → Hard Gate in a Single PR

The gate must not land as another || true. The rollout is two steps within the same PR (or two consecutive PRs with no gap):

Step 1 — Advisory run (pre-PR, local):

pip-audit -r requirements/base.txt

Run locally before opening the PR. Triage every finding:

• If a fix is available: bump the pin in requirements/base.txt
• If no fix is available: add to the ignore file with a per-CVE justification (see below)

Step 2 — Hard gate (committed in the PR):

- name: pip-audit (production dependencies)
  run: pip-audit -r requirements/base.txt --progress-spinner off

No || true. Non-zero exit blocks the PR.

Ignore File

Any genuinely unfixed CVE must be tracked in a committed ignore file, not suppressed with || true. Each entry requires an inline comment.

Example .pip-audit-ignore (or equivalent config):

# GHSA-xxxx-yyyy-zzzz: affects pdfplumber<=0.11.8, no fix available as of 2026-03-27.
# Review when pdfplumber>=0.12.0 is released.
GHSA-xxxx-yyyy-zzzz

This makes the suppression explicit, reviewable, and time-bounded.

Proposed Changes

1. requirements/ci.txt

-safety==3.7.0
+pip-audit>=2.7.0

Also remove safety from requirements/dev.txt and packages/parser-core/pyproject.toml dev optional dependencies.

2. .github/workflows/ci.yml — security job

-  - name: Safety scan
-    run: safety scan --json > safety-report.json || true
+  - name: pip-audit (production dependencies)
+    run: pip-audit -r requirements/base.txt --progress-spinner off
+
+  - name: pip-audit JSON report
+    if: always()
+    run: pip-audit -r requirements/base.txt -f json -o pip-audit-report.json || true

The JSON report step uses || true so the artifact is always generated even when the gate fails — for triage purposes.

3. Upload artifact

- name: Upload pip-audit report
  uses: actions/upload-artifact@v7
  if: always()
  with:
    name: pip-audit-report-${{ github.run_number }}
    path: pip-audit-report.json
    retention-days: 90

4. Commit ignore file

If any CVEs are found with no available fix during the advisory dry run, commit a .pip-audit-ignore file (or configure via pyproject.toml) with per-CVE justification comments.

What deptry was also evaluated

deptry (unused/missing/transitive dep detection) was considered. Given the small, intentional dependency footprint (7 runtime deps in parser-core, 1 in parser-free), it offers marginal CI value. Recommended as a one-time manual audit rather than a permanent CI gate:

pip install deptry
deptry packages/parser-core/src --src packages/parser-core/src
deptry packages/parser-free/src --src packages/parser-free/src

If the audit finds violations, raise a separate issue.

Acceptance Criteria

• safety removed from requirements/ci.txt, requirements/dev.txt, and packages/parser-core/pyproject.toml
• pip-audit>=2.7.0 added to requirements/ci.txt
• CI security job runs pip-audit -r requirements/base.txt with no || true — hard gate
• CI security job produces a JSON artifact (always, even on failure) for triage
• Advisory dry run performed locally before PR; findings documented in PR description
• Any unfixed CVEs added to a committed ignore file with per-CVE justification comment and review date
• CI security job passes cleanly with no unacknowledged vulnerabilities in requirements/base.txt
• deptry run manually; result documented in PR description