Archival

[lukasjuhrich]

Prepares #490

• Discuss whether we want to do this yearly or regularly && adapt code accordingly
• Test archival selection
  • test select_user_and_last_mem() select (used as CTE)
  • test cutoff filtering
  • test cutoff calculation from current date
  • test do-not-archive filtering
• Implement archive_users (cleanup)
• Add a web endpoint and button archiving all current users
• Add a celerybeat task executing the archival

(update 2026-02-01: in the interest of consolidating changes and moving this forward, we will tackle the follow-up items in separate PRs)

[lukasjuhrich]

This PR, as of now, does some base work that can directly be merged into develop.
All the rest can be coordinated in one or multiple different PRs (see the extended tasklist of #67).
Waiting for the tests to finish.

[Copilot]

Pull request overview

This PR implements data scrubbing functionality to comply with GDPR retention policies, enabling automated deletion of user data after membership ends. The implementation focuses on email address scrubbing with support for batch operations and comprehensive audit logging.

• Adds database model and migration for ScrubLog table to track all scrubbing operations
• Implements email scrubbing logic with user filtering based on membership end dates and archival criteria
• Creates test suite for archival user selection and mail scrubbing operations
• Updates dependencies (alembic, psycopg2-binary, pytest) and development tooling

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
web/templates/user/archivable_users.html	Adds UI section for displaying scrubbable email addresses with action button
web/blueprints/user/init.py	Implements /scrub/mails endpoint and updates archivable users view with mail scrubbing data
tests/lib/user/test_deletion.py	Tests for archivable member selection logic including cutoff calculation and filtering
tests/lib/scrubbing/test_mail_scrubbing.py	Tests for mail scrubbing operations (single and bulk)
tests/lib/scrubbing/conftest.py	Fixtures for scrubbing tests including archivable and non-archivable users
tests/factories/user.py	Adds traits for user creation log entries and tasks
tests/factories/task.py	Updates task factory with fuzzy type selection and self-created trait
tests/factories/log.py	Adds created_at field to log entry factory
pyproject.toml	Updates dependency versions and adds pytest-freezegun
pycroft/model/scrubbing.py	New model for audit logging of scrubbing operations
pycroft/model/alembic/versions/2025_11_02_2036-82f43cfa0f98_add_scrublog.py	Migration to create scrub_log table with indexes
pycroft/model/_all.py	Imports scrubbing model
pycroft/lib/user_deletion.py	Core logic for identifying and scrubbing archivable user data
pycroft/lib/membership.py	Extracts reusable query for selecting users with last membership
pycroft/lib/logging.py	Type annotation update
pycroft/helpers/i18n/serde.py	Moves identity function to shared helpers module
pycroft/helpers/i18n/message.py	Adds type annotations
pycroft/helpers/i18n/deferred.py	Adds type annotation to deferred_gettext
pycroft/helpers/functional.py	Adds identity helper function
docker/base.Dockerfile	Updates uv version
.pre-commit-config.yaml	Removes darker check arguments

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.