Add dedicated Celery queue for bulk consent imports

[Linker44]

Summary

• Adds BULK_CONSENT_IMPORT_QUEUE_NAME (fidesplus.bulk_consent_import) constant to fides.api.tasks
• Registers the new queue in the worker's all_queues list so it is consumed by default

This queue isolates bulk pre-verified consent record imports from the main fides.privacy_preferences queue, preventing large imports (e.g. 4M historical records) from blocking normal consent operations.

Test plan

• Verify existing tests pass (no behavioral change — just a new queue constant and worker registration)
• Companion fidesplus PR adds the endpoint and tests that use this queue

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Ignored	Preview	Mar 25, 2026 3:15pm
fides-privacy-center	Ignored		Mar 25, 2026 3:15pm

[greptile-apps]

Greptile Summary

This PR introduces a dedicated Celery queue (fidesplus.bulk_consent_import) to isolate bulk pre-verified consent record imports from the main fides.privacy_preferences queue, preventing large imports from starving normal consent operations. The changes are minimal and well-scoped.

• Adds BULK_CONSENT_IMPORT_QUEUE_NAME = "fidesplus.bulk_consent_import" constant to src/fides/api/tasks/__init__.py, following the established naming pattern and including an explanatory inline comment.
• Imports the new constant and appends it to all_queues in start_worker() inside src/fides/api/worker/__init__.py, ensuring the default worker consumes the new queue automatically.
• No behavioral changes to existing queues or tasks — purely additive infrastructure ahead of the companion fidesplus PR that will introduce the actual endpoint and tasks that use this queue.

Confidence Score: 5/5

• This PR is safe to merge; it is a purely additive, low-risk change that follows all established conventions.
• The change adds a single constant and one list entry. It follows all existing patterns (naming convention, alphabetical import ordering, inline comment), introduces no behavioral modifications to existing queues or tasks, and the companion PR with the actual task logic will provide the test coverage. No security, logic, or style issues identified.
• No files require special attention.

Important Files Changed

Filename	Overview
src/fides/api/tasks/init.py	Adds BULK_CONSENT_IMPORT_QUEUE_NAME constant following existing naming conventions with a clear explanatory comment.
src/fides/api/worker/init.py	Imports and registers the new BULK_CONSENT_IMPORT_QUEUE_NAME in all_queues; import is alphabetically ordered and follows existing patterns.

_{Reviews (1): Last reviewed commit: "Add BULK_CONSENT_IMPORT_QUEUE_NAME for b..." | Re-trigger Greptile}

[claude]

Code Review Summary

The change itself is correct and follows the established pattern exactly — constant naming, queue string format, inline comment style, alphabetical import ordering, and placement in all_queues all match existing conventions. The motivation (isolating bulk imports from the live consent queue) is sound.

Critical (Must Fix)

tests/ctl/api/test_worker.py is not updated. The test_start_worker_with_arguments parametrized test has hardcoded expected queue strings that omit fidesplus.bulk_consent_import. Three of the five parametrized cases will fail:

• The "all queues" case (queues=None, exclude_queues=None) — the actual result will include the new queue at the end.
• The "exclude some queues" case where fidesplus.bulk_consent_import is not in the excluded set.
• The "exclude fides,fides.dsr" case — the new queue survives and appears in the result.

The fix is to append fidesplus.bulk_consent_import to the expected queue strings in each affected test case. See the inline comment for the exact corrected string for the all-queues case.

Everything Else

No other issues — the two-file change is clean, minimal, and consistent with the codebase pattern.

[vcruces]

This looks good to me! Quick question, would it be useful to include the queue in get_queue_counts (src/fides/api/util/cache.py), or is it not worth it?

[Linker44]

Done — updated the three affected parametrized test cases in test_start_worker_with_arguments to include fidesplus.bulk_consent_import in the expected queue strings.

[Linker44]

Good call — added it to get_queue_counts so it shows up in the health/workers endpoint.