Default nQueueLimitWorkerCoresMin, nQueueLimitWorkerMemoryMin and nQueueLimitJobCoresMin; update worker diagmessage errorrcode map#301
Conversation
…moryMin based on nQueueLimitWorkerMin
There was a problem hiding this comment.
Pull request overview
This pull request implements two main improvements to the PanDA Harvester system:
-
Worker limit defaulting logic: When ratio-based worker limiting is enabled, the minimum cores and memory limits can now be automatically derived from
nQueueLimitWorkerMinif not explicitly configured. This ensures at least 1 core per worker and 1000 MB per worker as baseline minimums. -
Worker error pattern updates: Adds new error code mappings for HTTP response errors, site-specific hold reasons (runtime exceeded, memory exceeded, session directory creation failure), and a catch-all pattern for PeriodicRemove events. Also updates an existing pattern to be more flexible.
Changes:
- Enhanced worker limit calculation to default minimum cores and memory based on minimum worker count
- Added 5 new error code patterns for better error classification
- Modernized SQL debug logging to use f-strings
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pandaharvester/harvestercore/worker_errors.py | Added new error patterns (3689, 5573-5575, 8979) and updated pattern 8970 for more flexible matching |
| pandaharvester/harvestercore/db_proxy.py | Implemented defaulting logic for min cores/memory limits based on min worker count; refactored SQL logging to f-strings |
| pandaharvester/commit_timestamp.py | Updated commit timestamp to reflect deployment time |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
No description provided.