feat: Control maximum concurrent redis requests to avoid pool exhaustion#112
Merged
dolfim-ibm merged 3 commits intomainfrom Mar 25, 2026
Merged
feat: Control maximum concurrent redis requests to avoid pool exhaustion#112dolfim-ibm merged 3 commits intomainfrom
dolfim-ibm merged 3 commits intomainfrom
Conversation
…ion (#6) * add initial ray_fair orchestrator Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * implementation with ray serve Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * fix serialization Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * more serialization fixes Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * cannot msgpack the DocumentStream Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * hardening notifier Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * cleanup raydata param and add log level Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * cleanup params and implement object store memory Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * add mtls Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * more logging Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * more logging Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * launch all tasks Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * rename params Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * update docs Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * fix creation of redis pools Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * fix: Watchdog: update the RQ job statusto FAILED and remove it from StartedJobRegistry (#107) * fix: Watchdog: update the RQ job statusto FAILED and remove it from StartedJobRegistry Signed-off-by: Pawel Rein <pawel.rein@prezi.com> * fix formatter/linter Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> --------- Signed-off-by: Pawel Rein <pawel.rein@prezi.com> Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> Co-authored-by: Michele Dolfi <dol@zurich.ibm.com> * add metadata for orchestrator Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * fixes Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * add dispatch state Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * rename workers to actors Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * rename fair_ray to ray Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * more rename Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * fix dispatch vs running Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * add redis manager to the actors Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * fix running metrics Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * fix setting rtunning Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * actor cleanup Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * feat: Expose classification filters for picture description (#105) Preserve legacy picture description filters Signed-off-by: drk <drukpa1455@gmail.com> * feat: add on_result_fetched() no-op lifecycle hook to BaseOrchestrator * feat: add consumed_ttl and on_result_fetched() to RQOrchestrator Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add consumed_ttl and on_result_fetched() to LocalOrchestrator Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add expire_result() to RedisStateManager This method sets a TTL on an existing result key in Redis, enabling crash-safe single-use deletion of results after they are fetched. Implements test-driven development with unit test verification. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add consumed_ttl and on_result_fetched() to RayOrchestrator Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: add connect() guard to expire_result matching peer methods All 20+ other methods in RedisStateManager check `if not self.redis` before using the client. expire_result was missing this guard and would raise RuntimeError if called before connection establishment. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * ensure no asyncio.task can be GCed early Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * apply re-formatting Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * add ray actor logging to jobkit Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * fix: run RQ Job.fetch/get_status/get_position in thread pool to avoid blocking the event loop Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * Ensure control over max ongoing requests per ray replica Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * refactor: rename consumed_ttl back to result_removal_delay Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * upgrade uv.lock Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * move Redis gating and RQ durable status into jobkit Signed-off-by: Christoph Auer <cau@zurich.ibm.com> --------- Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> Signed-off-by: Pawel Rein <pawel.rein@prezi.com> Signed-off-by: drk <drukpa1455@gmail.com> Signed-off-by: Christoph Auer <cau@zurich.ibm.com> Co-authored-by: Michele Dolfi <dol@zurich.ibm.com> Co-authored-by: Paweł Rein <pawel.rein@prezi.com> Co-authored-by: drk <136856552+drukpa1455@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
cau-git
previously approved these changes
Mar 24, 2026
Contributor
|
✅ DCO Check Passed Thanks @dolfim-ibm, all your commits are properly signed off. 🎉 |
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
dolfim-ibm
changed the title
cau-git
approved these changes
Mar 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.