Skip to content

Add quorum parameter to WorkerPool and WorkerProxy #144

@conradbzura

Description

@conradbzura

Summary

Add a quorum: int | None = None parameter to WorkerProxy and WorkerPool that makes _start() block until len(workers) >= quorum before returning. This ensures a minimum number of workers are discovered before the proxy considers itself started, whether that happens eagerly via start() or lazily on first dispatch().

Motivation

Currently there is no way to require a minimum number of workers before dispatch begins. _await_workers() only waits for at least one worker. For workloads that require a minimum level of parallelism — such as map-reduce, ensemble inference, or sharded computation — callers need a guarantee that enough workers are present before the first task is routed.

Expected outcome

  • WorkerProxy(discovery=..., quorum=3) makes _start() block until at least 3 workers are discovered
  • If lease is set and quorum > lease, raise ValueError at construction — the quorum can never be satisfied
  • Lazy semantics are preserved: for lazy=True pools, the quorum blocks on first dispatch(); for lazy=False pools, it blocks on start() / __aenter__
  • WorkerPool forwards quorum to the underlying WorkerProxy

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or capability

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions