Skip to content

Add WorkerPool constructor overload that accepts DiscoverySubscriberLike to support external-only discovery patterns #106

@conradbzura

Description

@conradbzura

Description

Extend WorkerPool to accept DiscoverySubscriberLike in addition to DiscoveryLike for the discovery parameter. When only a subscriber is provided (no publisher), the pool operates in external mode only — it discovers externally managed workers but cannot spawn or publish its own. If size > 0 is requested with a subscriber-only discovery, the pool raises a clear error explaining that ephemeral workers require a discovery backend with a publisher.

# Full discovery — supports ephemeral, external, and hybrid modes (unchanged)
async with WorkerPool(size=4, discovery=SomeFullDiscovery()):
    ...

# Subscriber-only — external mode only
async with WorkerPool(discovery=some_discovery.subscriber):
    ...

# Subscriber-only + size > 0 — raises ValueError at construction
async with WorkerPool(size=4, discovery=some_discovery.subscriber):
    ...  # ValueError: ephemeral workers require a discovery backend with a publisher

Motivation

Some discovery backends are inherently observe-only. For example, a Docker discovery subscriber watching Swarm-managed containers can detect workers as they appear and disappear, but has no meaningful publish step — the Swarm scheduler owns container lifecycle and labeling. Forcing these backends to implement a no-op publisher to satisfy the DiscoveryLike protocol is unnecessary ceremony. Accepting a bare DiscoverySubscriberLike makes external-only backends a first-class concept and lets the pool validate compatibility at construction time rather than failing silently at runtime.

This change is fully backwards compatible — existing code passes DiscoveryLike instances which satisfy both paths.

Expected outcome

  • The discovery parameter on WorkerPool accepts DiscoveryLike | DiscoverySubscriberLike.
  • When a DiscoverySubscriberLike without a publisher is provided, the pool operates in external mode: it subscribes to discover existing workers but does not publish.
  • When size > 0 is combined with a subscriber-only discovery, the pool raises ValueError with a clear message at construction time.
  • No changes to existing DiscoveryLike implementations or callers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or capability

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions