Getting tons of: "Docker event stream error (aborted); reconnect attempt #1 in 1000ms"

[begunfx]

Not sure why this is happening. AI says it has something to do with the proxy timing out (it appears to timeout every 10 minutes). I have a timeout added, but it's still an issue. How can I fix this?

log:

3/20/2026, 2:14:09 AM
warn
watcher.docker.datavault
Docker event stream error (aborted); reconnect attempt #1 in 1000ms
3/20/2026, 2:14:10 AM
info
watcher.docker.datavault
Listening to docker events
3/20/2026, 2:24:10 AM
warn
watcher.docker.datavault
Docker event stream error (aborted); reconnect attempt #1 in 1000ms
3/20/2026, 2:24:11 AM
info
watcher.docker.datavault
Listening to docker events

Here's my drydock controller stack:

services:
  drydock:
    image: codeswhat/drydock:latest
    container_name: drydock
    restart: unless-stopped
    ports:
      - "3010:3000"
    labels:
      - "dd.watch.digest=true"
    depends_on:
      socket-proxy:
        condition: service_healthy
    volumes:
      - /volume1/docker/drydock:/store
    healthcheck:
      test: curl --fail http://localhost:${DD_SERVER_PORT:-3000}/health || exit 1
      interval: 10s
      timeout: 10s
      retries: 3
      start_period: 10s
    environment:
      # --- CRON SCHEDULE ---
      - DD_WATCHER_DATAVAULT_CRON=15 */6 * * *
      - DD_WATCHER_DATAVAULT_JITTER=900000

      # --- PERMISSIONS ---
      - PUID=1000
      - PGID=1000

      # --- OTHER ---
      - TZ=America/Los_Angeles
      - DD_SESSION_SECRET=<redacted>

      # --- SECURITY SCANNING ---
      - DD_SECURITY_SCANNER=trivy
      - DD_SECURITY_SCAN_CRON=0 2 * * *
      - DD_SECURITY_BLOCK_SEVERITY=NONE
      
      # --- BASIC AUTH ---
      - DD_AUTH_BASIC_ADMIN_USER=admin
      - DD_AUTH_BASIC_ADMIN_HASH=<redacted>
      
      # --- LOCAL DATAVAULT DISCOVERY (Global Watcher) ---
      - DD_WATCHER_DATAVAULT_HOST=drydock-socket-proxy
      - DD_WATCHER_DATAVAULT_PORT=2375
      - DD_WATCHER_DATAVAULT_WATCHBYDEFAULT=true

      # --- LOCAL DATAVAULT EXECUTION (Trigger Connection) ---
      - DD_TRIGGER_DOCKER_DATAVAULT_PRUNE=true

      # --- REMOTE HOST MEDIAVAULT ---
      - DD_AGENT_MEDIAVAULT_HOST=mediavault
      - DD_AGENT_MEDIAVAULT_PORT=3010
      - DD_AGENT_MEDIAVAULT_SECRET=<redacted>

      # --- REMOTE HOST TMVAULT ---
      - DD_AGENT_TMVAULT_HOST=tmvault
      - DD_AGENT_TMVAULT_PORT=3010
      - DD_AGENT_TMVAULT_SECRET=<redacted>

      # --- REMOTE HOST SERVICEVAULT ---
      - DD_AGENT_SERVICEVAULT_HOST=servicevault
      - DD_AGENT_SERVICEVAULT_PORT=3010
      - DD_AGENT_SERVICEVAULT_SECRET=<redacted>

      # --- REMOTE HOST SERVICEVAULT2 ---
      - DD_AGENT_SERVICEVAULT2_HOST=servicevault2
      - DD_AGENT_SERVICEVAULT2_PORT=3010
      - DD_AGENT_SERVICEVAULT2_SECRET=<redacted>
      
      # --- OBSERVABILITY ---
      - DD_METRICS_PROMETHEUS_ENABLE=true

      # --- GMAIL SMTP CONFIG ---
      - DD_TRIGGER_SMTP_GMAIL_HOST=smtp.gmail.com
      - DD_TRIGGER_SMTP_GMAIL_PORT=587
      - DD_TRIGGER_SMTP_GMAIL_USER=<redacted>@gmail.com
      - DD_TRIGGER_SMTP_GMAIL_PASS=<redacted>
      - DD_TRIGGER_SMTP_GMAIL_FROM=<redacted>@gmail.com
      - DD_TRIGGER_SMTP_GMAIL_TO=<redacted>
      - DD_TRIGGER_SMTP_GMAIL_TLS_ENABLED=false
      - DD_TRIGGER_SMTP_GMAIL_MODE=batch
      
      # --- PUSHOVER CONFIG ---
      - DD_TRIGGER_PUSHOVER_MOBILE_USER=<redacted>
      - DD_TRIGGER_PUSHOVER_MOBILE_TOKEN=<redacted>

      # --- DOCKER HUB (PUBLIC/PRIVATE) ---
      - DD_REGISTRY_HUB_PUBLIC_AUTH=<redacted>
      - DD_REGISTRY_HUB_PRIVATE_AUTH=<redacted>

      # --- GITHUB CONTAINER REGISTRY (GHCR) ---
      - DD_REGISTRY_GHCR_PRIVATE_USERNAME=<redacted>
      - DD_REGISTRY_GHCR_PRIVATE_TOKEN=<redacted>

  socket-proxy:
    image: lscr.io/linuxserver/socket-proxy:latest
    container_name: drydock-socket-proxy
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - CONTAINERS=1
      - IMAGES=1
      - EVENTS=1
      - SERVICES=1
      - POST=1
      - DELETE=1
      - NETWORKS=1
      - VOLUMES=1
      - TIMEOUT=300s
      - LOG_LEVEL=info
      - RATE_LIMIT_HTTP=100
      - RATE_LIMIT_INTERVAL=1m
    healthcheck:
      test: wget --spider http://localhost:2375/version || exit 1
      interval: 5s
      timeout: 3s
      retries: 3
      start_period: 5s

One of my agent stacks:

services:
  # 1. The Dedicated Internal Proxy (No ports exposed to host!)
  drydock-socket-proxy:
    image: lscr.io/linuxserver/socket-proxy:latest
    container_name: drydock-socket-proxy
    restart: unless-stopped
    read_only: true
    tmpfs:
      - /run
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - CONTAINERS=1
      - DISTRIBUTION=1
      - IMAGES=1
      - INFO=1
      - EVENTS=1
      - PING=1
      - VERSION=1
      - NETWORKS=1
      - VOLUMES=1
      - TIMEOUT=300s
      # --- EXECUTION ENABLED SAFELY ---
      # This is secure because the container has no external ports mapped
      - POST=1 
      - DELETE=1

  # 2. The Agent
  drydock-agent:
    image: codeswhat/drydock:latest
    container_name: drydock-agent
    restart: unless-stopped
    command: --agent
    depends_on:
      - drydock-socket-proxy
    ports:
      # REQUIRED: The Main Controller connects to this port over your LAN
      - "3010:3000"
    environment:
      # --- CRON SCHEDULE ---
      - DD_WATCHER_MEDIAVAULT_CRON=30 */6 * * *
      - DD_WATCHER_MEDIAVAULT_JITTER=900000
      
      # The shared secret matching the Controller's config
      - DD_AGENT_SECRET=<redacted>
      
      # --- LOCAL WATCHER ---
      - DD_WATCHER_MEDIAVAULT_HOST=drydock-socket-proxy
      - DD_WATCHER_MEDIAVAULT_PORT=2375

      - DD_TRIGGER_DOCKER_MEDIAVAULT_PRUNE=true

[begunfx]

The only workaround that seems to work is to create a tiny heartbeat container to keep the connection from timing out without having to create some sort of custom config for the proxies..

services:
  # Your existing proxy
  socket-proxy:
    image: lscr.io/linuxserver/socket-proxy:latest
    # ... your current config ...

  # Add this tiny service:
  socket-heartbeat:
    image: alpine:latest
    container_name: socket-heartbeat
    command: sh -c "sleep 300"
    restart: always

Open to any other suggestion please.

[s-b-e-n-s-o-n]

This is expected behavior with third-party socket proxies — the proxy's TIMEOUT setting kills idle HTTP connections, and Docker's event stream is a long-lived connection that can go idle between events. Drydock detects the disconnect and reconnects automatically (the "Listening to docker events" line confirms it succeeded). No events are lost because Drydock requests events since the last received timestamp on reconnect.

We've made two changes:

1. Downgraded the log from warn to info (a28cd7ed)

Since the first reconnect is expected behavior and resolves automatically, it no longer shows as a warning. Subsequent reconnect failures (indicating a real problem) still log as warn.

2. Our own proxy is on the roadmap

The root cause is that we can't control timeout behavior on third-party proxies like linuxserver/socket-proxy. When we ship our own purpose-built proxy, event stream connections will be exempt from idle timeouts entirely, eliminating this reconnect cycle.

In the meantime, your heartbeat container approach works, or you can bump TIMEOUT=86400s on the socket proxy to reduce the frequency.

[begunfx]

I would recommend updating the doc to include this timeout as expected behavior and suggested workarounds until the home grown proxy is available

[s-b-e-n-s-o-n]

Good call @begunfx — docs have been updated for the next release with:

• Log level behavior — first reconnect logs at info (expected), subsequent attempts escalate to warn
• Event gap transparency — honest note that container events during the brief reconnect window (~1s) are picked up on the next poll cycle, not guaranteed zero-loss (a future release will wire since timestamps to close this gap)
• Native proxy roadmap — forward reference to the planned Drydock socket proxy that will eliminate proxy-timeout reconnects entirely
• Your heartbeat workaround — included as an explicit option with an example

These will ship with the next release. Thanks for pushing for the docs improvement.

[s-b-e-n-s-o-n]

Docs have been updated — the FAQ now covers the reconnect behavior, proxy timeout workarounds, and what to expect with single vs repeated reconnects:

https://getdrydock.com/docs/faq#docker-event-stream-aborted-reconnects-every-few-minutes

Also in rc.2+, the first reconnect logs at info instead of warn so it does not clutter logs in normal proxy setups. Marking this as answered — if anything else comes up feel free to open an issue.