Skip to content

bug: chicken-and-egg ordering issues when deploying apps with attachments #87

@bnema

Description

@bnema

Summary

Several idempotency and ordering gaps exist in the attachment deployment lifecycle. The most severe are: (1) attachments are started but never waited on before the app container starts, causing race conditions; (2) attachment config changes made via `gordon attachments add` are not propagated to the running container service, requiring a full Gordon restart; (3) the in-memory attachment map is not rebuilt after a Gordon restart, causing orphaned containers after remove operations.

Issue 1: No readiness wait on attachments before app start

File: `internal/usecase/container/service.go:1885-1894`

The deployment sequence in `prepareDeployResources` starts all attachment containers (postgres, redis, etc.) and then immediately proceeds to create and start the app container — without any health or readiness check on the attachments:

```
deployAttachments() ← starts postgres, redis, etc.
GetImageExposedPorts()
loadEnvironment()
setupVolumes()
createStartedContainer() ← app starts immediately, attachments may not be ready
```

Compare with the app container, which has an explicit `waitForReady()` call (`service.go:440`). Attachments have no equivalent. If the app tries to connect to postgres on startup and postgres is still initializing, the app fails.

The `waitForReady()` mechanism already exists — it just needs to be applied to attachments, gated on a configurable health check path or a simple TCP-ready probe.

Issue 2: Attachment config changes require a Gordon restart

File: `internal/app/run.go:1137`

At startup, the container service receives a one-time snapshot of attachment config:

```go
Attachments: svc.configSvc.GetAttachments(),
```

When a user adds an attachment via `gordon attachments add`, the config is updated on disk and in `configSvc`, but `container.Service` is never notified. An `UpdateConfig()` method exists (`service.go:1057`) but is never called after config changes.

The `ConfigReloadHandler` in `internal/usecase/container/events.go:93` handles config reload events and re-deploys containers, but does not call `containerSvc.UpdateConfig()` with the new attachment map.

Result: Adding or removing an attachment takes effect only after restarting Gordon.

Issue 3: In-memory attachment map not rebuilt after Gordon restart

File: `internal/usecase/container/service.go:892-927`

`SyncContainers()` rebuilds the in-memory map of managed containers from Docker labels after a restart, but it only syncs main containers, not attachments:

```go
// s.attachments is never repopulated in SyncContainers()
for _, c := range allContainers {
if d, ok := c.Labels["gordon.domain"]; ok && c.Labels["gordon.managed"] == "true" {
managed[d] = c
}
}
```

After Gordon restarts, `s.attachments[domain]` is empty. When `Remove()` is called for an app container, `removeAttachments()` silently no-ops because it reads from the empty in-memory map. The attachment containers (postgres, redis, etc.) are left running as orphans even though the app container was removed.

The listing path (`getAllAttachments`) reads live from Docker labels and works correctly — only lifecycle operations (remove, restart-with-attachments) are broken.

Issue 4: Secrets and volume errors are silently swallowed during attachment deploy

File: `internal/usecase/container/service.go:1856-1865`

```go
volumes, err := s.setupVolumes(ctx, ...)
if err != nil {
log.Warn()... // non-fatal, continues with empty volumes
}
env, err := s.loadEnvironment(ctx, ...)
if err != nil {
log.Warn()... // non-fatal, continues with empty env
}
```

Missing secrets or volume setup failures are logged as warnings and the deployment continues. An attachment (e.g., a database) missing its credentials env vars will start but be misconfigured, which is harder to debug than an explicit deploy failure.

Affected Files

  • `internal/usecase/container/service.go:365-413` — `prepareDeployResources` (deployment ordering)
  • `internal/usecase/container/service.go:1792-1905` — `deployAttachedService` (no readiness wait)
  • `internal/usecase/container/service.go:892-927` — `SyncContainers` (no attachment rebuild)
  • `internal/usecase/container/service.go:1057` — `UpdateConfig` (exists but never called)
  • `internal/usecase/container/events.go:93-164` — `ConfigReloadHandler` (does not propagate new attachment config)
  • `internal/app/run.go:1137` — one-time config snapshot at boot

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions