Deploy Mosquitto MQTT Broker via bjw-s-labs `app-template` (no chart hoops, fix perms properly)

[SRF-Audio]

Goal

Deploy an internal Mosquitto broker into namespace infra-mqtt using the bjw-s-labs app-template Helm chart, with:

• TCP MQTT on port 1883
• Auth via existing Secret mosquitto-auth (generated by 1Password operator)
• Config via ConfigMap mosquitto-config
• No permission warnings (Mosquitto requires tight perms; we copy files into a writable volume and set owner/perms)
• Persistent data on storageClass nfs-synology-retain (Retain)
• ValuesObject contains only intentional overrides (no dumping defaults)

References

• App-template docs: https://bjw-s-labs.github.io/helm-charts/docs/app-template/ ([BJW S Labs]1)

• Common library storage types:

  • configMap persistence type ([BJW S Labs]2)
  • secret persistence type ([BJW S Labs]3)
  • emptyDir persistence type ([BJW S Labs]4)

• Mosquitto container UID/GID is commonly 1883 (we will use this for chown/runAs). ([Cedalo]5)

---

Tasks

1) Create / update ArgoCD Application for Mosquitto Helm release

File: argocd/apps/platform/mqtt-broker.yml (or your existing location for infra apps)

Create/update the Application to use:

• repoURL: https://bjw-s-labs.github.io/helm-charts
• chart: app-template
• targetRevision: 4.5.0 (matches current docs example schema line) ([BJW S Labs]1)
• namespace infra-mqtt
• sync wave "20"

IMPORTANT: valuesObject must include ONLY the overrides below.

Use this exact Application manifest (adjust only file path/name if needed):

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: mqtt-broker
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: "20"
spec:
  project: coachlight-k3s-infra
  source:
    repoURL: https://bjw-s-labs.github.io/helm-charts
    chart: app-template
    targetRevision: 4.5.0
    helm:
      valuesObject:
        controllers:
          main:
            strategy: Recreate
            initContainers:
              init-config:
                image:
                  repository: eclipse-mosquitto
                  tag: 2.0.22
                command:
                  - /bin/sh
                  - -ec
                args:
                  - |
                    mkdir -p /config
                    cp -f /config-src/mosquitto.conf /config/mosquitto.conf
                    cp -f /config-src/acl.conf /config/acl.conf
                    cp -f /secret-src/passwords.conf /config/passwords.conf
                    chown -R 1883:1883 /config
                    chmod 0700 /config/passwords.conf /config/acl.conf
            containers:
              main:
                image:
                  repository: eclipse-mosquitto
                  tag: 2.0.22
                args:
                  - -c
                  - /config/mosquitto.conf
                resources:
                  requests:
                    cpu: 10m
                    memory: 32Mi
                  limits:
                    cpu: 100m
                    memory: 128Mi
            pod:
              securityContext:
                runAsUser: 1883
                runAsGroup: 1883
                fsGroup: 1883

        service:
          main:
            controller: main
            ports:
              mqtt:
                port: 1883

        persistence:
          data:
            enabled: true
            type: persistentVolumeClaim
            storageClass: nfs-synology-retain
            size: 1Gi
            globalMounts:
              - path: /mosquitto/data
          config-src:
            enabled: true
            type: configMap
            name: mosquitto-config
            globalMounts:
              - path: /config-src
                readOnly: true
          secret-src:
            enabled: true
            type: secret
            name: mosquitto-auth
            globalMounts:
              - path: /secret-src
                readOnly: true
          config:
            enabled: true
            type: emptyDir
            globalMounts:
              - path: /config
  destination:
    server: https://kubernetes.default.svc
    namespace: infra-mqtt
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Notes (for Copilot):

• We intentionally use initContainers to copy files because ConfigMap/Secret mounts are read-only and don’t allow the ownership/perms Mosquitto demands (your current warnings). The copy into emptyDir + chmod/chown fixes that permanently.
• We set pod securityContext to run as UID/GID 1883 to match the Mosquitto image conventions. ([Stack Overflow]6)
• Do not add any other chart values (ingress, probes, extra annotations, etc.) unless explicitly required.

---

2) Ensure config is provided as a ConfigMap in the “secrets/resources” repo path

You already have a separate Argo app pointing at k8s/infra-mqtt with sync wave "10".

Add a ConfigMap here:

File: k8s/infra-mqtt/mosquitto-config.yml

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mosquitto-config
  namespace: infra-mqtt
data:
  mosquitto.conf: |
    listener 1883
    allow_anonymous false
    persistence true
    persistence_location /mosquitto/data

    password_file /config/passwords.conf
    acl_file /config/acl.conf
  acl.conf: |
    user admin
    topic readwrite #

Important: keep ACL simple. If you later want per-client topics, use %u / %c patterns and tighten up, but start with “admin has full access”.

---

3) Ensure the Secret mosquitto-auth exists and contains passwords.conf

This should already be generated by 1Password operator.

Requirement:

• Secret name: mosquitto-auth
• Namespace: infra-mqtt
• Key: passwords.conf
• Value: the full username:hash line from mosquitto_passwd

Example content:

admin:$7$101$...$...

---

4) Validation steps

After Argo sync:

1. Confirm pod is running and logs have no warnings about /config/passwords.conf or /config/acl.conf permissions.

2. Exec into pod and check:

   • ls -l /config/passwords.conf /config/acl.conf shows owner 1883 and perms -rwx------ (0700).

3. Test MQTT auth from inside cluster:

   • mosquitto_sub -h mqtt-broker.infra-mqtt.svc.cluster.local -p 1883 -u admin -P '<password>' -t test -d

---

Why this fixes your current log spam

Your log is complaining because those files are being mounted with permissive ownership/perms (typical for ConfigMap/Secret mounts). Mosquitto is warning now and (per the message) will refuse to load them in future versions. The initContainer copy + chmod/chown sidesteps that limitation cleanly.

---

If you want, paste the current mosquitto.conf, your mosquitto-auth Secret key names, and whether you’re using 1Password operator CRDs or a raw Secret — and I’ll adjust the issue to match your exact repo layout (so Copilot doesn’t “creatively” rename things).