diff --git a/CLAUDE.md b/CLAUDE.md index 3ae8566..90fd268 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -112,9 +112,9 @@ See @docs/firecrawl-setup.md See @docs/phases.md for full roadmap. -**Completed**: Setup, Core Models, Auth, Q&A, Spaces, UI, LDAP SSO, Search, Articles, Bookmarks, RAG/Chunking +**Completed**: Setup, Core Models, Auth, Q&A, Spaces, UI, LDAP SSO, Search, Articles, Bookmarks, RAG/Chunking, Helm Chart **In Progress**: Email digests, Q&A Wizard enhancements -**Not Started**: REST API, MCP Server, Helm Chart, Social SSO +**Not Started**: REST API, MCP Server, Social SSO --- diff --git a/README.md b/README.md index aa97281..a18d966 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,7 @@ We require 100% test coverage for all code. | `make lint-fix` | Auto-fix lint issues | | `make security` | Run security scans | -### Helm (Coming Soon) +### Helm | Command | Description | |---------|-------------| @@ -136,15 +136,32 @@ make ci # Run full CI pipeline (lint, security, test, helm) docker-compose up -d ``` -### Kubernetes (Helm) - Coming Soon +### Kubernetes (Helm) ```bash +# Install with internal PostgreSQL and Valkey helm install brimming helm/brimming \ - -f values.yaml \ + --set rails.secretKeyBase="$(openssl rand -hex 64)" \ + --set postgresql.auth.password="$(openssl rand -hex 32)" \ + -n brimming \ + --create-namespace + +# Or use custom values file +helm install brimming helm/brimming \ + -f my-values.yaml \ -n brimming \ --create-namespace ``` +The Helm chart supports: +- Internal or external PostgreSQL (with pgvector) +- Internal Valkey subchart or external Redis +- Horizontal Pod Autoscaling +- Ingress with TLS +- Pod Disruption Budget for high availability + +See `helm/brimming/values.yaml` for all configuration options. + ## Configuration ### Environment Variables diff --git a/docs/phases.md b/docs/phases.md index 3b9bb00..e558354 100644 --- a/docs/phases.md +++ b/docs/phases.md @@ -53,6 +53,22 @@ Content chunking, chunk embeddings, RAG query pipeline, citation support, prompt - [ ] Import from external FAQ sources - [ ] Special FAQ styling/badge in UI +### Phase 18: Helm Chart Foundation `[x]` +- [x] Chart structure with Chart.yaml, values.yaml, templates +- [x] App deployment with replicas, HPA, health probes, PDB +- [x] Sidekiq worker deployment with memory optimization +- [x] Database migration job (Helm hook, runs before app) +- [x] Seed job (creates single admin user on fresh install) +- [x] Internal PostgreSQL StatefulSet with pgvector/pg_trgm +- [x] External PostgreSQL support +- [x] Valkey subchart integration +- [x] External Valkey/Redis support +- [x] ConfigMaps and Secrets with checksum annotations +- [x] Ingress configuration with TLS support +- [x] Service Account and RBAC +- [x] helm-unittest test suites +- [~] Firecrawl deployment (templates exist but disabled - requires RabbitMQ) + ## Not Started ### Phase 16: REST API & Swagger @@ -61,8 +77,5 @@ API namespace with versioning, token auth, Swagger/OpenAPI docs. ### Phase 17: MCP Server Brimming as knowledge base backend for AI assistants. Tools: `retrieve()`, `ask()`, `list_spaces()`. -### Phase 18: Helm Chart Foundation -Kubernetes deployment with helm-unittest, PostgreSQL/Valkey subcharts. - ### Phase 19: SSO - Social Providers Google, Facebook, LinkedIn, GitHub, GitLab via OmniAuth. diff --git a/helm/brimming/.gitignore b/helm/brimming/.gitignore new file mode 100644 index 0000000..0f81ad9 --- /dev/null +++ b/helm/brimming/.gitignore @@ -0,0 +1,2 @@ +charts/*.tgz +values.yaml.testing diff --git a/helm/brimming/Chart.lock b/helm/brimming/Chart.lock new file mode 100644 index 0000000..b71dfef --- /dev/null +++ b/helm/brimming/Chart.lock @@ -0,0 +1,6 @@ +dependencies: +- name: valkey + repository: https://valkey.io/valkey-helm/ + version: 0.9.2 +digest: sha256:65961c380608e26ca8da18a3b386a7a59bf7d98ad2152fe41f3a99328bf58ff5 +generated: "2025-12-19T18:07:06.519743-05:00" diff --git a/helm/brimming/Chart.yaml b/helm/brimming/Chart.yaml new file mode 100644 index 0000000..9796042 --- /dev/null +++ b/helm/brimming/Chart.yaml @@ -0,0 +1,22 @@ +apiVersion: v2 +name: brimming +description: A Stack Overflow-style Q&A platform with AI-powered search +type: application +version: 0.1.0 +appVersion: "0.1.0" +home: https://github.com/tightline/brimming +maintainers: + - name: Tight Line LLC + url: https://www.tightlinesoftware.com +keywords: + - knowledge-base + - postgresql + - pgvector + - questions-and-answers + - rails + +dependencies: + - name: valkey + version: "~0.9.0" + repository: "https://valkey.io/valkey-helm/" + condition: valkey.enabled diff --git a/helm/brimming/templates/NOTES.txt b/helm/brimming/templates/NOTES.txt new file mode 100644 index 0000000..a221656 --- /dev/null +++ b/helm/brimming/templates/NOTES.txt @@ -0,0 +1,51 @@ +Brimming has been installed! + +{{- if .Values.ingress.enabled }} +Application URL: +{{- range $host := .Values.ingress.hosts }} + http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ (first $host.paths).path }} +{{- end }} +{{- else if contains "NodePort" .Values.app.service.type }} +Get the application URL by running these commands: + export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "brimming.fullname" . }}) + export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}") + echo http://$NODE_IP:$NODE_PORT +{{- else if contains "LoadBalancer" .Values.app.service.type }} +Get the application URL by running these commands: + NOTE: It may take a few minutes for the LoadBalancer IP to be available. + kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "brimming.fullname" . }} +{{- else if contains "ClusterIP" .Values.app.service.type }} +Get the application URL by running these commands: + export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "brimming.name" . }},app.kubernetes.io/instance={{ .Release.Name }},app.kubernetes.io/component=app" -o jsonpath="{.items[0].metadata.name}") + export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}") + echo "Visit http://127.0.0.1:8080 to use your application" + kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT +{{- end }} + +Components deployed: + - App (Rails web server): {{ .Values.app.replicaCount }} replica(s) + - Worker (Sidekiq): {{ .Values.worker.replicaCount }} replica(s) +{{- if .Values.postgresql.enabled }} + - PostgreSQL: Internal (pgvector/pgvector:{{ .Values.postgresql.image.tag }}) +{{- else }} + - PostgreSQL: External ({{ .Values.externalPostgresql.host }}:{{ .Values.externalPostgresql.port }}) +{{- end }} +{{- if .Values.valkey.enabled }} + - Valkey: Internal +{{- else }} + - Valkey: External ({{ .Values.externalValkey.host }}:{{ .Values.externalValkey.port }}) +{{- end }} +{{- if .Values.firecrawl.enabled }} + - Firecrawl: Enabled ({{ .Values.firecrawl.replicaCount }} replica(s)) +{{- end }} + +{{- if .Values.postgresql.enabled }} + +PostgreSQL connection: + Host: {{ include "brimming.postgresql.host" . }} + Port: 5432 + Database: {{ .Values.postgresql.auth.database }} + Username: {{ .Values.postgresql.auth.username }} +{{- end }} + +For more information, visit: https://github.com/tightline/brimming diff --git a/helm/brimming/templates/_helpers.tpl b/helm/brimming/templates/_helpers.tpl new file mode 100644 index 0000000..7501f98 --- /dev/null +++ b/helm/brimming/templates/_helpers.tpl @@ -0,0 +1,245 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "brimming.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +*/}} +{{- define "brimming.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "brimming.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "brimming.labels" -}} +helm.sh/chart: {{ include "brimming.chart" . }} +{{ include "brimming.selectorLabels" . }} +app.kubernetes.io/version: {{ .Values.image.tag | default .Chart.AppVersion | quote }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "brimming.selectorLabels" -}} +app.kubernetes.io/name: {{ include "brimming.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +App selector labels +*/}} +{{- define "brimming.app.selectorLabels" -}} +{{ include "brimming.selectorLabels" . }} +app.kubernetes.io/component: app +{{- end }} + +{{/* +Worker selector labels +*/}} +{{- define "brimming.worker.selectorLabels" -}} +{{ include "brimming.selectorLabels" . }} +app.kubernetes.io/component: worker +{{- end }} + +{{/* +Service account name +*/}} +{{- define "brimming.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "brimming.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} + +{{/* +Image name with tag +*/}} +{{- define "brimming.image" -}} +{{- $registry := .Values.global.imageRegistry | default "" }} +{{- $repository := .Values.image.repository }} +{{- $tag := .Values.image.tag | default .Chart.AppVersion }} +{{- if $registry }} +{{- printf "%s/%s:%s" $registry $repository $tag }} +{{- else }} +{{- printf "%s:%s" $repository $tag }} +{{- end }} +{{- end }} + +{{/* ========================================================================== + PostgreSQL Helpers + ========================================================================== */}} + +{{/* +PostgreSQL host - internal service or external +*/}} +{{- define "brimming.postgresql.host" -}} +{{- if .Values.postgresql.enabled }} +{{- printf "%s-postgresql" (include "brimming.fullname" .) }} +{{- else }} +{{- required "externalPostgresql.host is required when postgresql.enabled=false" .Values.externalPostgresql.host }} +{{- end }} +{{- end }} + +{{/* +PostgreSQL port +*/}} +{{- define "brimming.postgresql.port" -}} +{{- if .Values.postgresql.enabled }} +{{- 5432 }} +{{- else }} +{{- .Values.externalPostgresql.port | default 5432 }} +{{- end }} +{{- end }} + +{{/* +PostgreSQL database +*/}} +{{- define "brimming.postgresql.database" -}} +{{- if .Values.postgresql.enabled }} +{{- .Values.postgresql.auth.database | default "brimming" }} +{{- else }} +{{- .Values.externalPostgresql.database | default "brimming" }} +{{- end }} +{{- end }} + +{{/* +PostgreSQL username +*/}} +{{- define "brimming.postgresql.username" -}} +{{- if .Values.postgresql.enabled }} +{{- .Values.postgresql.auth.username | default "brimming" }} +{{- else }} +{{- .Values.externalPostgresql.username | default "brimming" }} +{{- end }} +{{- end }} + +{{/* +PostgreSQL secret name (for password) +*/}} +{{- define "brimming.postgresql.secretName" -}} +{{- if .Values.postgresql.enabled }} +{{- if .Values.postgresql.auth.existingSecret }} +{{- .Values.postgresql.auth.existingSecret }} +{{- else }} +{{- printf "%s-postgresql" (include "brimming.fullname" .) }} +{{- end }} +{{- else }} +{{- if .Values.externalPostgresql.existingSecret }} +{{- .Values.externalPostgresql.existingSecret }} +{{- else }} +{{- printf "%s-postgresql-external" (include "brimming.fullname" .) }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +PostgreSQL secret key +*/}} +{{- define "brimming.postgresql.secretKey" -}} +{{- if .Values.postgresql.enabled }} +{{- .Values.postgresql.auth.existingSecretKey | default "password" }} +{{- else }} +{{- .Values.externalPostgresql.existingSecretKey | default "password" }} +{{- end }} +{{- end }} + +{{/* ========================================================================== + Valkey/Redis Helpers + ========================================================================== */}} + +{{/* +Valkey host - internal subchart or external +*/}} +{{- define "brimming.valkey.host" -}} +{{- if .Values.valkey.enabled }} +{{- printf "%s-valkey" (include "brimming.fullname" .) }} +{{- else }} +{{- required "externalValkey.host is required when valkey.enabled=false" .Values.externalValkey.host }} +{{- end }} +{{- end }} + +{{/* +Valkey port +*/}} +{{- define "brimming.valkey.port" -}} +{{- if .Values.valkey.enabled }} +{{- 6379 }} +{{- else }} +{{- .Values.externalValkey.port | default 6379 }} +{{- end }} +{{- end }} + +{{/* +Valkey database (for Rails - database 0) +*/}} +{{- define "brimming.valkey.database" -}} +{{- if .Values.valkey.enabled }} +{{- 0 }} +{{- else }} +{{- .Values.externalValkey.database | default 0 }} +{{- end }} +{{- end }} + +{{/* +Redis URL for Rails (REDIS_URL env var) +*/}} +{{- define "brimming.redisUrl" -}} +{{- $host := include "brimming.valkey.host" . }} +{{- $port := include "brimming.valkey.port" . }} +{{- $db := include "brimming.valkey.database" . }} +{{- printf "redis://%s:%s/%s" $host (toString $port) (toString $db) }} +{{- end }} + +{{/* +Redis URL for Firecrawl (uses database 1) +*/}} +{{- define "brimming.firecrawl.redisUrl" -}} +{{- $host := include "brimming.valkey.host" . }} +{{- $port := include "brimming.valkey.port" . }} +{{- $db := .Values.firecrawl.valkeyDatabase | default 1 }} +{{- printf "redis://%s:%s/%s" $host (toString $port) (toString $db) }} +{{- end }} + +{{/* ========================================================================== + Rails Helpers + ========================================================================== */}} + +{{/* +Rails secret name +*/}} +{{- define "brimming.rails.secretName" -}} +{{- if .Values.rails.existingSecret }} +{{- .Values.rails.existingSecret }} +{{- else }} +{{- printf "%s-rails" (include "brimming.fullname" .) }} +{{- end }} +{{- end }} + +{{/* +Rails secret key for SECRET_KEY_BASE +*/}} +{{- define "brimming.rails.secretKey" -}} +{{- .Values.rails.existingSecretKey | default "secret-key-base" }} +{{- end }} diff --git a/helm/brimming/templates/configmap.yaml b/helm/brimming/templates/configmap.yaml new file mode 100644 index 0000000..7dc4e61 --- /dev/null +++ b/helm/brimming/templates/configmap.yaml @@ -0,0 +1,30 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "brimming.fullname" . }} + labels: + {{- include "brimming.labels" . | nindent 4 }} +data: + DB_HOST: {{ include "brimming.postgresql.host" . | quote }} + DB_PORT: {{ include "brimming.postgresql.port" . | quote }} + DB_USER: {{ include "brimming.postgresql.username" . | quote }} + DB_NAME: {{ include "brimming.postgresql.database" . | quote }} + REDIS_URL: {{ include "brimming.redisUrl" . | quote }} + BRIMMING_BASE_URL: {{ .Values.rails.baseUrl | quote }} + {{- range $key, $value := .Values.rails.env }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- if .Values.smtp.enabled }} + SMTP_HOST: {{ .Values.smtp.host | quote }} + SMTP_PORT: {{ .Values.smtp.port | quote }} + {{- if .Values.smtp.tls }} + SMTP_TLS: {{ .Values.smtp.tls | quote }} + {{- end }} + {{- if .Values.smtp.auth }} + SMTP_AUTH: {{ .Values.smtp.auth | quote }} + {{- end }} + {{- if .Values.smtp.domain }} + SMTP_DOMAIN: {{ .Values.smtp.domain | quote }} + {{- end }} + MAILER_FROM: {{ .Values.smtp.from | quote }} + {{- end }} diff --git a/helm/brimming/templates/deployment-app.yaml b/helm/brimming/templates/deployment-app.yaml new file mode 100644 index 0000000..10363f9 --- /dev/null +++ b/helm/brimming/templates/deployment-app.yaml @@ -0,0 +1,135 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "brimming.fullname" . }}-app + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: app +spec: + {{- if not .Values.app.autoscaling.enabled }} + replicas: {{ .Values.app.replicaCount }} + {{- end }} + selector: + matchLabels: + {{- include "brimming.app.selectorLabels" . | nindent 6 }} + template: + metadata: + annotations: + checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} + checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }} + {{- with .Values.app.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + labels: + {{- include "brimming.app.selectorLabels" . | nindent 8 }} + {{- with .Values.app.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} + spec: + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + serviceAccountName: {{ include "brimming.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.app.podSecurityContext | nindent 8 }} + initContainers: + - name: fix-permissions + image: {{ include "brimming.image" . }} + command: ['sh', '-c', 'chmod 1777 /tmp && chmod 1777 /rails/tmp'] + securityContext: + runAsUser: 0 + runAsNonRoot: false + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + containers: + - name: app + securityContext: + {{- toYaml .Values.app.containerSecurityContext | nindent 12 }} + image: {{ include "brimming.image" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + # Bypass Thruster and run Rails directly (k8s has ingress for SSL/caching) + command: ["./bin/rails"] + args: ["server", "-p", "3000", "-b", "0.0.0.0"] + ports: + - name: http + containerPort: 3000 + protocol: TCP + envFrom: + - configMapRef: + name: {{ include "brimming.fullname" . }} + env: + - name: SECRET_KEY_BASE + valueFrom: + secretKeyRef: + name: {{ include "brimming.rails.secretName" . }} + key: {{ include "brimming.rails.secretKey" . }} + - name: DB_PASS + valueFrom: + secretKeyRef: + name: {{ include "brimming.postgresql.secretName" . }} + key: {{ include "brimming.postgresql.secretKey" . }} + {{- if or .Values.encryption.primaryKey .Values.encryption.existingSecret }} + - name: ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.primaryKey | default "primary-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.deterministicKey | default "deterministic-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.keyDerivationSalt | default "key-derivation-salt" }} + {{- end }} + {{- if and .Values.smtp.enabled .Values.smtp.user }} + - name: SMTP_USER + valueFrom: + secretKeyRef: + name: {{ .Values.smtp.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.smtp.existingSecretKeys.user | default "smtp-user" }} + {{- end }} + {{- if and .Values.smtp.enabled .Values.smtp.password }} + - name: SMTP_PASS + valueFrom: + secretKeyRef: + name: {{ .Values.smtp.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.smtp.existingSecretKeys.password | default "smtp-password" }} + {{- end }} + livenessProbe: + {{- toYaml .Values.app.livenessProbe | nindent 12 }} + readinessProbe: + {{- toYaml .Values.app.readinessProbe | nindent 12 }} + {{- with .Values.app.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + volumes: + - name: tmp + emptyDir: {} + - name: rails-tmp + emptyDir: {} + {{- with .Values.app.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.app.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.app.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} diff --git a/helm/brimming/templates/deployment-worker.yaml b/helm/brimming/templates/deployment-worker.yaml new file mode 100644 index 0000000..9a39806 --- /dev/null +++ b/helm/brimming/templates/deployment-worker.yaml @@ -0,0 +1,109 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "brimming.fullname" . }}-worker + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: worker +spec: + replicas: {{ .Values.worker.replicaCount }} + selector: + matchLabels: + {{- include "brimming.worker.selectorLabels" . | nindent 6 }} + template: + metadata: + annotations: + checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} + checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }} + {{- with .Values.worker.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + labels: + {{- include "brimming.worker.selectorLabels" . | nindent 8 }} + spec: + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + serviceAccountName: {{ include "brimming.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.app.podSecurityContext | nindent 8 }} + initContainers: + - name: fix-permissions + image: {{ include "brimming.image" . }} + command: ['sh', '-c', 'chmod 1777 /tmp && chmod 1777 /rails/tmp'] + securityContext: + runAsUser: 0 + runAsNonRoot: false + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + containers: + - name: worker + securityContext: + {{- toYaml .Values.app.containerSecurityContext | nindent 12 }} + image: {{ include "brimming.image" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: ["bundle", "exec", "sidekiq"] + envFrom: + - configMapRef: + name: {{ include "brimming.fullname" . }} + env: + - name: SECRET_KEY_BASE + valueFrom: + secretKeyRef: + name: {{ include "brimming.rails.secretName" . }} + key: {{ include "brimming.rails.secretKey" . }} + - name: DB_PASS + valueFrom: + secretKeyRef: + name: {{ include "brimming.postgresql.secretName" . }} + key: {{ include "brimming.postgresql.secretKey" . }} + {{- if or .Values.encryption.primaryKey .Values.encryption.existingSecret }} + - name: ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.primaryKey | default "primary-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.deterministicKey | default "deterministic-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.keyDerivationSalt | default "key-derivation-salt" }} + {{- end }} + # Sidekiq memory optimization + - name: MALLOC_ARENA_MAX + value: "2" + {{- with .Values.worker.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + volumes: + - name: tmp + emptyDir: {} + - name: rails-tmp + emptyDir: {} + {{- with .Values.worker.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.worker.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.worker.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} diff --git a/helm/brimming/templates/firecrawl/configmap-init.yaml b/helm/brimming/templates/firecrawl/configmap-init.yaml new file mode 100644 index 0000000..395ff2b --- /dev/null +++ b/helm/brimming/templates/firecrawl/configmap-init.yaml @@ -0,0 +1,109 @@ +{{- if .Values.firecrawl.enabled }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "brimming.fullname" . }}-firecrawl-init + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: firecrawl +data: + firecrawl-init.sql: | + -- Firecrawl NUQ schema (for job queuing) + -- Based on: https://github.com/mendableai/firecrawl/blob/main/apps/nuq-postgres/nuq.sql + + CREATE EXTENSION IF NOT EXISTS pgcrypto; + + CREATE SCHEMA IF NOT EXISTS nuq; + + DO $$ BEGIN + CREATE TYPE nuq.job_status AS ENUM ('queued', 'active', 'completed', 'failed'); + EXCEPTION + WHEN duplicate_object THEN null; + END $$; + + DO $$ BEGIN + CREATE TYPE nuq.group_status AS ENUM ('active', 'completed', 'cancelled'); + EXCEPTION + WHEN duplicate_object THEN null; + END $$; + + CREATE TABLE IF NOT EXISTS nuq.queue_scrape ( + id uuid NOT NULL DEFAULT gen_random_uuid(), + status nuq.job_status NOT NULL DEFAULT 'queued'::nuq.job_status, + data jsonb, + created_at timestamp with time zone NOT NULL DEFAULT now(), + priority int NOT NULL DEFAULT 0, + lock uuid, + locked_at timestamp with time zone, + stalls integer, + finished_at timestamp with time zone, + listen_channel_id text, + returnvalue jsonb, + failedreason text, + owner_id uuid, + group_id uuid, + CONSTRAINT queue_scrape_pkey PRIMARY KEY (id) + ); + + ALTER TABLE nuq.queue_scrape + SET (autovacuum_vacuum_scale_factor = 0.01, + autovacuum_analyze_scale_factor = 0.01, + autovacuum_vacuum_cost_limit = 2000, + autovacuum_vacuum_cost_delay = 2); + + CREATE INDEX IF NOT EXISTS queue_scrape_active_locked_at_idx ON nuq.queue_scrape USING btree (locked_at) WHERE (status = 'active'::nuq.job_status); + CREATE INDEX IF NOT EXISTS nuq_queue_scrape_queued_optimal_2_idx ON nuq.queue_scrape (priority ASC, created_at ASC, id) WHERE (status = 'queued'::nuq.job_status); + CREATE INDEX IF NOT EXISTS nuq_queue_scrape_failed_created_at_idx ON nuq.queue_scrape USING btree (created_at) WHERE (status = 'failed'::nuq.job_status); + CREATE INDEX IF NOT EXISTS nuq_queue_scrape_completed_created_at_idx ON nuq.queue_scrape USING btree (created_at) WHERE (status = 'completed'::nuq.job_status); + + CREATE TABLE IF NOT EXISTS nuq.queue_scrape_backlog ( + id uuid NOT NULL DEFAULT gen_random_uuid(), + data jsonb, + created_at timestamp with time zone NOT NULL DEFAULT now(), + priority int NOT NULL DEFAULT 0, + listen_channel_id text, + owner_id uuid, + group_id uuid, + times_out_at timestamptz, + CONSTRAINT queue_scrape_backlog_pkey PRIMARY KEY (id) + ); + + CREATE TABLE IF NOT EXISTS nuq.queue_crawl_finished ( + id uuid NOT NULL DEFAULT gen_random_uuid(), + status nuq.job_status NOT NULL DEFAULT 'queued'::nuq.job_status, + data jsonb, + created_at timestamp with time zone NOT NULL DEFAULT now(), + priority int NOT NULL DEFAULT 0, + lock uuid, + locked_at timestamp with time zone, + stalls integer, + finished_at timestamp with time zone, + listen_channel_id text, + returnvalue jsonb, + failedreason text, + owner_id uuid, + group_id uuid, + CONSTRAINT queue_crawl_finished_pkey PRIMARY KEY (id) + ); + + ALTER TABLE nuq.queue_crawl_finished + SET (autovacuum_vacuum_scale_factor = 0.01, + autovacuum_analyze_scale_factor = 0.01, + autovacuum_vacuum_cost_limit = 2000, + autovacuum_vacuum_cost_delay = 2); + + CREATE INDEX IF NOT EXISTS queue_crawl_finished_active_locked_at_idx ON nuq.queue_crawl_finished USING btree (locked_at) WHERE (status = 'active'::nuq.job_status); + CREATE INDEX IF NOT EXISTS nuq_queue_crawl_finished_queued_optimal_2_idx ON nuq.queue_crawl_finished (priority ASC, created_at ASC, id) WHERE (status = 'queued'::nuq.job_status); + + CREATE TABLE IF NOT EXISTS nuq.group_crawl ( + id uuid NOT NULL, + status nuq.group_status NOT NULL DEFAULT 'active'::nuq.group_status, + created_at timestamptz NOT NULL DEFAULT now(), + owner_id uuid NOT NULL, + ttl int8 NOT NULL DEFAULT 86400000, + expires_at timestamptz, + CONSTRAINT group_crawl_pkey PRIMARY KEY (id) + ); + + CREATE INDEX IF NOT EXISTS idx_group_crawl_status ON nuq.group_crawl (status) WHERE status = 'active'::nuq.group_status; +{{- end }} diff --git a/helm/brimming/templates/firecrawl/deployment.yaml b/helm/brimming/templates/firecrawl/deployment.yaml new file mode 100644 index 0000000..33d9fbb --- /dev/null +++ b/helm/brimming/templates/firecrawl/deployment.yaml @@ -0,0 +1,89 @@ +{{- if .Values.firecrawl.enabled }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "brimming.fullname" . }}-firecrawl + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: firecrawl +spec: + replicas: {{ .Values.firecrawl.replicaCount }} + selector: + matchLabels: + {{- include "brimming.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: firecrawl + template: + metadata: + {{- with .Values.firecrawl.podAnnotations }} + annotations: + {{- toYaml . | nindent 8 }} + {{- end }} + labels: + {{- include "brimming.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: firecrawl + spec: + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: firecrawl + image: "{{ .Values.firecrawl.image.repository }}:{{ .Values.firecrawl.image.tag }}" + imagePullPolicy: {{ .Values.firecrawl.image.pullPolicy }} + ports: + - name: http + containerPort: 3002 + protocol: TCP + env: + - name: PORT + value: "3002" + - name: HOST + value: "0.0.0.0" + - name: NUM_WORKERS_PER_QUEUE + value: {{ .Values.firecrawl.workersPerQueue | quote }} + - name: REDIS_URL + value: {{ include "brimming.firecrawl.redisUrl" . | quote }} + - name: REDIS_RATE_LIMIT_URL + value: {{ include "brimming.firecrawl.redisUrl" . | quote }} + - name: NUQ_DATABASE_URL + value: "postgres://{{ include "brimming.postgresql.username" . }}:$(DB_PASS)@{{ include "brimming.postgresql.host" . }}:{{ include "brimming.postgresql.port" . }}/{{ include "brimming.postgresql.database" . }}" + - name: DB_PASS + valueFrom: + secretKeyRef: + name: {{ include "brimming.postgresql.secretName" . }} + key: {{ include "brimming.postgresql.secretKey" . }} + - name: USE_DB_AUTHENTICATION + value: {{ .Values.firecrawl.useDbAuthentication | quote }} + - name: TEST_API_KEY + value: {{ .Values.firecrawl.testApiKey | quote }} + - name: ALLOW_LOCAL_WEBHOOKS + value: "true" + livenessProbe: + httpGet: + path: / + port: http + initialDelaySeconds: 30 + periodSeconds: 10 + readinessProbe: + httpGet: + path: / + port: http + initialDelaySeconds: 5 + periodSeconds: 5 + {{- with .Values.firecrawl.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.firecrawl.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.firecrawl.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.firecrawl.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/helm/brimming/templates/firecrawl/service.yaml b/helm/brimming/templates/firecrawl/service.yaml new file mode 100644 index 0000000..3e4d3e3 --- /dev/null +++ b/helm/brimming/templates/firecrawl/service.yaml @@ -0,0 +1,19 @@ +{{- if .Values.firecrawl.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "brimming.fullname" . }}-firecrawl + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: firecrawl +spec: + type: {{ .Values.firecrawl.service.type }} + ports: + - port: {{ .Values.firecrawl.service.port }} + targetPort: http + protocol: TCP + name: http + selector: + {{- include "brimming.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: firecrawl +{{- end }} diff --git a/helm/brimming/templates/hpa.yaml b/helm/brimming/templates/hpa.yaml new file mode 100644 index 0000000..9d40484 --- /dev/null +++ b/helm/brimming/templates/hpa.yaml @@ -0,0 +1,33 @@ +{{- if .Values.app.autoscaling.enabled }} +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "brimming.fullname" . }}-app + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: app +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: {{ include "brimming.fullname" . }}-app + minReplicas: {{ .Values.app.autoscaling.minReplicas }} + maxReplicas: {{ .Values.app.autoscaling.maxReplicas }} + metrics: + {{- if .Values.app.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.app.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.app.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.app.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} diff --git a/helm/brimming/templates/ingress.yaml b/helm/brimming/templates/ingress.yaml new file mode 100644 index 0000000..169211a --- /dev/null +++ b/helm/brimming/templates/ingress.yaml @@ -0,0 +1,41 @@ +{{- if .Values.ingress.enabled -}} +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: {{ include "brimming.fullname" . }} + labels: + {{- include "brimming.labels" . | nindent 4 }} + {{- with .Values.ingress.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + {{- if .Values.ingress.className }} + ingressClassName: {{ .Values.ingress.className }} + {{- end }} + {{- if .Values.ingress.tls }} + tls: + {{- range .Values.ingress.tls }} + - hosts: + {{- range .hosts }} + - {{ . | quote }} + {{- end }} + secretName: {{ .secretName }} + {{- end }} + {{- end }} + rules: + {{- range .Values.ingress.hosts }} + - host: {{ .host | quote }} + http: + paths: + {{- range .paths }} + - path: {{ .path }} + pathType: {{ .pathType }} + backend: + service: + name: {{ include "brimming.fullname" $ }} + port: + name: http + {{- end }} + {{- end }} +{{- end }} diff --git a/helm/brimming/templates/job-migrate.yaml b/helm/brimming/templates/job-migrate.yaml new file mode 100644 index 0000000..6ecdc10 --- /dev/null +++ b/helm/brimming/templates/job-migrate.yaml @@ -0,0 +1,123 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ include "brimming.fullname" . }}-migrate-{{ .Release.Revision }} + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: migrate + annotations: + # post-install: runs after fresh install (PostgreSQL just created) + # pre-upgrade: runs before deployments update (migrations complete first) + "helm.sh/hook": post-install,pre-upgrade + "helm.sh/hook-weight": "0" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded +spec: + backoffLimit: 3 + activeDeadlineSeconds: 300 + template: + metadata: + labels: + {{- include "brimming.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: migrate + spec: + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + serviceAccountName: {{ include "brimming.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.app.podSecurityContext | nindent 8 }} + restartPolicy: Never + initContainers: + - name: wait-for-postgres + image: {{ include "brimming.image" . }} + command: ['sh', '-c'] + args: + - | + echo "Waiting for PostgreSQL..." + until pg_isready -h $DB_HOST -p $DB_PORT -U $DB_USER; do + echo "PostgreSQL not ready, waiting..." + sleep 2 + done + echo "PostgreSQL is ready!" + env: + - name: DB_HOST + value: {{ include "brimming.postgresql.host" . | quote }} + - name: DB_PORT + value: {{ include "brimming.postgresql.port" . | quote }} + - name: DB_USER + value: {{ include "brimming.postgresql.username" . | quote }} + - name: fix-permissions + image: {{ include "brimming.image" . }} + command: ['sh', '-c', 'chmod 1777 /tmp && chmod 1777 /rails/tmp'] + securityContext: + runAsUser: 0 + runAsNonRoot: false + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + containers: + - name: migrate + securityContext: + {{- toYaml .Values.app.containerSecurityContext | nindent 12 }} + image: {{ include "brimming.image" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: ['sh', '-c'] + args: + - | + # db:prepare runs db:seed on fresh databases, which we don't want + # Instead: create database if needed, load schema, then run migrations + ./bin/rails db:create 2>/dev/null || true + ./bin/rails db:schema:load 2>/dev/null || true + ./bin/rails db:migrate + envFrom: + - configMapRef: + name: {{ include "brimming.fullname" . }} + env: + - name: SECRET_KEY_BASE + valueFrom: + secretKeyRef: + name: {{ include "brimming.rails.secretName" . }} + key: {{ include "brimming.rails.secretKey" . }} + - name: DB_PASS + valueFrom: + secretKeyRef: + name: {{ include "brimming.postgresql.secretName" . }} + key: {{ include "brimming.postgresql.secretKey" . }} + {{- if or .Values.encryption.primaryKey .Values.encryption.existingSecret }} + - name: ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.primaryKey | default "primary-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.deterministicKey | default "deterministic-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.keyDerivationSalt | default "key-derivation-salt" }} + {{- end }} + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + volumes: + - name: tmp + emptyDir: {} + - name: rails-tmp + emptyDir: {} + {{- with .Values.app.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.app.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} diff --git a/helm/brimming/templates/job-seed.yaml b/helm/brimming/templates/job-seed.yaml new file mode 100644 index 0000000..c05d4d8 --- /dev/null +++ b/helm/brimming/templates/job-seed.yaml @@ -0,0 +1,147 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ include "brimming.fullname" . }}-seed-{{ .Release.Revision }} + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: seed + annotations: + # Only run on initial install, not upgrades + "helm.sh/hook": post-install + "helm.sh/hook-weight": "1" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded +spec: + backoffLimit: 3 + activeDeadlineSeconds: 120 + template: + metadata: + labels: + {{- include "brimming.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: seed + spec: + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + serviceAccountName: {{ include "brimming.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.app.podSecurityContext | nindent 8 }} + restartPolicy: Never + initContainers: + - name: wait-for-migrate + image: {{ include "brimming.image" . }} + command: ['sh', '-c'] + args: + - | + echo "Waiting for database to be ready..." + until PGPASSWORD=$DB_PASS psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "SELECT 1 FROM ar_internal_metadata LIMIT 1" > /dev/null 2>&1; do + echo "Database not ready, waiting..." + sleep 2 + done + echo "Database is ready!" + env: + - name: DB_HOST + value: {{ include "brimming.postgresql.host" . | quote }} + - name: DB_PORT + value: {{ include "brimming.postgresql.port" . | quote }} + - name: DB_USER + value: {{ include "brimming.postgresql.username" . | quote }} + - name: DB_NAME + value: {{ include "brimming.postgresql.database" . | quote }} + - name: DB_PASS + valueFrom: + secretKeyRef: + name: {{ include "brimming.postgresql.secretName" . }} + key: {{ include "brimming.postgresql.secretKey" . }} + - name: fix-permissions + image: {{ include "brimming.image" . }} + command: ['sh', '-c', 'chmod 1777 /tmp && chmod 1777 /rails/tmp'] + securityContext: + runAsUser: 0 + runAsNonRoot: false + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + containers: + - name: seed + securityContext: + {{- toYaml .Values.app.containerSecurityContext | nindent 12 }} + image: {{ include "brimming.image" . }} + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: ['sh', '-c'] + args: + - | + ./bin/rails runner " + if User.exists? + puts 'Users already exist, skipping seed' + else + password = ENV['SEED_ADMIN_PASSWORD'].presence || SecureRandom.alphanumeric(24) + User.create!( + email: ENV['SEED_ADMIN_EMAIL'], + username: ENV['SEED_ADMIN_USERNAME'], + password: password, + password_confirmation: password, + role: :admin + ) + puts \"Admin user created: #{ENV['SEED_ADMIN_EMAIL']}\" + puts \"Password: #{password}\" if ENV['SEED_ADMIN_PASSWORD'].blank? + puts 'IMPORTANT: Save this password now - it will not be shown again!' if ENV['SEED_ADMIN_PASSWORD'].blank? + end + " + envFrom: + - configMapRef: + name: {{ include "brimming.fullname" . }} + env: + - name: SEED_ADMIN_EMAIL + value: {{ .Values.seedAdmin.email | quote }} + - name: SEED_ADMIN_USERNAME + value: {{ .Values.seedAdmin.username | quote }} + - name: SEED_ADMIN_PASSWORD + value: {{ .Values.seedAdmin.password | quote }} + - name: SECRET_KEY_BASE + valueFrom: + secretKeyRef: + name: {{ include "brimming.rails.secretName" . }} + key: {{ include "brimming.rails.secretKey" . }} + - name: DB_PASS + valueFrom: + secretKeyRef: + name: {{ include "brimming.postgresql.secretName" . }} + key: {{ include "brimming.postgresql.secretKey" . }} + {{- if or .Values.encryption.primaryKey .Values.encryption.existingSecret }} + - name: ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.primaryKey | default "primary-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.deterministicKey | default "deterministic-key" }} + - name: ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT + valueFrom: + secretKeyRef: + name: {{ .Values.encryption.existingSecret | default (printf "%s-rails" (include "brimming.fullname" .)) }} + key: {{ .Values.encryption.existingSecretKeys.keyDerivationSalt | default "key-derivation-salt" }} + {{- end }} + volumeMounts: + - name: tmp + mountPath: /tmp + - name: rails-tmp + mountPath: /rails/tmp + volumes: + - name: tmp + emptyDir: {} + - name: rails-tmp + emptyDir: {} + {{- with .Values.app.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.app.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} diff --git a/helm/brimming/templates/pdb.yaml b/helm/brimming/templates/pdb.yaml new file mode 100644 index 0000000..e533d3b --- /dev/null +++ b/helm/brimming/templates/pdb.yaml @@ -0,0 +1,19 @@ +{{- if .Values.podDisruptionBudget.enabled }} +apiVersion: policy/v1 +kind: PodDisruptionBudget +metadata: + name: {{ include "brimming.fullname" . }}-app + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: app +spec: + {{- if .Values.podDisruptionBudget.minAvailable }} + minAvailable: {{ .Values.podDisruptionBudget.minAvailable }} + {{- end }} + {{- if .Values.podDisruptionBudget.maxUnavailable }} + maxUnavailable: {{ .Values.podDisruptionBudget.maxUnavailable }} + {{- end }} + selector: + matchLabels: + {{- include "brimming.app.selectorLabels" . | nindent 6 }} +{{- end }} diff --git a/helm/brimming/templates/postgresql/configmap-init.yaml b/helm/brimming/templates/postgresql/configmap-init.yaml new file mode 100644 index 0000000..e6f9b54 --- /dev/null +++ b/helm/brimming/templates/postgresql/configmap-init.yaml @@ -0,0 +1,24 @@ +{{- if .Values.postgresql.enabled }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "brimming.fullname" . }}-postgresql-init + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: postgresql +data: + init.sql: | + -- Create the brimming schema for app tables + CREATE SCHEMA IF NOT EXISTS brimming; + + -- Grant usage to the brimming user + GRANT ALL ON SCHEMA brimming TO {{ .Values.postgresql.auth.username }}; + + -- Set default privileges so new tables are accessible + ALTER DEFAULT PRIVILEGES IN SCHEMA brimming GRANT ALL ON TABLES TO {{ .Values.postgresql.auth.username }}; + ALTER DEFAULT PRIVILEGES IN SCHEMA brimming GRANT ALL ON SEQUENCES TO {{ .Values.postgresql.auth.username }}; + + -- Enable required extensions in public schema (pgvector image has these available) + CREATE EXTENSION IF NOT EXISTS vector SCHEMA public; + CREATE EXTENSION IF NOT EXISTS pg_trgm SCHEMA public; +{{- end }} diff --git a/helm/brimming/templates/postgresql/secret.yaml b/helm/brimming/templates/postgresql/secret.yaml new file mode 100644 index 0000000..21acb18 --- /dev/null +++ b/helm/brimming/templates/postgresql/secret.yaml @@ -0,0 +1,12 @@ +{{- if and .Values.postgresql.enabled (not .Values.postgresql.auth.existingSecret) }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "brimming.fullname" . }}-postgresql + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: postgresql +type: Opaque +stringData: + password: {{ required "postgresql.auth.password is required when postgresql.enabled=true and existingSecret is not set" .Values.postgresql.auth.password | quote }} +{{- end }} diff --git a/helm/brimming/templates/postgresql/service.yaml b/helm/brimming/templates/postgresql/service.yaml new file mode 100644 index 0000000..0383849 --- /dev/null +++ b/helm/brimming/templates/postgresql/service.yaml @@ -0,0 +1,20 @@ +{{- if .Values.postgresql.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "brimming.fullname" . }}-postgresql + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: postgresql +spec: + type: ClusterIP + clusterIP: None + ports: + - port: 5432 + targetPort: postgresql + protocol: TCP + name: postgresql + selector: + {{- include "brimming.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: postgresql +{{- end }} diff --git a/helm/brimming/templates/postgresql/statefulset.yaml b/helm/brimming/templates/postgresql/statefulset.yaml new file mode 100644 index 0000000..fb056dc --- /dev/null +++ b/helm/brimming/templates/postgresql/statefulset.yaml @@ -0,0 +1,98 @@ +{{- if .Values.postgresql.enabled }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: {{ include "brimming.fullname" . }}-postgresql + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: postgresql +spec: + serviceName: {{ include "brimming.fullname" . }}-postgresql + replicas: 1 + selector: + matchLabels: + {{- include "brimming.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: postgresql + template: + metadata: + labels: + {{- include "brimming.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: postgresql + spec: + containers: + - name: postgresql + image: "{{ .Values.postgresql.image.repository }}:{{ .Values.postgresql.image.tag }}" + imagePullPolicy: {{ .Values.postgresql.image.pullPolicy }} + ports: + - name: postgresql + containerPort: 5432 + protocol: TCP + env: + - name: POSTGRES_USER + value: {{ .Values.postgresql.auth.username | quote }} + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "brimming.postgresql.secretName" . }} + key: {{ include "brimming.postgresql.secretKey" . }} + - name: POSTGRES_DB + value: {{ .Values.postgresql.auth.database | quote }} + - name: PGDATA + value: /var/lib/postgresql/data/pgdata + livenessProbe: + exec: + command: + - pg_isready + - -U + - {{ .Values.postgresql.auth.username }} + initialDelaySeconds: 30 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 6 + readinessProbe: + exec: + command: + - pg_isready + - -U + - {{ .Values.postgresql.auth.username }} + initialDelaySeconds: 5 + periodSeconds: 5 + timeoutSeconds: 5 + failureThreshold: 6 + {{- with .Values.postgresql.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + volumeMounts: + - name: data + mountPath: /var/lib/postgresql/data + - name: init-scripts + mountPath: /docker-entrypoint-initdb.d + volumes: + - name: init-scripts + configMap: + name: {{ include "brimming.fullname" . }}-postgresql-init + {{- with .Values.postgresql.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.postgresql.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.postgresql.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: ["ReadWriteOnce"] + {{- if .Values.postgresql.storage.storageClass }} + storageClassName: {{ .Values.postgresql.storage.storageClass | quote }} + {{- end }} + resources: + requests: + storage: {{ .Values.postgresql.storage.size }} +{{- end }} diff --git a/helm/brimming/templates/secret.yaml b/helm/brimming/templates/secret.yaml new file mode 100644 index 0000000..cf1259a --- /dev/null +++ b/helm/brimming/templates/secret.yaml @@ -0,0 +1,40 @@ +{{- if not .Values.rails.existingSecret }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "brimming.fullname" . }}-rails + labels: + {{- include "brimming.labels" . | nindent 4 }} +type: Opaque +stringData: + secret-key-base: {{ required "rails.secretKeyBase is required when rails.existingSecret is not set" .Values.rails.secretKeyBase | quote }} + {{- if .Values.encryption.primaryKey }} + primary-key: {{ .Values.encryption.primaryKey | quote }} + {{- end }} + {{- if .Values.encryption.deterministicKey }} + deterministic-key: {{ .Values.encryption.deterministicKey | quote }} + {{- end }} + {{- if .Values.encryption.keyDerivationSalt }} + key-derivation-salt: {{ .Values.encryption.keyDerivationSalt | quote }} + {{- end }} + {{- if and .Values.smtp.enabled (not .Values.smtp.existingSecret) }} + {{- if .Values.smtp.user }} + smtp-user: {{ .Values.smtp.user | quote }} + {{- end }} + {{- if .Values.smtp.password }} + smtp-password: {{ .Values.smtp.password | quote }} + {{- end }} + {{- end }} +{{- end }} +--- +{{- if and (not .Values.postgresql.enabled) (not .Values.externalPostgresql.existingSecret) .Values.externalPostgresql.password }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "brimming.fullname" . }}-postgresql-external + labels: + {{- include "brimming.labels" . | nindent 4 }} +type: Opaque +stringData: + password: {{ .Values.externalPostgresql.password | quote }} +{{- end }} diff --git a/helm/brimming/templates/service-app.yaml b/helm/brimming/templates/service-app.yaml new file mode 100644 index 0000000..01d85ae --- /dev/null +++ b/helm/brimming/templates/service-app.yaml @@ -0,0 +1,16 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ include "brimming.fullname" . }} + labels: + {{- include "brimming.labels" . | nindent 4 }} + app.kubernetes.io/component: app +spec: + type: {{ .Values.app.service.type }} + ports: + - port: {{ .Values.app.service.port }} + targetPort: http + protocol: TCP + name: http + selector: + {{- include "brimming.app.selectorLabels" . | nindent 4 }} diff --git a/helm/brimming/templates/serviceaccount.yaml b/helm/brimming/templates/serviceaccount.yaml new file mode 100644 index 0000000..4fa44fb --- /dev/null +++ b/helm/brimming/templates/serviceaccount.yaml @@ -0,0 +1,13 @@ +{{- if .Values.serviceAccount.create -}} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "brimming.serviceAccountName" . }} + labels: + {{- include "brimming.labels" . | nindent 4 }} + {{- with .Values.serviceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +automountServiceAccountToken: {{ .Values.serviceAccount.automount }} +{{- end }} diff --git a/helm/brimming/tests/app_deployment_test.yaml b/helm/brimming/tests/app_deployment_test.yaml new file mode 100644 index 0000000..e4cbba8 --- /dev/null +++ b/helm/brimming/tests/app_deployment_test.yaml @@ -0,0 +1,122 @@ +suite: test app deployment +templates: + - templates/deployment-app.yaml +tests: + - it: should render deployment with correct name + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - isKind: + of: Deployment + - equal: + path: metadata.name + value: RELEASE-NAME-brimming-app + + - it: should use image from values + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + image.repository: my-registry/brimming + image.tag: "1.2.3" + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: my-registry/brimming:1.2.3 + + - it: should default to Chart.AppVersion for image tag + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - matchRegex: + path: spec.template.spec.containers[0].image + pattern: ":1\\.0\\.0$" + + - it: should set correct replica count + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + app.replicaCount: 5 + asserts: + - equal: + path: spec.replicas + value: 5 + + - it: should not set replicas when autoscaling is enabled + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + app.autoscaling.enabled: true + asserts: + - isNull: + path: spec.replicas + + - it: should set security context + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - equal: + path: spec.template.spec.securityContext.runAsUser + value: 1000 + - equal: + path: spec.template.spec.containers[0].securityContext.readOnlyRootFilesystem + value: true + + - it: should mount tmp directories for read-only root filesystem + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - contains: + path: spec.template.spec.volumes + content: + name: tmp + emptyDir: {} + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: tmp + mountPath: /tmp + + - it: should set resource limits when specified + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + app.resources: + requests: + cpu: 500m + memory: 1Gi + limits: + cpu: 2000m + memory: 2Gi + asserts: + - equal: + path: spec.template.spec.containers[0].resources.requests.cpu + value: 500m + - equal: + path: spec.template.spec.containers[0].resources.limits.memory + value: 2Gi + + - it: should include checksum annotations for config changes + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - exists: + path: spec.template.metadata.annotations["checksum/config"] + - exists: + path: spec.template.metadata.annotations["checksum/secret"] + + - it: should set correct healthcheck probes + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - equal: + path: spec.template.spec.containers[0].livenessProbe.httpGet.path + value: /up + - equal: + path: spec.template.spec.containers[0].readinessProbe.httpGet.path + value: /up diff --git a/helm/brimming/tests/external_postgres_test.yaml b/helm/brimming/tests/external_postgres_test.yaml new file mode 100644 index 0000000..1d98bd6 --- /dev/null +++ b/helm/brimming/tests/external_postgres_test.yaml @@ -0,0 +1,73 @@ +suite: test external postgresql configuration +templates: + - templates/configmap.yaml + - templates/secret.yaml + - templates/deployment-app.yaml +tests: + - it: should use external postgresql host when postgresql.enabled=false + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: false + externalPostgresql.host: external-pg.example.com + externalPostgresql.port: 5433 + externalPostgresql.database: mydb + externalPostgresql.username: myuser + externalPostgresql.password: mypassword + asserts: + - template: templates/configmap.yaml + equal: + path: data.DB_HOST + value: external-pg.example.com + - template: templates/configmap.yaml + equal: + path: data.DB_PORT + value: "5433" + - template: templates/configmap.yaml + equal: + path: data.DB_NAME + value: mydb + - template: templates/configmap.yaml + equal: + path: data.DB_USER + value: myuser + + - it: should create external postgresql secret when password provided + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: false + externalPostgresql.host: external-pg.example.com + externalPostgresql.password: mypassword + asserts: + - template: templates/secret.yaml + containsDocument: + kind: Secret + apiVersion: v1 + name: RELEASE-NAME-brimming-postgresql-external + + - it: should use existing secret when specified + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: false + externalPostgresql.host: external-pg.example.com + externalPostgresql.existingSecret: my-pg-secret + externalPostgresql.existingSecretKey: pg-password + asserts: + - template: templates/deployment-app.yaml + contains: + path: spec.template.spec.containers[0].env + content: + name: DB_PASS + valueFrom: + secretKeyRef: + name: my-pg-secret + key: pg-password + + - it: should fail when postgresql disabled and no external host + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: false + externalPostgresql.host: "" + asserts: + - template: templates/configmap.yaml + failedTemplate: + errorMessage: "externalPostgresql.host is required when postgresql.enabled=false" diff --git a/helm/brimming/tests/external_valkey_test.yaml b/helm/brimming/tests/external_valkey_test.yaml new file mode 100644 index 0000000..a054e52 --- /dev/null +++ b/helm/brimming/tests/external_valkey_test.yaml @@ -0,0 +1,36 @@ +suite: test external valkey configuration +templates: + - templates/configmap.yaml +tests: + - it: should use internal valkey host when valkey.enabled=true + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + valkey.enabled: true + asserts: + - equal: + path: data.REDIS_URL + value: redis://RELEASE-NAME-brimming-valkey:6379/0 + + - it: should use external valkey host when valkey.enabled=false + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + valkey.enabled: false + externalValkey.host: redis.example.com + externalValkey.port: 6380 + externalValkey.database: 2 + asserts: + - equal: + path: data.REDIS_URL + value: redis://redis.example.com:6380/2 + + - it: should fail when valkey disabled and no external host + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + valkey.enabled: false + externalValkey.host: "" + asserts: + - failedTemplate: + errorMessage: "externalValkey.host is required when valkey.enabled=false" diff --git a/helm/brimming/tests/firecrawl_test.yaml b/helm/brimming/tests/firecrawl_test.yaml new file mode 100644 index 0000000..6ed75d3 --- /dev/null +++ b/helm/brimming/tests/firecrawl_test.yaml @@ -0,0 +1,80 @@ +suite: test firecrawl optional component +templates: + - templates/firecrawl/deployment.yaml + - templates/firecrawl/service.yaml +tests: + - it: should not render when firecrawl.enabled=false + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + firecrawl.enabled: false + asserts: + - template: templates/firecrawl/deployment.yaml + hasDocuments: + count: 0 + - template: templates/firecrawl/service.yaml + hasDocuments: + count: 0 + + - it: should render deployment when firecrawl.enabled=true + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + firecrawl.enabled: true + asserts: + - template: templates/firecrawl/deployment.yaml + isKind: + of: Deployment + - template: templates/firecrawl/deployment.yaml + equal: + path: metadata.name + value: RELEASE-NAME-brimming-firecrawl + + - it: should use correct firecrawl image + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + firecrawl.enabled: true + asserts: + - template: templates/firecrawl/deployment.yaml + equal: + path: spec.template.spec.containers[0].image + value: ghcr.io/mendableai/firecrawl:latest + + - it: should use valkey database 1 for firecrawl + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + firecrawl.enabled: true + asserts: + - template: templates/firecrawl/deployment.yaml + contains: + path: spec.template.spec.containers[0].env + content: + name: REDIS_URL + value: redis://RELEASE-NAME-brimming-valkey:6379/1 + + - it: should render service when firecrawl.enabled=true + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + firecrawl.enabled: true + asserts: + - template: templates/firecrawl/service.yaml + isKind: + of: Service + - template: templates/firecrawl/service.yaml + equal: + path: spec.ports[0].port + value: 3002 + + - it: should have firecrawl component label + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + firecrawl.enabled: true + asserts: + - template: templates/firecrawl/deployment.yaml + equal: + path: metadata.labels["app.kubernetes.io/component"] + value: firecrawl diff --git a/helm/brimming/tests/ingress_test.yaml b/helm/brimming/tests/ingress_test.yaml new file mode 100644 index 0000000..60a811c --- /dev/null +++ b/helm/brimming/tests/ingress_test.yaml @@ -0,0 +1,69 @@ +suite: test ingress +templates: + - templates/ingress.yaml +tests: + - it: should not render when ingress.enabled=false + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + ingress.enabled: false + asserts: + - hasDocuments: + count: 0 + + - it: should render ingress when enabled + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + ingress.enabled: true + ingress.hosts: + - host: brimming.example.com + paths: + - path: / + pathType: Prefix + asserts: + - isKind: + of: Ingress + - equal: + path: spec.rules[0].host + value: brimming.example.com + + - it: should set ingress class when specified + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + ingress.enabled: true + ingress.className: nginx + asserts: + - equal: + path: spec.ingressClassName + value: nginx + + - it: should configure TLS when specified + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + ingress.enabled: true + ingress.tls: + - secretName: brimming-tls + hosts: + - brimming.example.com + asserts: + - equal: + path: spec.tls[0].secretName + value: brimming-tls + - contains: + path: spec.tls[0].hosts + content: brimming.example.com + + - it: should add annotations when specified + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + ingress.enabled: true + ingress.annotations: + cert-manager.io/cluster-issuer: letsencrypt-prod + asserts: + - equal: + path: metadata.annotations["cert-manager.io/cluster-issuer"] + value: letsencrypt-prod diff --git a/helm/brimming/tests/postgresql_test.yaml b/helm/brimming/tests/postgresql_test.yaml new file mode 100644 index 0000000..ea7f531 --- /dev/null +++ b/helm/brimming/tests/postgresql_test.yaml @@ -0,0 +1,125 @@ +suite: test postgresql templates +templates: + - templates/postgresql/statefulset.yaml + - templates/postgresql/service.yaml + - templates/postgresql/secret.yaml + - templates/postgresql/configmap-init.yaml +tests: + - it: should render statefulset when postgresql.enabled=true + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.password: test-pg-pass + asserts: + - template: templates/postgresql/statefulset.yaml + isKind: + of: StatefulSet + - template: templates/postgresql/statefulset.yaml + equal: + path: metadata.name + value: RELEASE-NAME-brimming-postgresql + + - it: should use pgvector image + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.password: test-pg-pass + asserts: + - template: templates/postgresql/statefulset.yaml + equal: + path: spec.template.spec.containers[0].image + value: pgvector/pgvector:pg17 + + - it: should allow custom image tag + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.password: test-pg-pass + postgresql.image.tag: pg16 + asserts: + - template: templates/postgresql/statefulset.yaml + equal: + path: spec.template.spec.containers[0].image + value: pgvector/pgvector:pg16 + + - it: should create headless service + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.password: test-pg-pass + asserts: + - template: templates/postgresql/service.yaml + isKind: + of: Service + - template: templates/postgresql/service.yaml + equal: + path: spec.clusterIP + value: None + + - it: should create secret with password + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.password: my-pg-password + asserts: + - template: templates/postgresql/secret.yaml + isKind: + of: Secret + - template: templates/postgresql/secret.yaml + equal: + path: stringData.password + value: my-pg-password + + - it: should not create secret when existingSecret is set + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.existingSecret: my-existing-secret + asserts: + - template: templates/postgresql/secret.yaml + hasDocuments: + count: 0 + + - it: should create init configmap with extensions + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.password: test-pg-pass + asserts: + - template: templates/postgresql/configmap-init.yaml + isKind: + of: ConfigMap + - template: templates/postgresql/configmap-init.yaml + matchRegex: + path: data["init.sql"] + pattern: "CREATE EXTENSION IF NOT EXISTS vector" + - template: templates/postgresql/configmap-init.yaml + matchRegex: + path: data["init.sql"] + pattern: "CREATE EXTENSION IF NOT EXISTS pg_trgm" + + - it: should set correct storage size + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: true + postgresql.auth.password: test-pg-pass + postgresql.storage.size: 50Gi + asserts: + - template: templates/postgresql/statefulset.yaml + equal: + path: spec.volumeClaimTemplates[0].spec.resources.requests.storage + value: 50Gi + + - it: should not render when postgresql.enabled=false + set: + rails.secretKeyBase: test-secret-key + postgresql.enabled: false + externalPostgresql.host: external-pg.example.com + externalPostgresql.password: ext-pass + asserts: + - template: templates/postgresql/statefulset.yaml + hasDocuments: + count: 0 + - template: templates/postgresql/service.yaml + hasDocuments: + count: 0 diff --git a/helm/brimming/tests/worker_deployment_test.yaml b/helm/brimming/tests/worker_deployment_test.yaml new file mode 100644 index 0000000..6d9debc --- /dev/null +++ b/helm/brimming/tests/worker_deployment_test.yaml @@ -0,0 +1,64 @@ +suite: test worker deployment +templates: + - templates/deployment-worker.yaml +tests: + - it: should render deployment with correct name + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - isKind: + of: Deployment + - equal: + path: metadata.name + value: RELEASE-NAME-brimming-worker + + - it: should use same image as app + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + image.repository: my-registry/brimming + image.tag: "1.2.3" + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: my-registry/brimming:1.2.3 + + - it: should run sidekiq command + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - equal: + path: spec.template.spec.containers[0].command + value: ["bundle", "exec", "sidekiq"] + + - it: should set MALLOC_ARENA_MAX for memory optimization + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: MALLOC_ARENA_MAX + value: "2" + + - it: should set correct replica count + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + worker.replicaCount: 3 + asserts: + - equal: + path: spec.replicas + value: 3 + + - it: should have worker component label + set: + rails.secretKeyBase: test-secret-key + postgresql.auth.password: test-pg-pass + asserts: + - equal: + path: metadata.labels["app.kubernetes.io/component"] + value: worker diff --git a/helm/brimming/values.yaml b/helm/brimming/values.yaml new file mode 100644 index 0000000..758e7e5 --- /dev/null +++ b/helm/brimming/values.yaml @@ -0,0 +1,317 @@ +# ============================================================================= +# BRIMMING HELM CHART VALUES +# ============================================================================= +# Use `~` (null) for nested values you want users to override without defaults +# ============================================================================= + +# ----------------------------------------------------------------------------- +# Global Settings +# ----------------------------------------------------------------------------- +global: + imageRegistry: "" + +# ----------------------------------------------------------------------------- +# Application Image +# ----------------------------------------------------------------------------- +image: + repository: ghcr.io/tightline/brimming + tag: "" # Defaults to Chart.appVersion + pullPolicy: IfNotPresent + +imagePullSecrets: [] +nameOverride: "" +fullnameOverride: "" + +# ----------------------------------------------------------------------------- +# Rails Application (app) +# ----------------------------------------------------------------------------- +app: + replicaCount: 2 + + resources: ~ + # Example: + # requests: + # cpu: 250m + # memory: 512Mi + # limits: + # cpu: 1000m + # memory: 1Gi + + livenessProbe: + httpGet: + path: /up + port: http + initialDelaySeconds: 30 + periodSeconds: 10 + + readinessProbe: + httpGet: + path: /up + port: http + initialDelaySeconds: 5 + periodSeconds: 5 + + service: + type: ClusterIP + port: 80 + + autoscaling: + enabled: false + minReplicas: 2 + maxReplicas: 10 + targetCPUUtilizationPercentage: 70 + + podAnnotations: {} + podLabels: {} + nodeSelector: {} + tolerations: [] + affinity: ~ + + podSecurityContext: + runAsUser: 1000 + runAsGroup: 1000 + fsGroup: 1000 + + containerSecurityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: + drop: ["ALL"] + +# ----------------------------------------------------------------------------- +# Sidekiq Worker +# ----------------------------------------------------------------------------- +worker: + replicaCount: 1 + + resources: ~ + # Example: + # requests: + # cpu: 100m + # memory: 256Mi + # limits: + # cpu: 500m + # memory: 512Mi + + podAnnotations: {} + nodeSelector: {} + tolerations: [] + affinity: ~ + +# ----------------------------------------------------------------------------- +# Service Account +# ----------------------------------------------------------------------------- +serviceAccount: + create: true + automount: true + annotations: {} + name: "" + +# ----------------------------------------------------------------------------- +# Ingress +# ----------------------------------------------------------------------------- +ingress: + enabled: false + className: "" + annotations: {} + # kubernetes.io/ingress.class: nginx + # cert-manager.io/cluster-issuer: letsencrypt-prod + hosts: + - host: brimming.example.com + paths: + - path: / + pathType: Prefix + tls: [] + # - secretName: brimming-tls + # hosts: + # - brimming.example.com + +# ----------------------------------------------------------------------------- +# Rails Configuration +# ----------------------------------------------------------------------------- +rails: + # Required - Rails secret key base + secretKeyBase: "" + existingSecret: "" + existingSecretKey: "secret-key-base" + + # Application base URL (used for links in emails, etc.) + baseUrl: "https://brimming.example.com" + + # Additional environment variables + env: + RAILS_ENV: production + RAILS_LOG_TO_STDOUT: "true" + RAILS_MAX_THREADS: "5" + +# Initial admin account (created on fresh install only) +# IMPORTANT: Change these values in production! +seedAdmin: + email: "admin@example.com" + username: "admin" + # If password is empty, a random 24-character password will be generated + # and displayed in the seed job logs + password: "" + +# Active Record Encryption (optional but recommended for production) +encryption: + primaryKey: "" + deterministicKey: "" + keyDerivationSalt: "" + existingSecret: "" + existingSecretKeys: + primaryKey: "primary-key" + deterministicKey: "deterministic-key" + keyDerivationSalt: "key-derivation-salt" + +# SMTP Configuration (optional) +smtp: + enabled: false + host: "" + port: 587 + tls: "" # "true", "false", or "" (auto) + user: "" + password: "" + auth: "plain" # plain, login, cram_md5 + domain: "" + from: "noreply@brimming.example.com" + existingSecret: "" + existingSecretKeys: + user: "smtp-user" + password: "smtp-password" + +# ----------------------------------------------------------------------------- +# PostgreSQL - Internal (Custom StatefulSet) +# ----------------------------------------------------------------------------- +postgresql: + enabled: true + + image: + repository: pgvector/pgvector + tag: pg17 + pullPolicy: IfNotPresent + + auth: + database: brimming + username: brimming + password: "" # Required if existingSecret not set + existingSecret: "" + existingSecretKey: "password" + + storage: + size: 10Gi + storageClass: ~ # Use cluster default + + resources: ~ + # Example: + # requests: + # cpu: 100m + # memory: 256Mi + # limits: + # cpu: 1000m + # memory: 1Gi + + nodeSelector: {} + tolerations: [] + affinity: ~ + +# External PostgreSQL (when postgresql.enabled=false) +externalPostgresql: + host: "" + port: 5432 + database: "brimming" + username: "brimming" + password: "" + existingSecret: "" + existingSecretKey: "password" + +# ----------------------------------------------------------------------------- +# Valkey (Redis-compatible) - Internal Subchart +# ----------------------------------------------------------------------------- +valkey: + enabled: true + + # Values passed to the valkey subchart + # See: https://github.com/valkey-io/valkey-helm + image: + registry: docker.io + repository: valkey/valkey + tag: "9.0.0" + + # Auth config + auth: + enabled: false + + # Persistent storage config + dataStorage: + enabled: true + requestedSize: 1Gi + className: "" # Use cluster default storage class + keepPvc: true # Preserve data on helm uninstall + + resources: + limits: + cpu: 100m + memory: 128Mi + requests: + cpu: 100m + memory: 128Mi + +# External Valkey/Redis (when valkey.enabled=false) +externalValkey: + host: "" + port: 6379 + password: "" + database: 0 + existingSecret: "" + existingSecretKey: "password" + tls: + enabled: false + +# ----------------------------------------------------------------------------- +# Firecrawl - Optional Web Scraper +# ----------------------------------------------------------------------------- +# NOTE: Firecrawl support is incomplete. The deployment templates exist but +# Firecrawl requires RabbitMQ which is not yet included in this chart. +# Leave disabled until RabbitMQ integration is added. +firecrawl: + enabled: false + + image: + repository: ghcr.io/mendableai/firecrawl + tag: latest + pullPolicy: IfNotPresent + + replicaCount: 1 + workersPerQueue: 4 + + # Firecrawl uses Valkey database 1 (separate from Rails which uses 0) + valkeyDatabase: 1 + + # API authentication (optional for self-hosted) + testApiKey: "fc-dev" + useDbAuthentication: "false" + + service: + type: ClusterIP + port: 3002 + + resources: ~ + + podAnnotations: {} + nodeSelector: {} + tolerations: [] + affinity: ~ + +# ----------------------------------------------------------------------------- +# Pod Disruption Budget +# ----------------------------------------------------------------------------- +podDisruptionBudget: + enabled: true + minAvailable: 1 + +# ----------------------------------------------------------------------------- +# Network Policy (optional) +# ----------------------------------------------------------------------------- +networkPolicy: + enabled: false