A Kubernetes-native security detection pipeline that collects, enriches, and ships runtime security events from Falco, Tetragon, and workload logs to Elasticsearch via Kafka.
┌─────────────────────────────────────────────────────────────────────┐
│ EKS Cluster │
│ │
│ ┌─────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Pod Security │ │ Falco │ │ Tetragon │ │
│ │ Annotator │ │ (DaemonSet) │ │ (DaemonSet) │ │
│ │ │ │ │ │ │ │
│ │ Watches pods │ │ syscall + K8s │ │ eBPF process │ │
│ │ Patches security│ │ audit events │ │ tracing │ │
│ │ annotations │ │ → stdout (JSON) │ │ → stdout (JSON) │ │
│ └────────┬────────┘ └────────┬─────────┘ └────────┬─────────┘ │
│ │ security.k8s.io/* │ │ │
│ │ ▼ ▼ │
│ │ ┌──────────────────────────────────────────┐ │
│ └─────────►│ Fluent Bit (DaemonSet) │ │
│ │ │ │
│ │ INPUT: tail /var/log/containers/*.log │ │
│ │ FILTER: kubernetes (metadata + annots) │ │
│ │ FILTER: record_modifier (source_type) │ │
│ │ OUTPUT: Kafka │ │
│ └────────────────────┬─────────────────────┘ │
└───────────────────────────────────────────│─────────────────────────┘
│
▼
┌────────────────────────┐
│ AWS MSK │
│ │
│ siem-falco │
│ siem-tetragon │
│ siem-k8s │
└───────────┬────────────┘
│
▼
┌────────────────────────┐
│ Logstash │
│ │
│ 0200_filter_falco │
│ 0201_filter_tetragon │
│ 0202_filter_k8s │
└───────────┬────────────┘
│
▼
┌────────────────────────┐
│ Elasticsearch │
│ │
│ logs-falco-siem │
│ logs-tetragon-siem │
│ logs-kubernetes-siem │
└────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ EKS Cluster │
│ │
│ kube-apiserver ──► CloudWatch Logs (/aws/eks/<cluster>/cluster) │
│ │
└─────────────────────────────┬───────────────────────────────────────┘
│ Subscription Filter (push-based)
▼
┌────────────────────────┐
│ Lambda │
│ cloudwatch-to-kafka │
│ │
│ - IRSA (no static │
│ credentials) │
│ - VPC-attached │
│ - injects source_type │
└───────────┬────────────┘
│
▼
┌────────────────────────┐
│ AWS MSK │
│ │
│ siem-k8s-audit │
└───────────┬────────────┘
│
▼
┌────────────────────────┐
│ Logstash │
│ │
│ 0203_filter_k8s_audit │
└───────────┬────────────┘
│
▼
┌────────────────────────┐
│ Elasticsearch │
│ │
│ logs-kubernetes-siem │
└────────────────────────┘
EKS does not expose the control plane, so API server audit logs are shipped via CloudWatch Logs
rather than collected directly. This means audit log events like kubectl exec, secret enumeration, and
RBAC changes arrive with no pod-level security context. You cannot tell from the audit log alone
whether the pod involved was privileged, had hostPath mounts, or had a ServiceAccount token auto-mounted.
The Pod Security Annotator solves this by running as a watch controller that continuously patches
security.k8s.io/* annotations onto every pod based on its spec. Because Fluent Bit reads these
annotations when it collects container logs, and because the audit log pipeline correlates events
by pod identity, every event like runtime logs, Falco alerts, Tetragon traces, and audit logs ends
up carrying the same security context without any join at query time.
Annotations added:
| Annotation | Description |
|---|---|
security.k8s.io/privileged |
Whether any container runs as privileged |
security.k8s.io/host-pid |
hostPID enabled |
security.k8s.io/host-network |
hostNetwork enabled |
security.k8s.io/host-ipc |
hostIPC enabled |
security.k8s.io/host-path |
hostPath volumes present |
security.k8s.io/host-path-mounts |
hostPath mount paths |
security.k8s.io/host-path-sensitive |
Sensitive paths (/etc, /proc, etc.) |
security.k8s.io/capabilities-added |
Added Linux capabilities |
security.k8s.io/allow-privilege-escalation |
allowPrivilegeEscalation |
security.k8s.io/run-as-non-root |
runAsNonRoot |
security.k8s.io/run-as-user |
runAsUser |
security.k8s.io/automount-sa-token |
automountServiceAccountToken |
security.k8s.io/network-policy-applied |
NetworkPolicy covers this pod |
security.k8s.io/service-account |
ServiceAccount name |
security.k8s.io/container-ports |
Declared container ports |
security.k8s.io/image-tag-latest |
Uses :latest tag |
security.k8s.io/image-pull-policy |
Image pull policy |
DaemonSet that collects container logs from every node.
- Input:
tailon/var/log/containers/*.log— separate inputs for Falco, Tetragon, and workload logs - Filter:
kubernetes— enriches with pod labels, annotations (including security.k8s.io/*), namespace, node info - Filter:
record_modifier— addscluster_nameandsource_type - Output: Kafka (MSK, unauthenticated on internal network)
EKS API server audit logs are collected via CloudWatch Logs and forwarded to Kafka using a Lambda function, then normalized to ECS by Logstash.
Collection flow:
kube-apiserver
→ CloudWatch Logs (/aws/eks/<cluster>/cluster)
→ Lambda (Subscription Filter, push-based, real-time)
→ MSK Kafka (siem-k8s-audit)
→ Logstash (0203_filter_k8s_audit.conf)
→ Elasticsearch (siem-k8s-audit)
Infrastructure:
- Lambda runs inside VPC with IRSA — no static credentials
- CloudWatch Subscription Filter triggers Lambda in real-time (no polling)
- Deployed and managed via Terraform
ECS Field Mapping:
| ECS Field | Source Field |
|---|---|
event.action |
verb |
event.outcome |
responseStatus.code (2xx → success, 4xx → failure) |
user.name |
user.username |
source.ip |
sourceIPs |
url.path |
requestURI |
kubernetes.audit.objectRef.resource |
objectRef.resource |
kubernetes.audit.objectRef.subresource |
objectRef.subresource |
kubernetes.audit.objectRef.namespace |
objectRef.namespace |
kubernetes.audit.responseStatus.code |
responseStatus.code |
kubernetes.audit.authorization.decision |
annotations.authorization.k8s.io/decision |
kubernetes.audit.authorization.reason |
annotations.authorization.k8s.io/reason |
| File | Role |
|---|---|
0000_input.conf |
Kafka consumer for all siem-* topics |
0200_filter_falco.conf |
ECS mapping for Falco alerts — maps to rule.*, process.*, event.kind: alert |
0201_filter_tetragon.conf |
ECS mapping for Tetragon events — maps to process.*, event type routing |
0202_filter_k8s.conf |
ECS mapping for K8s workload logs — promotes security.k8s.io/* to kubernetes.security.* |
0203_filter_k8s_audit.conf |
ECS mapping for K8s Audit Log — maps to event.action, user.name, kubernetes.audit.* |
9999_output.conf |
Routes to Elasticsearch Data Streams or regular indices |
All events are mapped to Elastic Common Schema (ECS) 8.x:
host.name ← kubernetes.host
container.name ← kubernetes.container_name
container.image.name ← kubernetes.container_image
container.id ← kubernetes.docker_id
orchestrator.cluster.name ← cluster_name
orchestrator.namespace ← kubernetes.namespace_name
orchestrator.resource.name ← kubernetes.pod_name
orchestrator.resource.ip ← kubernetes.pod_ip
orchestrator.type → "kubernetes"
kubernetes.security.* ← security.k8s.io/* annotations
| Data Stream | Source | ILM |
|---|---|---|
logs-falco-siem |
Falco alerts | siem policy |
logs-tetragon-siem |
Tetragon process events | siem policy |
logs-kubernetes-siem |
K8s workload logs | siem policy |
siem-k8s-audit |
K8s API server audit logs | siem policy |
K8S-DETECTION-PIPELINE/
├── falco/
│ └── values.yaml # Helm values (json_output: true)
├── fluent-bit/
│ ├── configmap.yaml # Pipeline config (inputs/filters/outputs)
│ ├── daemonset.yaml # DaemonSet + env vars
│ └── rbac.yaml # ServiceAccount + ClusterRole
├── pod-security-annotator/
│ ├── pod_security_annotator.py # Python watch controller
│ ├── requirements.txt
│ ├── Dockerfile
│ └── k8s/
│ └── manifests.yaml # RBAC + Deployment
├── lambda/
│ └── cloudwatch_to_kafka/
│ ├── main.py # CloudWatch → Kafka forwarder
│ ├── requirements.txt
│ └── builder.sh # Lambda layer build script
└── logstash/
└── pipeline/
├── 0000_input.conf
├── 0200_filter_falco.conf
├── 0201_filter_tetragon.conf
├── 0202_filter_k8s.conf
├── 0203_filter_k8s_audit.conf
└── 9999_output.conf
With security annotations enriched in every log event, the following queries become trivial in Kibana/Elastic SIEM:
# Privileged pod executed a shell
kubernetes.security.privileged: true AND rule.name: *shell*
# Process exec in a pod without NetworkPolicy
kubernetes.security.network_policy_applied: false AND event.module: tetragon
# Falco alert from a pod with sensitive host mounts
kubernetes.security.host_path_sensitive: true AND event.module: falco
# SA token auto-mounted + running as root
kubernetes.security.automount_sa_token: true AND
kubernetes.security.run_as_non_root: false AND
event.module: falco
# Secret enumeration via kubectl
kubernetes.audit.objectRef.resource: secrets AND
event.action: (get OR list) AND
event.outcome: success
# Pod exec (kubectl exec)
kubernetes.audit.objectRef.resource: pods AND
kubernetes.audit.objectRef.subresource: exec
# ClusterRoleBinding created
event.action: create AND
kubernetes.audit.objectRef.resource: clusterrolebindings
Platform-independent detection rules mapped to MITRE ATT&CK for Containers.
See sigma-rules/ for Secret Enumeration, Pod Exec,
ClusterRoleBinding creation rules.
Copyright 2026 jaypark81 Licensed under the Apache License, Version 2.0. See LICENSE for details.