-
Notifications
You must be signed in to change notification settings - Fork 447
Description
Summary
ECS currently defines several fields using type: object with zero defined children. These fields create namespaces where any producer can define arbitrary subfields with any type, bypassing ECS's core guarantee: that field names, types, and semantics are coordinated across all data sources. We need two changes:
- The project should give guidance on when open-ended
objectfields are acceptable and be clear that other submissions will be rejected. - The entity fields need to be refined before moving them beyond beta status.
Motivation
The entity field set is the primary example. It introduced 5 open object containers, each replicated across 15 reuse locations. Before the entity field set, ECS had only 2 intentionally open object fields (labels and container.labels).
entity.attributesentity.behaviorentity.lifecycleentity.metricsentity.raw
This problem was flagged during review but we still proceeded.
If we allow this pattern to continue, creating consistent data producers and consumers will be harder than it needs to be.
1. Field type conflicts across integrations
If one integration defines entity.attributes.managed as boolean and another defines it as keyword, documents from both landing in the same data view will produce mapping conflicts. There is no schema enforcement because the subfields were never defined in ECS.
2. Bypasses the ECS review process
Fields like entity.attributes.mfa_enabled, entity.attributes.granted_permissions, and entity.relationships.supervises carry the authority of the ECS namespace but were never proposed, reviewed, or ratified as ECS fields. Individual integration teams define them independently, which is exactly the coordination problem ECS exists to solve.
Detailed Design
1. Document the policy and update the RFC template
Add guidance to the schema documentation and the RFC template clarifying when object and flattened field types are acceptable, and when explicit leaf fields must be used instead. See #2594.
2. Address existing beta entity fields
Since entity.attributes, entity.behavior, entity.lifecycle, and entity.metrics are all still beta, the subfields that integrations need (e.g., entity.attributes.mfa_enabled, entity.lifecycle.first_seen, entity.attributes.granted_permissions) should be proposed as explicit leaf fields through the RFC process. This converts the parent objects from empty containers into structured field sets with defined children, types, and descriptions. This is the same pattern used successfully throughout the rest of ECS (e.g., user.group with children .id, .name, .domain).
The stated purpose of these fields is "advanced searching and correlation of normalized values across different providers/sources." That purpose is only achievable when the field names and types are coordinated. An open object with no defined children provides a namespace but no coordination. It shifts the type-safety problem from "undefined" to "defined independently by each integration," which is worse because it creates the appearance of compatibility without the substance.
entity.raw should remain as-is. It is explicitly documented as source-specific opaque data and is the correct use of an open object.