Skip to content

Restrict use of open object typed fields in ECS schema #2595

@andrewkroh

Description

@andrewkroh

Summary

ECS currently defines several fields using type: object with zero defined children. These fields create namespaces where any producer can define arbitrary subfields with any type, bypassing ECS's core guarantee: that field names, types, and semantics are coordinated across all data sources. We need two changes:

  1. The project should give guidance on when open-ended object fields are acceptable and be clear that other submissions will be rejected.
  2. The entity fields need to be refined before moving them beyond beta status.

Motivation

The entity field set is the primary example. It introduced 5 open object containers, each replicated across 15 reuse locations. Before the entity field set, ECS had only 2 intentionally open object fields (labels and container.labels).

  • entity.attributes
  • entity.behavior
  • entity.lifecycle
  • entity.metrics
  • entity.raw

This problem was flagged during review but we still proceeded.

If we allow this pattern to continue, creating consistent data producers and consumers will be harder than it needs to be.

1. Field type conflicts across integrations

If one integration defines entity.attributes.managed as boolean and another defines it as keyword, documents from both landing in the same data view will produce mapping conflicts. There is no schema enforcement because the subfields were never defined in ECS.

2. Bypasses the ECS review process

Fields like entity.attributes.mfa_enabled, entity.attributes.granted_permissions, and entity.relationships.supervises carry the authority of the ECS namespace but were never proposed, reviewed, or ratified as ECS fields. Individual integration teams define them independently, which is exactly the coordination problem ECS exists to solve.

Detailed Design

1. Document the policy and update the RFC template

Add guidance to the schema documentation and the RFC template clarifying when object and flattened field types are acceptable, and when explicit leaf fields must be used instead. See #2594.

2. Address existing beta entity fields

Since entity.attributes, entity.behavior, entity.lifecycle, and entity.metrics are all still beta, the subfields that integrations need (e.g., entity.attributes.mfa_enabled, entity.lifecycle.first_seen, entity.attributes.granted_permissions) should be proposed as explicit leaf fields through the RFC process. This converts the parent objects from empty containers into structured field sets with defined children, types, and descriptions. This is the same pattern used successfully throughout the rest of ECS (e.g., user.group with children .id, .name, .domain).

The stated purpose of these fields is "advanced searching and correlation of normalized values across different providers/sources." That purpose is only achievable when the field names and types are coordinated. An open object with no defined children provides a namespace but no coordination. It shifts the type-safety problem from "undefined" to "defined independently by each integration," which is worse because it creates the appearance of compatibility without the substance.

entity.raw should remain as-is. It is explicitly documented as source-specific opaque data and is the correct use of an open object.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions