Skip to content

Add OTEL Semantic Conventions support#549

Open
Mpdreamz wants to merge 4 commits intomainfrom
feature/otel-mappings
Open

Add OTEL Semantic Conventions support#549
Mpdreamz wants to merge 4 commits intomainfrom
feature/otel-mappings

Conversation

@Mpdreamz
Copy link
Copy Markdown
Member

@Mpdreamz Mpdreamz commented Feb 18, 2026

Summary

  • Add OTel semantic convention mappings with bidirectional ECS↔OTel support. Parse otel mappings from the ECS 9.x YAML spec, generate bidirectional mapping tables, and expose AssignOTelField, SemConv constants, and OTelMappings dictionaries. OTel attributes are stored in an Attributes passthrough dictionary and serialized under an attributes object alongside standard ECS fields.
  • Consolidate Labels and Metadata into the single Attributes passthrough. Both properties are now [Obsolete]; serialization merges all three into a single "attributes" JSON key (Attributes wins over Metadata wins over Labels). Deserialization reads "labels", "metadata", and "attributes" JSON keys all into Attributes. The ES base component gains an attributes: passthrough mapping.

OTEL support

Parse otel mappings from the ECS 9.x YAML spec, generate bidirectional
mapping tables (ECS <-> OTel), and support reading/writing OTel attributes
alongside standard ECS fields.

ECS fields are always serialized as ECS fields. OTel semantic convention
attributes are stored in the Attributes passthrough dictionary and
serialized under an attributes object.

Setting OTel attributes

Use SemConv constants with AssignOTelField to set an OTel attribute
that is stored in Attributes and simultaneously mapped to its ECS
property:

var doc = new EcsDocument();
doc.AssignOTelField(SemConv.ExceptionMessage, "connection refused");

// doc.Error.Message == "connection refused"
// doc.Attributes["exception.message"] == "connection refused"

Or set attributes directly on the dictionary:

var doc = new EcsDocument
{
    Message = "Request completed",
    Attributes = new MetadataDictionary
    {
        [SemConv.ExceptionMessage] = "connection refused",
        ["custom.field"] = "custom value",
    }
};

Serialization

The standard serializer writes ECS fields as ECS fields and the
Attributes dictionary as the attributes object:

var json = doc.Serialize();
// {
//   "@timestamp": "...",
//   "message": "Request completed",
//   "ecs.version": "...",
//   "attributes": {
//     "exception.message": "connection refused",
//     "custom.field": "custom value"
//   }
// }

Reading OTel format

EcsDocument.Deserialize() handles JSON that includes an attributes
object. OTel attribute names are automatically mapped to their ECS
equivalents:

var json = """{"attributes":{"exception.message":"timeout"}}""";
var doc = EcsDocument.Deserialize(json);

// doc.Error.Message == "timeout"
// doc.Attributes["exception.message"] == "timeout"

OTEL_RESOURCE_ATTRIBUTES integration

CreateNewWithDefaults processes OTEL_RESOURCE_ATTRIBUTES entries
through the mapping table. Equivalent mappings (e.g.
deployment.environment.name) set both the ECS field and Attributes.
Unknown attributes are stored in Attributes only.

…pport

Parse `otel` mappings from the ECS 9.x YAML spec, generate bidirectional
mapping tables (ECS <-> OTel), and support reading/writing OTel attributes
alongside standard ECS fields.

ECS fields are always serialized as ECS fields. OTel semantic convention
attributes are stored in the `Attributes` passthrough dictionary and
serialized under an `attributes` object.

## Setting OTel attributes

Use `SemConv` constants with `AssignOTelField` to set an OTel attribute
that is stored in `Attributes` and simultaneously mapped to its ECS
property:

```csharp
var doc = new EcsDocument();
doc.AssignOTelField(SemConv.ExceptionMessage, "connection refused");

// doc.Error.Message == "connection refused"
// doc.Attributes["exception.message"] == "connection refused"
```

Or set attributes directly on the dictionary:

```csharp
var doc = new EcsDocument
{
    Message = "Request completed",
    Attributes = new MetadataDictionary
    {
        [SemConv.ExceptionMessage] = "connection refused",
        ["custom.field"] = "custom value",
    }
};
```

## Serialization

The standard serializer writes ECS fields as ECS fields and the
`Attributes` dictionary as the `attributes` object:

```csharp
var json = doc.Serialize();
// {
//   "@timestamp": "...",
//   "message": "Request completed",
//   "ecs.version": "...",
//   "attributes": {
//     "exception.message": "connection refused",
//     "custom.field": "custom value"
//   }
// }
```

## Reading OTel format

`EcsDocument.Deserialize()` handles JSON that includes an `attributes`
object. OTel attribute names are automatically mapped to their ECS
equivalents:

```csharp
var json = """{"attributes":{"exception.message":"timeout"}}""";
var doc = EcsDocument.Deserialize(json);

// doc.Error.Message == "timeout"
// doc.Attributes["exception.message"] == "timeout"
```

## OTEL_RESOURCE_ATTRIBUTES integration

`CreateNewWithDefaults` processes `OTEL_RESOURCE_ATTRIBUTES` entries
through the mapping table. Equivalent mappings (e.g.
`deployment.environment.name`) set both the ECS field and `Attributes`.
Unknown attributes are stored in `Attributes` only.
…assthrough

Labels and Metadata are now [Obsolete] and route to Attributes. Serialization
merges all three into a single "attributes" JSON key (Attributes wins over
Metadata wins over Labels). Deserialization reads "labels", "metadata", and
"attributes" JSON keys all into the Attributes property. The ES base component
now includes an "attributes" passthrough mapping.
@github-actions
Copy link
Copy Markdown

🤖 GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@Mpdreamz Mpdreamz changed the title feature/otel mappings Add OTEL Semantic Conventions support Feb 18, 2026
Elasticsearch 9 requires a non-negative priority on passthrough-type
mappings. The ECS 9.3.0 base component template defined attributes as
passthrough without a priority, causing a mapper_parsing_exception when
bootstrapping the ecs_9.3.0_base component template.

Fix the generator to inject priority: 10 when reading passthrough-type
component mappings that lack one, and regenerate.
@Mpdreamz Mpdreamz enabled auto-merge (squash) February 24, 2026 20:03
@Mpdreamz Mpdreamz disabled auto-merge February 24, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant