[WP] Serialize tracer metadata struct in process#48487
Open
[WP] Serialize tracer metadata struct in process#48487
Conversation
Gui774ume
added
changelog/no-changelog
Contributor
Go Package Import DifferencesBaseline: 605a112
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b7bdb225ed
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Gui774ume
force-pushed
the
will/add-missing-tracermetadata-tags
branch
from
ff9ff94 to
5a02aa1
Compare
Gui774ume
requested review from
charan-manjunath and
nathan-b
and removed request for
a team
Gui774ume
force-pushed
the
will/serialize-tracermetadata-in-process
branch
from
b7bdb22 to
c4b9d82
Compare
Gui774ume
changed the title
Gui774ume
changed the title
Gui774ume
force-pushed
the
will/serialize-tracermetadata-in-process
branch
from
c4b9d82 to
837298a
Compare
Gui774ume
requested review from
a team
and removed request for
a team,
charan-manjunath and
nathan-b
Gui774ume
changed the title
Gui774ume
force-pushed
the
will/serialize-tracermetadata-in-process
branch
from
837298a to
ff4f74c
Compare
Gui774ume
requested review from
a team and
Janaeq
and removed request for
a team
Gui774ume
force-pushed
the
will/add-missing-tracermetadata-tags
branch
from
5a02aa1 to
ed4e5cf
Compare
Gui774ume
force-pushed
the
will/serialize-tracermetadata-in-process
branch
2 times, most recently
from
8ae0132 to
ef90f46
Compare
Gui774ume
force-pushed
the
will/add-missing-tracermetadata-tags
branch
from
ed4e5cf to
13310e3
Compare
Gui774ume
force-pushed
the
will/serialize-tracermetadata-in-process
branch
from
ef90f46 to
d3df440
Compare
Contributor
Files inventory check summaryFile checks results against ancestor 605a1124: Results for datadog-agent_7.79.0~devel.git.223.7f30aed.pipeline.104786624-1_amd64.deb:No change detected |
Contributor
Static quality checks✅ Please find below the results from static quality gates Successful checksInfo
16 successful checks with minimal change (< 2 KiB)
On-wire sizes (compressed)
|
Regression DetectorRegression Detector ResultsMetrics dashboard Baseline: 605a112 Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | docker_containers_cpu | % cpu utilization | +0.92 | [-2.06, +3.90] | 1 | Logs |
Fine details of change detection per experiment
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | docker_containers_cpu | % cpu utilization | +0.92 | [-2.06, +3.90] | 1 | Logs |
| ➖ | ddot_metrics | memory utilization | +0.45 | [+0.27, +0.62] | 1 | Logs |
| ➖ | ddot_metrics_sum_delta | memory utilization | +0.21 | [+0.05, +0.37] | 1 | Logs |
| ➖ | file_tree | memory utilization | +0.19 | [+0.13, +0.24] | 1 | Logs |
| ➖ | ddot_logs | memory utilization | +0.17 | [+0.11, +0.23] | 1 | Logs |
| ➖ | otlp_ingest_metrics | memory utilization | +0.11 | [-0.05, +0.27] | 1 | Logs |
| ➖ | ddot_metrics_sum_cumulativetodelta_exporter | memory utilization | +0.08 | [-0.14, +0.31] | 1 | Logs |
| ➖ | ddot_metrics_sum_cumulative | memory utilization | +0.05 | [-0.09, +0.19] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api_v3 | ingress throughput | +0.01 | [-0.18, +0.21] | 1 | Logs |
| ➖ | file_to_blackhole_100ms_latency | egress throughput | +0.01 | [-0.08, +0.10] | 1 | Logs |
| ➖ | uds_dogstatsd_to_api | ingress throughput | -0.01 | [-0.20, +0.19] | 1 | Logs |
| ➖ | tcp_dd_logs_filter_exclude | ingress throughput | -0.01 | [-0.11, +0.09] | 1 | Logs |
| ➖ | quality_gate_logs | % cpu utilization | -0.01 | [-1.62, +1.60] | 1 | Logs bounds checks dashboard |
| ➖ | file_to_blackhole_0ms_latency | egress throughput | -0.02 | [-0.51, +0.47] | 1 | Logs |
| ➖ | docker_containers_memory | memory utilization | -0.03 | [-0.13, +0.06] | 1 | Logs |
| ➖ | file_to_blackhole_1000ms_latency | egress throughput | -0.04 | [-0.48, +0.40] | 1 | Logs |
| ➖ | file_to_blackhole_500ms_latency | egress throughput | -0.06 | [-0.45, +0.33] | 1 | Logs |
| ➖ | quality_gate_idle | memory utilization | -0.15 | [-0.20, -0.10] | 1 | Logs bounds checks dashboard |
| ➖ | uds_dogstatsd_20mb_12k_contexts_20_senders | memory utilization | -0.17 | [-0.23, -0.12] | 1 | Logs |
| ➖ | quality_gate_idle_all_features | memory utilization | -0.18 | [-0.22, -0.14] | 1 | Logs bounds checks dashboard |
| ➖ | otlp_ingest_logs | memory utilization | -0.26 | [-0.36, -0.17] | 1 | Logs |
| ➖ | quality_gate_metrics_logs | memory utilization | -0.30 | [-0.53, -0.06] | 1 | Logs bounds checks dashboard |
| ➖ | tcp_syslog_to_blackhole | ingress throughput | -0.75 | [-0.88, -0.62] | 1 | Logs |
Bounds Checks: ❌ Failed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | docker_containers_cpu | simple_check_run | 10/10 | 691 ≥ 26 | |
| ✅ | docker_containers_memory | memory_usage | 10/10 | 272.26MiB ≤ 370MiB | |
| ✅ | docker_containers_memory | simple_check_run | 10/10 | 717 ≥ 26 | |
| ✅ | file_to_blackhole_0ms_latency | memory_usage | 10/10 | 0.19GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_0ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_1000ms_latency | memory_usage | 10/10 | 0.23GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_1000ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_100ms_latency | memory_usage | 10/10 | 0.20GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_100ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | file_to_blackhole_500ms_latency | memory_usage | 10/10 | 0.21GiB ≤ 1.20GiB | |
| ✅ | file_to_blackhole_500ms_latency | missed_bytes | 10/10 | 0B = 0B | |
| ✅ | quality_gate_idle | intake_connections | 10/10 | 3 = 3 | bounds checks dashboard |
| ❌ | quality_gate_idle | memory_usage | 9/10 | 175.91MiB > 175MiB | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | intake_connections | 10/10 | 3 = 3 | bounds checks dashboard |
| ✅ | quality_gate_idle_all_features | memory_usage | 10/10 | 497.69MiB ≤ 550MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_logs | memory_usage | 10/10 | 205.68MiB ≤ 220MiB | bounds checks dashboard |
| ✅ | quality_gate_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | cpu_usage | 10/10 | 351.81 ≤ 2000 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | intake_connections | 10/10 | 4 ≤ 6 | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | memory_usage | 10/10 | 400.48MiB ≤ 475MiB | bounds checks dashboard |
| ✅ | quality_gate_metrics_logs | missed_bytes | 10/10 | 0B = 0B | bounds checks dashboard |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
CI Pass/Fail Decision
❌ Failed. Some Quality Gates were violated.
- quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_metrics_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check missed_bytes: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
- quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
- quality_gate_idle, bounds check memory_usage: 9/10 replicas passed. Failed 1 which is > 0. Gate FAILED.
- quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
Gui774ume
force-pushed
the
will/add-missing-tracermetadata-tags
branch
from
13310e3 to
a3b1833
Compare
Gui774ume
force-pushed
the
will/serialize-tracermetadata-in-process
branch
from
d3df440 to
7f30aed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR replaces the
TracerTags []stringfield on theProcessstruct with the fullTracerMetadatastruct. This allows security events and CWS serializers to carry the complete tracer metadata (language, version, hostname, service name/env/version, process tags, etc.) instead of a pre-flattened list of tag strings.Key changes:
Process.TracerTags []stringreplaced byProcess.TracerMetadata tracermetadata.TracerMetadatain the secl model (Unix + Windows).TracerMetadatastruct as"tracer"JSON instead of a flatmap[string]string.TracerMetadata.Tags()directly instead of consuming pre-built tag slices.TracerMetadataon cache entries instead of converting to tags at resolution time.Motivation
Storing only flattened tags (
[]string) on theProcessstruct discards structured tracer metadata (schema version, runtime ID, hostname, logs collection status, etc.) that downstream consumers need. By keeping the fullTracerMetadatastruct through the pipeline, we:Tags()without re-parsing.Describe how you validated your changes
pkg/network/events/,pkg/security/tests/, andpkg/discovery/tracermetadata/updated and passing.Additional Notes
docs/cloud-workload-security/) has been updated to reflect the newtracerfield structure (full struct instead ofmap[string]string)."tracer"field in CWS events — it changes from{"key": "value", ...}to the fullTracerMetadataJSON representation with named fields (tracer_language,service_name, etc.).