feat(eap): Trim spans with new trimming processor#5616
feat(eap): Trim spans with new trimming processor#5616loewenheim wants to merge 20 commits intomasterfrom
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| let trimming_root_state = { | ||
| let mut attrs = FieldAttrs::default(); | ||
| if let Some(span_config) = ctx.project_info.config().trimming.span { | ||
| attrs = attrs.max_bytes(span_config.max_size as usize); | ||
| } | ||
| ProcessingState::new_root(Some(Cow::Owned(attrs)), []) | ||
| }; |
There was a problem hiding this comment.
I'd really like to hide this in a utility or builder. I did something similar for the time normalization in processing/utils/normalize, could hide it there to create the processing state.
Or maybe a builder/better function to run the trimming alltogether.
Nothing important though, can also do that in a follow-up
There was a problem hiding this comment.
I added a TODO about this, my proposed PR for improving this is #5618.
There was a problem hiding this comment.
Better! Just awkward formatting from rust formatter.
loewenheim
force-pushed
the
sebastian/trim-eap-items
branch
from
71dd15b to
b7f3288
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| return Ok(()); | ||
| } | ||
|
|
||
| // This counts the lengths of all attribute keys regardless of whether |
There was a problem hiding this comment.
Object trimming drops last remarked key via split_off
Medium Severity
The process_object split logic sets i = key.as_str() before the match and never advances it afterward, so split_off(&i) removes the last key that received WithRemark. In contrast, process_array and process_attributes both increment i after the match, correctly preserving remarked entries. When all remaining keys fit within removed_key_byte_budget, the final remarked key is silently dropped. This code was previously gated behind #[allow(unused)] and is now activated for span processing.
Additional Locations (1)
2 participants
This puts the new trimming processor into practice by enabling it for spans. It also introduces some necessary configuration:
max_removed_attribute_key_bytes(default 10KiB) that controls up to which limit Relay will store invalid/removed attribute keys.trimmingsection in the project config which (for now) allows configuring the maximum bytes size above which spans should be trimmed. If this is not set, spans will not be trimmed.ref: INGEST-732.