feat: add gen_ai.input.messages original length attribute

[obostjancic]

Adds sdk_meta attribute that is used to indicate how many messages sdk discarded before sending the span.

More context: https://www.notion.so/sentry/Gen-AI-Limits-SDK-changes-2ee8b10e4b5d80ebbefcfe70de7399da?source=copy_link#2ee8b10e4b5d80ed941ec2cec60fdb67

[github-actions]

Semver Impact of This PR

🟡 Minor (new features)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).

---

New Features ✨

Attributes

• Add sentry.profiler_id attribute by Lms24 in #242
• Add attribute gen_ai.response.time_to_first_token by constantinius in #227
• Add sentry.status_code attribute by mjq in #223
• Backfill db.system attribute by mjq in #224
• Add sentry.category attribute by mjq in #218
• Add sentry.normalized_db_query.hash by Ahmed-Labs in #200

Other

• (ai) Add input tokens cache write by shellmayr in #217
• (gen_ai) Add new Gen AI attributes by obostjancic in #221

• Add gen_ai.input.messages original length attribute by obostjancic in #241

• Add page listing all attributes by Lms24 in #207

Bug Fixes 🐛

Attributes

• Migrating several numeric fields to pii=maybe by constantinius in #228
• Improve gen_ai.operation.name description by obostjancic in #225

Other

• (db) Clarify db.query.text and db.query.summary attributes by Lms24 in #208

Internal Changes 🔧

Attributes

• Update sentry.span.source with known values by Lms24 in #214
• Add legacy attribute definitions for transaction spans by Ahmed-Labs in #212
• Normalize deprecated db attribute names by Ahmed-Labs in #199

Deps Dev

• Bump tar from 7.5.4 to 7.5.7 by dependabot in #239
• Bump tar from 7.5.3 to 7.5.4 by dependabot in #226
• Bump tar from 7.4.3 to 7.5.3 by dependabot in #220

Repo

• Add Claude Code settings with basic permissions by philipphofmann in #222
• Unstale yarn lock by Lms24 in #215

Other

• (ai) Remove gen_ai_usage_total_cost attribute by vgrozdanic in #206
• (lint) Don't lint Claude's config files by mjq in #219
• (release) Switch from action-prepare-release to Craft by BYK in #216
• Add statuses permission to Changelog Preview by lcian in #232

Other

• Add more CODEOWNERS by cleptric in #240
• Add @getsentry/ingest to CODEOWNERS by cleptric in #236
• Update CODEOWNERS to change global ownership by cleptric in #230

---

_{🤖 This preview updates automatically when you update the PR.}

[Lms24]

(Feel free to disregard, I basically just came across this and wanted to share a thought)

I don't wanna stir up too much bike shedding here but did we settle on this name yet?

As in: is sentry.sdk_meta the "namespace" we want to use for (I guess) annotating other, non-sentry, attributes? I'd like us to just take a moment here and think this through because this name is very long and nested. I don't have enough context around AI monitoring to popose a well thought through alternative but at least a bit shorter and less nested:

sentry.meta.gen_ai_input_messages.original_length

My primary concern here is the product showing nested keys in nested levels. So maybe we remove the . from the original attribute name to also indicate that this attribute describes the very attribute in its name?

(Nesting example)

[obostjancic]

Hey, yeah I share a lot of the same concerns but we agreed on this already and sdks actually already implement it. You are right, this is very nested but we wanted it to relate to the original attribute (in this case it is already non-short gen_ai.input.messages) :/

Product nesting looks ok though:

[Dav1dde]

I think ideally we/SDKs secure a namesapce for such information and possibly their own format/spec, then we hide the entire namespace in the UI by default and render it nicely.

[Lms24]

Based on #228, I believe this should be pii: 'maybe', correct?

[obostjancic]

yep, fixed!