feat(autofix): Add analytics events for autofix phase start and completion#112098
Merged
feat(autofix): Add analytics events for autofix phase start and completion#112098
Conversation
github-actions
bot
added
the
Scope: Backend
Contributor
Backend Test FailuresFailures on
|
trevor-e
approved these changes
Apr 2, 2026
Member
trevor-e
left a comment
trevor-e
left a comment
There was a problem hiding this comment.
LGTM but might want to wait for Tony's review.
Zylphrex
approved these changes
Apr 2, 2026
chromy
force-pushed
the
chromy/feat/autofix-phase-analytics
branch
from
c3ac170 to
9a5f34f
Compare
chromy
force-pushed
the
chromy/feat/autofix-phase-analytics
branch
from
9a5f34f to
f431887
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Code changes completed event fires spuriously on PR creation
- Modified analytics event firing logic to only fire AiAutofixPrCreatedCompletedEvent when webhook_action_type is PR_CREATED, preventing duplicate AiAutofixCodeChangesCompletedEvent from inflating analytics counts.
Or push these changes by commenting:
@cursor push 38e57f1b9a
Preview (38e57f1b9a)
diff --git a/src/sentry/seer/autofix/on_completion_hook.py b/src/sentry/seer/autofix/on_completion_hook.py
--- a/src/sentry/seer/autofix/on_completion_hook.py
+++ b/src/sentry/seer/autofix/on_completion_hook.py
@@ -237,16 +237,6 @@
metrics.incr(
"autofix.explorer.complete", tags={"step": current_step.value, "referrer": referrer}
)
- completed_event_cls = STEP_CONFIGS[current_step].completed_event
- if completed_event_cls is not None and referrer is not None:
- analytics.record(
- completed_event_cls(
- organization_id=organization.id,
- project_id=group.project_id,
- group_id=group.id,
- referrer=referrer,
- )
- )
if webhook_action_type == SeerActionType.PR_CREATED and referrer is not None:
analytics.record(
AiAutofixPrCreatedCompletedEvent(
@@ -256,6 +246,17 @@
referrer=referrer,
)
)
+ elif webhook_action_type != SeerActionType.PR_CREATED:
+ completed_event_cls = STEP_CONFIGS[current_step].completed_event
+ if completed_event_cls is not None and referrer is not None:
+ analytics.record(
+ completed_event_cls(
+ organization_id=organization.id,
+ project_id=group.project_id,
+ group_id=group.id,
+ referrer=referrer,
+ )
+ )
@classmethod
def _maybe_trigger_supergroups_embedding(This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.
Zylphrex
reviewed
Apr 2, 2026
Comment on lines
+42
to
+59
| @analytics.eventclass("ai.autofix.impact_assessment.started") | ||
| class AiAutofixImpactAssessmentStartedEvent(AiAutofixPhaseEvent): | ||
| pass | ||
|
|
||
|
|
||
| @analytics.eventclass("ai.autofix.impact_assessment.completed") | ||
| class AiAutofixImpactAssessmentCompletedEvent(AiAutofixPhaseEvent): | ||
| pass | ||
|
|
||
|
|
||
| @analytics.eventclass("ai.autofix.triage.started") | ||
| class AiAutofixTriageStartedEvent(AiAutofixPhaseEvent): | ||
| pass | ||
|
|
||
|
|
||
| @analytics.eventclass("ai.autofix.triage.completed") | ||
| class AiAutofixTriageCompletedEvent(AiAutofixPhaseEvent): | ||
| pass |
Member
There was a problem hiding this comment.
These shouldn't be in use in production so I would expect they never fire
chromy
force-pushed
the
chromy/feat/autofix-phase-analytics
branch
from
f431887 to
6839918
Compare
chromy
force-pushed
the
chromy/feat/autofix-phase-analytics
branch
from
6839918 to
0e0a05a
Compare
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0e0a05a. Configure here.
…etion Record per-phase analytics events when each autofix pipeline phase (root_cause, solution, code_changes) starts and completes. Each phase gets its own event class so they can be queried independently. Events include group_id, referrer, organization_id, and project_id. Started events fire in trigger_autofix_explorer, completed events fire in the on_completion_hook webhook handler. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Agent transcript: https://claudescope.sentry.dev/share/botteEgndhtvdoXxpJ2frzQPLVabdvt8jNcgx9NaA3k
chromy
force-pushed
the
chromy/feat/autofix-phase-analytics
branch
from
0e0a05a to
f98a76a
Compare
george-sentry
pushed a commit
that referenced
this pull request
Apr 9, 2026
…etion (#112098) Add per-phase analytics events for the autofix pipeline so we can track when each phase (root cause, solution, code changes, impact assessment, triage) starts and completes. Each phase gets its own event class (`ai.autofix.root_cause.started`, `ai.autofix.root_cause.completed`, etc.) plus a dedicated `ai.autofix.pr_created.completed` event. All events include `group_id`, `referrer`, `organization_id`, and `project_id`. Started events are recorded in `trigger_autofix_explorer` when a phase kicks off. Completed events are recorded in `AutofixOnCompletionHook._send_step_webhook` when the on-completion hook fires.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add per-phase analytics events for the autofix pipeline so we can track
when each phase (root cause, solution, code changes, impact assessment,
triage) starts and completes.
Each phase gets its own event class (
ai.autofix.root_cause.started,ai.autofix.root_cause.completed, etc.) plus a dedicatedai.autofix.pr_created.completedevent. All events includegroup_id,referrer,organization_id, andproject_id.Started events are recorded in
trigger_autofix_explorerwhen a phase kicksoff. Completed events are recorded in
AutofixOnCompletionHook._send_step_webhookwhen the on-completion hook fires.
Also fixes double-counting of
ai.autofix.code_changes.completed: whencurrent_stepisCODE_CHANGESandrepo_pr_statesis populated, thehook was firing both
AiAutofixCodeChangesCompletedEvent(fromSTEP_CONFIGS) andAiAutofixPrCreatedCompletedEvent. Sincepush_changespasseson_completion_hookin its payload, the hook firesagain after PR creation — meaning the code-changes event was recorded both
when coding finished and when the PR was created. The fix gates the
STEP_CONFIGScompleted event onwebhook_action_type != PR_CREATEDsoonly the PR-created event fires in that path.