Skip to content

feat(issue-detection): Add org-level scheduling for AI issue detection#113060

Draft
roggenkemper wants to merge 6 commits intomasterfrom
roggenkemper/feat/budget-paced-detection-scheduling
Draft

feat(issue-detection): Add org-level scheduling for AI issue detection#113060
roggenkemper wants to merge 6 commits intomasterfrom
roggenkemper/feat/budget-paced-detection-scheduling

Conversation

@roggenkemper
Copy link
Copy Markdown
Member

Replace the hourly per-project dispatch with a 15-minute bucketed dispatcher that spreads org dispatches across time slots using hashing. Each org is assigned to a deterministic slot and dispatched once per full cycle.

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 15, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Backend Test Failures

Failures on 44421d5 in this run:

tests/sentry/tasks/test_llm_issue_detection.py::TestRunLLMIssueDetection::test_dispatches_orgs_in_current_slotlog
[gw0] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
src/sentry/features/manager.py:217: in _get_feature_class
    return self._feature_registry[name]
E   KeyError: 'organizations:ai-issue-detection'

During handling of the above exception, another exception occurred:
src/sentry/testutils/helpers/features.py:72: in features_override
    feature = features.get(name, None)
src/sentry/features/manager.py:227: in get
    cls = self._get_feature_class(name)
src/sentry/features/manager.py:219: in _get_feature_class
    raise FeatureNotRegistered(name)
E   sentry.features.exceptions.FeatureNotRegistered: The "organizations:ai-issue-detection" feature has not been registered. Ensure that a feature has been added to sentry.features.default_manager

During handling of the above exception, another exception occurred:
tests/sentry/tasks/test_llm_issue_detection.py:70: in test_dispatches_orgs_in_current_slot
    run_llm_issue_detection()
.venv/lib/python3.13/site-packages/taskbroker_client/task.py:92: in __call__
    return self._func(*args, **kwargs)
src/sentry/tasks/llm_issue_detection/detection.py:292: in run_llm_issue_detection
    if not features.has("organizations:ai-issue-detection", org):
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1167: in __call__
    return self._mock_call(*args, **kwargs)
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1171: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1232: in _execute_mock_call
    result = effect(*args, **kwargs)
src/sentry/testutils/helpers/features.py:74: in features_override
    raise ValueError("Unregistered feature flag: %s", repr(name))
E   ValueError: ('Unregistered feature flag: %s', "'organizations:ai-issue-detection'")
tests/sentry/tasks/test_llm_issue_detection.py::TestRunLLMIssueDetection::test_skips_orgs_with_hidden_ailog
[gw0] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
src/sentry/features/manager.py:217: in _get_feature_class
    return self._feature_registry[name]
E   KeyError: 'organizations:ai-issue-detection'

During handling of the above exception, another exception occurred:
src/sentry/testutils/helpers/features.py:72: in features_override
    feature = features.get(name, None)
src/sentry/features/manager.py:227: in get
    cls = self._get_feature_class(name)
src/sentry/features/manager.py:219: in _get_feature_class
    raise FeatureNotRegistered(name)
E   sentry.features.exceptions.FeatureNotRegistered: The "organizations:ai-issue-detection" feature has not been registered. Ensure that a feature has been added to sentry.features.default_manager

During handling of the above exception, another exception occurred:
tests/sentry/tasks/test_llm_issue_detection.py:121: in test_skips_orgs_with_hidden_ai
    run_llm_issue_detection()
.venv/lib/python3.13/site-packages/taskbroker_client/task.py:92: in __call__
    return self._func(*args, **kwargs)
src/sentry/tasks/llm_issue_detection/detection.py:292: in run_llm_issue_detection
    if not features.has("organizations:ai-issue-detection", org):
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1167: in __call__
    return self._mock_call(*args, **kwargs)
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1171: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1232: in _execute_mock_call
    result = effect(*args, **kwargs)
src/sentry/testutils/helpers/features.py:74: in features_override
    raise ValueError("Unregistered feature flag: %s", repr(name))
E   ValueError: ('Unregistered feature flag: %s', "'organizations:ai-issue-detection'")

Comment thread src/sentry/tasks/llm_issue_detection/detection.py
roggenkemper and others added 2 commits April 15, 2026 15:56
…ssue detection

Replace the hourly per-project dispatch with a 15-minute bucketed dispatcher
that spreads org dispatches across time slots using md5 hashing. Each org is
assigned to a deterministic slot and dispatched once per full cycle.

- Add `organizations:ai-issue-detection` feature flag (FlagPole)
- Rewrite dispatcher to iterate active orgs with RangeQuerySetWrapper
- Add `detect_llm_issues_for_org` task: picks random project, sends 1 trace
- Remove legacy `detect_llm_issues_for_project` and project allowlist path
- Change Celery Beat from hourly to every 15 minutes
- NUM_DISPATCH_SLOTS=10 (~2.5h cycle), increase toward 67 as org count grows

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@roggenkemper roggenkemper force-pushed the roggenkemper/feat/budget-paced-detection-scheduling branch from 173d79f to b12c4d1 Compare April 15, 2026 19:58
roggenkemper and others added 3 commits April 15, 2026 15:59
… detection

Respect the user's project-level setting to avoid wasting Snuba queries
and Seer calls when AI detection is disabled for the selected project.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…dition

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/sentry/tasks/llm_issue_detection/detection.py
@roggenkemper
Copy link
Copy Markdown
Member Author

bugbot run

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5f1e8ae. Configure here.

Comment thread src/sentry/conf/server.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant