feat(seer): Add Night Shift nightly autofix cron scaffolding#112429
feat(seer): Add Night Shift nightly autofix cron scaffolding#112429
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Function name contains spurious
_d7suffix- Renamed function from _get_eligible_org_ids_from_batch_d7 to _get_eligible_org_ids_from_batch, removing the spurious suffix.
- ✅ Fixed: Unused module-level constant
NIGHT_SHIFT_AUTO_RUN_SOURCE- Removed the unused NIGHT_SHIFT_AUTO_RUN_SOURCE constant and its associated SeerAutomationSource import.
Or push these changes by commenting:
@cursor push 6aeb4ee3ef
Preview (6aeb4ee3ef)
diff --git a/src/sentry/tasks/seer/night_shift.py b/src/sentry/tasks/seer/night_shift.py
--- a/src/sentry/tasks/seer/night_shift.py
+++ b/src/sentry/tasks/seer/night_shift.py
@@ -8,7 +8,6 @@
from sentry import features, options
from sentry.models.organization import Organization, OrganizationStatus
-from sentry.seer.autofix.constants import SeerAutomationSource
from sentry.tasks.base import instrumented_task
from sentry.taskworker.namespaces import seer_tasks
from sentry.utils.iterators import chunked
@@ -24,9 +23,7 @@
"organizations:gen-ai-features",
]
-NIGHT_SHIFT_AUTO_RUN_SOURCE = SeerAutomationSource.NIGHT_SHIFT.value
-
@instrumented_task(
name="sentry.tasks.seer.night_shift.schedule_night_shift",
namespace=seer_tasks,
@@ -50,7 +47,7 @@
),
100,
):
- eligible_ids = _get_eligible_org_ids_from_batch_d7(org_batch)
+ eligible_ids = _get_eligible_org_ids_from_batch(org_batch)
org_map = {org.id: org for org in org_batch}
for org_id in eligible_ids:
@@ -102,7 +99,7 @@
)
-def _get_eligible_org_ids_from_batch_d7(
+def _get_eligible_org_ids_from_batch(
orgs: Sequence[Organization],
) -> list[int]:
"""This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e65687b. Configure here.
Backend Test FailuresFailures on
|
Add the scaffolding for a nightly cron job that fans out per-org to trigger Seer Autofix on top issues. This decouples automation from the hot post_process pipeline, enabling better issue selection and serving lower-volume orgs. Includes: - Scheduler task with batched feature flag checks and jitter - Stub per-org worker task (logic TBD) - Feature flag (organizations:seer-night-shift) and options - NIGHT_SHIFT referrer and automation source Co-Authored-By: Claude <noreply@anthropic.com>
Remove spurious _d7 suffix from function name and remove unused NIGHT_SHIFT_AUTO_RUN_SOURCE constant (will be added when worker logic is implemented). Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2447b23 to
9308041
Compare
| for org_batch in chunked( | ||
| RangeQuerySetWrapper[Organization]( | ||
| Organization.objects.filter(status=OrganizationStatus.ACTIVE), | ||
| step=1000, | ||
| ), | ||
| 100, | ||
| ): |
There was a problem hiding this comment.
Not sure if this would be useful for you since you're scheduling all of these at the same time, but I recently added a generic CursoredScheduler that might do what you want. Instead of running all the scheduled items at once, it stripes them over a period of time. This might be helpful for you so that you're not hitting Seer with a bunch of requests all at once.
https://github.com/getsentry/sentry/blob/987d0433540d52624da4390b353961c8ac0749bb/src/sentry/integrations/source_code_management/sync_repos.py#L271-L288 is an example of it being used.
Probably it'd need a few changes to let it filter tasks before scheduling, but I'd be happy enough to include them if it's helpful for you.
There was a problem hiding this comment.
Something like CursoredScheduler seems compelling. If I understand CursoredScheduler correctly it ends up exposing an API like "Run foo() once per org per 24h" (where org / 24h etc can be configured). That's pretty close to the long term future of nightshift probably?
In the short term of nightshift (and more generally for experimental things) - it may be worth splitting the two parts:
- One off running a function over all orgs over some period.
- Scheduling when 1. happens.
In particular if you a) discover some bug in the recurring task and want to restart it/rerun it/stop it or b) want to run some one time job over all orgs. That might diverge a bit too far from the current design of CursoredScheduler though.
There was a problem hiding this comment.
Yea CursoredScheduler looks interesting but the scheduling seems restrictive for our use-case right now since the Seer rollout is still fairly small. Will keep an eye on this though like Hector mentioned. I do think our team could actually use it for the explorer index building job that was recently added.
There was a problem hiding this comment.
Sounds good. Feel free to ping me if you want to make any changes to it to support your needs. At the moment it's a fairly simple design, and if the number of items in the queue changes over time then it can result in the scheduler starting the next cycle slightly earlier/later than it originally started. I think in a lot of cases that's not too important, since we're usually saying "run this about once a day"
chromy
left a comment
There was a problem hiding this comment.
lgtm % some minor questions
| for org_batch in chunked( | ||
| RangeQuerySetWrapper[Organization]( | ||
| Organization.objects.filter(status=OrganizationStatus.ACTIVE), | ||
| step=1000, | ||
| ), | ||
| 100, | ||
| ): |
There was a problem hiding this comment.
Something like CursoredScheduler seems compelling. If I understand CursoredScheduler correctly it ends up exposing an API like "Run foo() once per org per 24h" (where org / 24h etc can be configured). That's pretty close to the long term future of nightshift probably?
In the short term of nightshift (and more generally for experimental things) - it may be worth splitting the two parts:
- One off running a function over all orgs over some period.
- Scheduling when 1. happens.
In particular if you a) discover some bug in the recurring task and want to restart it/rerun it/stop it or b) want to run some one time job over all orgs. That might diverge a bit too far from the current design of CursoredScheduler though.
Instead of silently falling back to per-org feature checks, raise an error so we notice if the batch handler is broken. Co-Authored-By: Claude <noreply@anthropic.com>
Keep Organization objects throughout the filtering pipeline instead of converting to IDs and back. Also shrinks the list each iteration so subsequent batch_has calls check fewer orgs. Co-Authored-By: Claude <noreply@anthropic.com>
Add the scaffolding for a nightly cron job ("Night Shift") that fans out
per-org to trigger Seer Autofix on top issues. This decouples automation
from the hot `post_process` pipeline, enabling better issue selection
and serving lower-volume orgs.
The scheduler iterates active orgs using batched feature flag checks
(`batch_has_for_organizations`) to avoid N+1, applies deterministic
jitter to spread load, and dispatches per-org worker tasks. The worker
task is currently a stub that just logs — the issue selection and
autofix triggering logic will be added in a follow-up once we've decided
on the approach.
**What's included:**
- `schedule_night_shift()` — cron scheduler task (daily at 10:00 AM UTC
/ ~2-3 AM PT)
- `run_night_shift_for_org()` — per-org worker task (stub for now)
- Feature flag: `organizations:seer-night-shift` (flagpole, disabled by
default)
- Options: `seer.night_shift.enable` (global killswitch),
`seer.night_shift.issues_per_org`
- `NIGHT_SHIFT` referrer and automation source enums
- Cron schedule entry in `TASKWORKER_REGION_SCHEDULES`
**Not included (follow-ups):**
- Issue selection logic in the worker task
- Options-automator config (must deploy this PR first)
Refs
https://www.notion.so/sentry/Seer-Night-Shift-3338b10e4b5d807e80a6fbd6d70b3f60
---------
Co-authored-by: Claude <noreply@anthropic.com>


Add the scaffolding for a nightly cron job ("Night Shift") that fans out per-org to trigger Seer Autofix on top issues. This decouples automation from the hot
post_processpipeline, enabling better issue selection and serving lower-volume orgs.The scheduler iterates active orgs using batched feature flag checks (
batch_has_for_organizations) to avoid N+1, applies deterministic jitter to spread load, and dispatches per-org worker tasks. The worker task is currently a stub that just logs — the issue selection and autofix triggering logic will be added in a follow-up once we've decided on the approach.What's included:
schedule_night_shift()— cron scheduler task (daily at 10:00 AM UTC / ~2-3 AM PT)run_night_shift_for_org()— per-org worker task (stub for now)organizations:seer-night-shift(flagpole, disabled by default)seer.night_shift.enable(global killswitch),seer.night_shift.issues_per_orgNIGHT_SHIFTreferrer and automation source enumsTASKWORKER_REGION_SCHEDULESNot included (follow-ups):
Refs https://www.notion.so/sentry/Seer-Night-Shift-3338b10e4b5d807e80a6fbd6d70b3f60