Skip to content

Conversation

@amandazhuyilan
Copy link
Contributor

@amandazhuyilan amandazhuyilan commented Nov 24, 2025

Description

AAI-538: implement a tighter SES retry policy for queued emails

Changes

  • Added first_attempt_at tracking plus a schema migration so we can limit delivery attempts per notification.
  • Updated the scheduler to cap emails at 2 attempts, retry only on transient/throttling/network failures, randomly delay retries 15–30 minutes, and stop once an hour has passed since the first try.
    • Max attempts per email: 2 (initial + 1 retry)
    • Retry only on temporary errors: SES transient / throttling / network
    • Retry delay: 15–30 minutes after the first failure
    • Max window: 1 hour from first attempt (no retries after that)
  • Updated tests

Checklist

  • I have commented my code, particularly in hard-to-understand areas
  • I have added unit / integration tests that prove my fix is effective or that my feature works
  • I have run all tests locally and they pass
  • I have updated the documentation (if applicable)
  • For any new secrets, I have updated the shared spreadsheet and the GitHub Secrets.

marius-mather
marius-mather previously approved these changes Nov 24, 2025
Copy link
Collaborator

@marius-mather marius-mather left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but we might need to allow for more retries in future, so last_attempt_at might make more sense than first_attempt_at

Copy link
Collaborator

@marius-mather marius-mather left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to go

@amandazhuyilan amandazhuyilan merged commit e727497 into main Nov 24, 2025
4 checks passed
@amandazhuyilan amandazhuyilan deleted the fix-robust-email-sending branch November 24, 2025 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants