Skip to content

feat(test): add stress-testing to known flaky tests reruns#112578

Open
JoshuaKGoldberg wants to merge 3 commits intomasterfrom
known-flake-stress-testing
Open

feat(test): add stress-testing to known flaky tests reruns#112578
JoshuaKGoldberg wants to merge 3 commits intomasterfrom
known-flake-stress-testing

Conversation

@JoshuaKGoldberg
Copy link
Copy Markdown
Member

@JoshuaKGoldberg JoshuaKGoldberg commented Apr 9, 2026

#111860 added a way to rerun known flaky tests 50x locally or in CI. But that didn't reliably reproduce errors on all of them. I think that's because many tests only failed when the CI machine is separately slow, so if the machine is running fine, running 1x vs 50x doesn't make a difference.

This changes flaky tests to run under each of 5 different simulated slowdowns, 10x each (so still 50x each total):

  • withDelayedFetch: Simulates a slow network by adding delays to global fetch calls.
  • withMainThreadCpuLoad: Simulates an overloaded main thread by doing busy-work on a fixed interval.
  • withMemoryPressure: Simulates low available memory by retaining a large buffer of bytes data.
  • withMicrotaskChurn: Simulates a busy machine by continuously queueing microtasks.
  • withRealWallClockDelay: Simulates a slow machine with many timers by continuously running no-op async delays.

We can see in this PR's CI that although the flaky tests still don't always fail, failures do happen a lot more now. 🥳

Any CI failures (e.g. Jest (1), Jest (2)) are IMO expected and shouldn't block review. The tests themselves aren't changed, and master doesn't run with the flake label.

@JoshuaKGoldberg JoshuaKGoldberg added the Frontend: Rerun Flaky Tests Known flaky tests should be run many times, just to be safe. label Apr 9, 2026
@github-actions github-actions bot added the Scope: Frontend Automatically applied to PRs that change frontend components label Apr 9, 2026
@JoshuaKGoldberg JoshuaKGoldberg force-pushed the known-flake-stress-testing branch from f4e0855 to 4b4de37 Compare April 9, 2026 14:00
@JoshuaKGoldberg JoshuaKGoldberg marked this pull request as ready for review April 9, 2026 15:57
@JoshuaKGoldberg JoshuaKGoldberg requested review from a team as code owners April 9, 2026 15:57
Comment thread tests/js/sentry-test/isKnownFlake/withRealWallClockDelay.ts
Copy link
Copy Markdown
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4b4de37. Configure here.

Comment thread tests/js/sentry-test/isKnownFlake/withDelayedFetch.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Frontend: Rerun Flaky Tests Known flaky tests should be run many times, just to be safe. Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant