ci: run reth and kona against all acceptance tests#19787
Open
ci: run reth and kona against all acceptance tests#19787
Conversation
Contributor
Author
|
Should we just remove gates as a concept? Better to define skips inside the test instead imho. |
Member
|
So I am not sure this is going to work. Kona doesn't implement all the interfaces needed to be able to run all the acceptance tests (there is a dev interface that only works for the sync-start tests). It may be worth removing the base gate for the opn-reth case |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #19787 +/- ##
===========================================
+ Coverage 75.6% 75.9% +0.3%
===========================================
Files 195 684 +489
Lines 11348 73164 +61816
===========================================
+ Hits 8581 55554 +46973
- Misses 2623 17466 +14843
Partials 144 144
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Contributor
Author
|
Ah thanks for the context @theochap. My main concern with the gate setup is that new tests do not run against kona and reth by default in ci. I think what I'll do is add skips to failing tests, and then remove gates. That way new tests will run. |
joshklop
force-pushed
the
joshklop/ci
branch
3 times, most recently
from
3588930 to
91b56aa
Compare
joshklop
force-pushed
the
joshklop/ci
branch
4 times, most recently
from
d91fb9e to
ee3c17c
Compare
teddyknox
approved these changes
Mar 31, 2026
Contributor
teddyknox
left a comment
teddyknox
left a comment
There was a problem hiding this comment.
Some failing checks but intent looks good to me.
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
Remove the base gate from the op-node/op-reth and kona/op-reth CI jobs so they run all acceptance tests instead of only the base subset. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Gates are no longer needed now that all acceptance tests run for every implementation variant. Skips are defined inside individual tests instead. - Remove gate parameter from op-acceptance-tests justfile. - Remove gate parameter from CircleCI op-acceptance-tests job. - Remove gates/base.txt. - Update kona tests justfile and README. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
See this workflow. I also reran the failing tests locally to ensure these aren't simple flakes. https://app.circleci.com/pipelines/github/ethereum-optimism/optimism/121247/workflows/38caeea6-ffda-4f83-a135-61a3864348bd/jobs/4699628/tests
See this ci run. The previous commit already skips many of these since they are op-reth failures. I ran all of the remaining tests locally and removed the ones that succeeded: all that remain are the ones that are actual failures or very intermittent flakes. https://app.circleci.com/pipelines/github/ethereum-optimism/optimism/121247/workflows/38caeea6-ffda-4f83-a135-61a3864348bd/jobs/4699629/tests
joshklop
force-pushed
the
joshklop/ci
branch
2 times, most recently
from
c222c14 to
75825e4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
instead of just the base gate.