chore: upgrade fork to v0.38.17 by pthmas · Pull Request #49 · dydxprotocol/cometbft

pthmas · 2025-09-02T18:30:14Z

This PR updates the dYdX CometBFT fork to v0.38.17.

Notes

All tests pass locally except for the subset of tests that are already failing on other recent PRs.

state/execution.go

pthmas

Added some comments to the code I had to edit but kept in the original commits.
In order the keep the original commits intact the rest of the changes I made are now in separate commits.

state/execution_test.go

pthmas · 2025-09-24T11:37:46Z

types/validator_set.go

 	var res *Validator
 	for _, val := range vals.Validators {
-		res = res.CompareProposerPriority(val)
+		// Only consider validators that can propose


This check was removed upstream, but I decided to add it again as I think it is relevant.

To clarify, which check are you referring to was removed upstream? I don't see diff between getValWithMostPriority across v0.38.5 and v0.38.17

Right now this is the same code as dydx-fork-v0.38.5 which is right

Interesting, the git diffs were very different, but the outcome is the same, see for yourself:

This is the diff from the 0.38.5

This is the diff from the 0.38.17

Most important is that it is valid in the end 😉

pthmas · 2025-09-24T11:37:59Z

types/validator_set.go

 func (vals *ValidatorSet) findPreviousProposer() *Validator {
 	var previousProposer *Validator
 	for _, val := range vals.Validators {
+		// Only consider validators that can propose


Same as previous comment

types/validator_set_reduced_proposer_set_test.go

pthmas · 2025-09-24T11:40:38Z

abci/example/kvstore/helpers.go

 	pubkey := cmtrand.Bytes(32)
 	power := cmtrand.Uint16() + 1
-	v := types.UpdateValidator(pubkey, int64(power), "")
+	v := types.UpdateValidator(pubkey, int64(power), "", true)


This code was not present in 38.5. I added the changes here to be coherent with the rest of this commit.

pthmas · 2025-09-24T11:41:08Z

abci/example/kvstore/kvstore.go

 				panic(err)
 			}
-			app.valUpdates = append(app.valUpdates, types.UpdateValidator(pubKey, power, keyType))
+			app.valUpdates = append(app.valUpdates, types.UpdateValidator(pubKey, power, keyType, true))


This code was not present in 38.5. I added the changes here to be coherent with the rest of this commit.

* purge based upon block height and block time * filter out short term order placements and cancellations

This is effectively a copy of the fully unsynchronized local client found in tendermint/tendermint#9660. Note that this was reverted and replaced by a version that has a mutex per instance of the client in tendermint/tendermint#9830. This change can be removed once a fully unsynchronized client is added back.

…/cancels. (#7) * generate protos for dydx * add the matches and tx protos * add dydx order id logic and constants * IsClobOrderTransaction => IsShortTermClobOrderTransaction * disable lint for dydx order id constants * more specific linter disables * rename proto package due to global proto registry name clash * Update import paths for dydx clob types * helpful text string function for order id * minor pr comments, no logic changes * README updates

…tbft#1175) * consensus: remove logic to unlock block on +2/3 prevote for nil * add tests * add changelog * fix tests, remove extra logic * update to match forward port

* Minor * Add check for non-`nil` in `enterCommit`

…not match locked block (cometbft#1203) * consensus: update state to prevote nil when proposal block does not match locked block. (#6986) * add failing test * tweak comments in failing test * failing test comment * initial attempt at removing prevote locked block logic * comment out broken function * undo reset on prevotes * fixing TestProposeValidBlock test * update test for completed POL update * comment updates * further unlock testing * update comments * Update internal/consensus/state.go * spacing nit * comment cleanup * nil check in addVote * update unlock description * update precommit on relock comment * add ensure new timeout back * rename IsZero to IsNil and replace uses of block len check with helper * add testing.T to new assertions * begin removing unlock condition * fix TestStateProposerSelection2 to precommit for nil correctly * remove erroneous sleep * update TestStatePOL comment * update relock test to be more clear * add _ into test names * rename slashing * udpate no relock function to be cleaner * do not relock on old proposal test cleanup * con state name update * remove all references to unlock * update test comments to include new * add relock test * add ensureRelock to common_test * remove all event unlock * remove unlock checks * no lint add space * lint ++ * add test for nil prevote on different proposal * fix prevote nil condition * fix defaultDoPrevote * state_test.go fixes to accomodate prevoting for nil * add failing test for POL from previous round case * update prevote logic to prevote POL from previous round * state.go comment fixes * update validatePrevotes to correctly look for nil * update new test name and comment * update POLFromPreviousRound test * fixes post merge * fix spacing * make the linter happy * change prevote log message * update prevote nil debug line * update enterPrevote comment * lint * Update internal/consensus/state.go Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * Update internal/consensus/state.go Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * add english description of alg rules * Update internal/consensus/state.go Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * comment fixes from review * fix comment * fix comment Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * Fix UTs * Addressed comments * Add changelog * Update consensus/state.go Co-authored-by: Hernán Vanzetto <15466498+hvanz@users.noreply.github.com> --------- Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Hernán Vanzetto <15466498+hvanz@users.noreply.github.com>

…d it (cometbft#1231) * Request ProcessProposal only the first time * Avoid validation when proposal matches locked block * comments * Add changelog * validate inside LockedRound == -1 block * Prevote if valid block matches proposal * Update consensus/state.go Co-authored-by: Sergio Mena <sergio@informal.systems> * Update consensus/state.go Co-authored-by: lasaro <lasaro@informal.systems> * Typo in spec --------- Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: lasaro <lasaro@informal.systems>

…metbft#2218) This bug was possibly introduced by cometbft#1203. This PR refactored the consensus operation, propose -> prevote transition (`defaultDoPrevote` method), to match the pseudo-code. The refactoring assumed that when `cs.ProposalBlock` is unset (nil), so should be `cs.Proposal` (meaning that the node didn't receive the round proposal). In this case, the node must prevote nil. The added test unit describes a scenario where `cs.Proposal` is nil (not received), but the block ` cs.ProposalBlock` is received because the node sees a Polka for a block, then receives the full associated block. The node must prevote nil in this scenario, in line with cometbft#1203. But the lack of the test on `cs.Proposal` leads to a bug. This bug does not affect previous releases except for `v1.0`. --- - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments - [ ] Title follows the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

…ydxprotocol#34) * Backport timeout_commit change * change test * Fix byzantine test * Fix more consensus test

@greg-szabo

* feat(blocksync): set the max number of (concurrently) downloaded blocks (backport cometbft#2467) (cometbft#2515) Manual backport of cometbft#2467 * feat(blocksync): sort peers by download rate & multiple requests for closer blocks (backport cometbft#2475) (cometbft#2576) This is an automatic backport of pull request cometbft#2475 done by [Mergify](https://mergify.com). Cherry-pick of f8366fc has failed: ``` On branch mergify/bp/v0.38.x/pr-2475 Your branch is up to date with 'origin/v0.38.x'. You are currently cherry-picking commit f8366fc. (fix conflicts and run "git cherry-pick --continue") (use "git cherry-pick --skip" to skip this patch) (use "git cherry-pick --abort" to cancel the cherry-pick operation) Changes to be committed: new file: .changelog/unreleased/improvements/2475-blocksync-2nd-request.md new file: .changelog/unreleased/improvements/2475-blocksync-no-block-response.md new file: .changelog/unreleased/improvements/2475-blocksync-sort-peers.md modified: blocksync/reactor.go Unmerged paths: (use "git add <file>..." to mark resolution) both modified: blocksync/pool.go ``` To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally --- <details> <summary>Mergify commands and options</summary> <br /> More conditions and actions can be found in the [documentation](https://docs.mergify.com/). You can also trigger Mergify actions by commenting on this pull request: - `@Mergifyio refresh` will re-evaluate the rules - `@Mergifyio rebase` will rebase this PR on its base branch - `@Mergifyio update` will merge the base branch into this PR - `@Mergifyio backport <destination>` will backport this PR on `<destination>` branch Additionally, on Mergify [dashboard](https://dashboard.mergify.com) you can: - look at your merge queues - generate the Mergify configuration with the config editor. Finally, you can contact us on https://mergify.com </details> --------- Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * fix(blocksync): use timer instead of time.After (backport cometbft#2584) (cometbft#2587) This is an automatic backport of pull request cometbft#2584 done by [Mergify](https://mergify.com). --- <details> <summary>Mergify commands and options</summary> <br /> More conditions and actions can be found in the [documentation](https://docs.mergify.com/). You can also trigger Mergify actions by commenting on this pull request: - `@Mergifyio refresh` will re-evaluate the rules - `@Mergifyio rebase` will rebase this PR on its base branch - `@Mergifyio update` will merge the base branch into this PR - `@Mergifyio backport <destination>` will backport this PR on `<destination>` branch Additionally, on Mergify [dashboard](https://dashboard.mergify.com) you can: - look at your merge queues - generate the Mergify configuration with the config editor. Finally, you can contact us on https://mergify.com </details> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * ABC-0013 fix and broken test * blocksync pool ban test * Simplified blocksync malicious node test, fix nil pointer error * Test assertions have more detail * Update blocksync/pool_test.go Co-authored-by: Sergio Mena <sergio@informal.systems> * Update blocksync/pool_test.go Co-authored-by: Sergio Mena <sergio@informal.systems> * Update blocksync/pool_test.go * Remove one thread to make test more compact * Removed defers from test * Readded good peer to test * Update blocksync/pool_test.go * Release/v0.38.8 (cometbft#3350)  Release v0.38.8 [CHANGELOG](https://github.com/cometbft/cometbft/blob/6814b5fff269f5ec6988a4832f24f4804d705ca9/CHANGELOG.md) --- - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments - [ ] Title follows the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec * fix(test): fix TestBlockPoolMaliciousNode DATA RACE (backport cometbft#4636) (cometbft#4641) Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> * fix(test): `TestBlockPoolMaliciousNode` shutdown threads at exit (backport cometbft#4633) (cometbft#4635) This is a drive-by fix of a test that doesn't shut its threads down until the whole `go test` execution finishes. I think we have a bunch of these, but I came across this one during an unrelated troubleshooting. Is it worth fixing this? It's not really causing any issues, it's just sloppy coding. The only way to see any difference is to run the `go test` until it reaches its time limit and panics. In that case, the trace will contain references to the threads. For example: ``` go test github.com/cometbft/cometbft/blocksync -v -run TestBlockPoolMaliciousNode -count 100 -failfast -race -timeout 30s ``` After 30 seconds the test didn't run 100 times yet, hence `go test` panics. Because the test has been run multiple times already, multiple sets of threads will be reported in the panic. With the fix, only one set is reported. Author: @greg-szabo <hr>This is an automatic backport of pull request cometbft#4633 done by [Mergify](https://mergify.com). --------- Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Merge commit from fork lower than what was previously reported GHSA-22qq-3xwm-r5x4 --------- Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Greg Szabo <greg@philosobear.com> Co-authored-by: Greg Szabo <16846635+greg-szabo@users.noreply.github.com> Co-authored-by: Sergio Mena <sergio@informal.systems>

…ydxprotocol#48) Co-authored-by: Tian <tian@dydx.exchange> Co-authored-by: Pierrick Thomas <pierrick@binary.builders>

pthmas marked this pull request as ready for review September 2, 2025 19:53

pthmas self-assigned this Sep 2, 2025

pthmas force-pushed the dydx-fork-v0.38.17 branch 2 times, most recently from 74e345c to 2f4497f Compare September 5, 2025 08:48

James Jia and others added 2 commits September 5, 2025 10:54

update LICENSE (dydxprotocol#25)

4c9785f

Add fork workflow instructions

daccf42

pthmas force-pushed the dydx-fork-v0.38.17 branch from 2f4497f to d40cfe0 Compare September 5, 2025 08:55

teddyding reviewed Sep 8, 2025

View reviewed changes

state/execution.go Outdated Show resolved Hide resolved

pthmas force-pushed the dydx-fork-v0.38.17 branch from 1fac393 to 02a8965 Compare September 9, 2025 14:49

pthmas force-pushed the dydx-fork-v0.38.17 branch from 02a8965 to aca2c1e Compare September 24, 2025 11:27

pthmas commented Sep 24, 2025

View reviewed changes

lcwik and others added 15 commits September 29, 2025 11:21

Ignore PlaceOrder/CancelOrder in mempool reaping and recheck

225cf1b

Add locking around processing of a block

5776821

Add dydx-specific changes to CList mempool:

1c8c9e3

* purge based upon block height and block time * filter out short term order placements and cancellations

Set KeepInvalidTxsInCache to true in mempool config

ce7e7a9

consensus: remove logic to unlock block on +2/3 prevote for nil (come…

0c9a6d2

…tbft#1175) * consensus: remove logic to unlock block on +2/3 prevote for nil * add tests * add changelog * fix tests, remove extra logic * update to match forward port

Add check for non-nil in enterCommit (cometbft#1208)

19318a1

* Minor * Add check for non-`nil` in `enterCommit`

exclude MsgBatchCancel from mempool (dydxprotocol#28)

4078f79

Fix proto/lint/e2e (dydxprotocol#33)

c01f8fc

Log out proposer when proposal is rejected (dydxprotocol#32)

0112f3e

Merge commit from fork (dydxprotocol#37)

f5bf0f2

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

pthmas force-pushed the dydx-fork-v0.38.17 branch 4 times, most recently from f68df2b to b7fc67d Compare September 29, 2025 10:11

vincentwschau and others added 11 commits September 29, 2025 12:15

Upgrade cosmossdk.io/math to v1.4.0 (dydxprotocol#38)

b943e8b

feat(abci): Backport: move timeout_commit into FinalizeBlockResponse (d…

f0c3174

…ydxprotocol#34) * Backport timeout_commit change * change test * Fix byzantine test * Fix more consensus test

replace CanPropose with ProposeDisabled for backward compatibility (d…

879ed0c

…ydxprotocol#48) Co-authored-by: Tian <tian@dydx.exchange> Co-authored-by: Pierrick Thomas <pierrick@binary.builders>

Update LICENSE 2025 (dydxprotocol#50)

fdde4c1

do not include ProposeDisabled in validator hash (dydxprotocol#51)

38dd3a7

Add locking to applyverifiedblock

fdde69a

Fix doUpdate test

969cc9e

Skip testing build for 32bits ARM Linux

0d895fd

fix protobuf version

8d7b63f

Remove double triggering of tests

e8c66d8

pthmas force-pushed the dydx-fork-v0.38.17 branch from b7fc67d to e8c66d8 Compare September 29, 2025 10:15

bump go version for mock generation

a56bf8c

northstar456 approved these changes Oct 9, 2025

View reviewed changes

northstar456 merged commit 4e26d25 into dydxprotocol:dydx-fork-v0.38.17 Oct 9, 2025
14 of 16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: upgrade fork to v0.38.17#49

chore: upgrade fork to v0.38.17#49
northstar456 merged 29 commits intodydxprotocol:dydx-fork-v0.38.17from
01builders:dydx-fork-v0.38.17

pthmas commented Sep 2, 2025 •

edited

Loading

Uh oh!

Uh oh!

pthmas left a comment

Uh oh!

Uh oh!

pthmas Sep 24, 2025

Uh oh!

northstar456 Sep 29, 2025 •

edited

Loading

Uh oh!

pthmas Sep 29, 2025 •

edited

Loading

Uh oh!

pthmas Sep 24, 2025

Uh oh!

Uh oh!

pthmas Sep 24, 2025

Uh oh!

pthmas Sep 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

13 participants

Conversation

pthmas commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notes

Uh oh!

Uh oh!

pthmas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pthmas Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

northstar456 Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pthmas Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pthmas Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pthmas Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

pthmas Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

13 participants

pthmas commented Sep 2, 2025 •

edited

Loading

northstar456 Sep 29, 2025 •

edited

Loading

pthmas Sep 29, 2025 •

edited

Loading