Skip to content

EIP-4844: Dynamic topic lists for ReqResp and Gossip#4848

Merged
twoeths merged 4 commits intounstablefrom
dapplion/dynamic-reqresp
Dec 7, 2022
Merged

EIP-4844: Dynamic topic lists for ReqResp and Gossip#4848
twoeths merged 4 commits intounstablefrom
dapplion/dynamic-reqresp

Conversation

@dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 5, 2022

Motivation

Our current implementation of ReqResp protocols is static. Starting with EIP4844, we must register and un-register protocols at fork boundaries

Description

  • Implement dynamic list of ReqResp protocols + register / un-register logic at fork boundaries
  • De-duplicate dynamic list of gossip topics

@dapplion dapplion requested a review from a team as a code owner December 5, 2022 08:57
@dapplion dapplion changed the title Dynamic topic lists for ReqResp and Gossip EIP-4844: Dynamic topic lists for ReqResp and Gossip Dec 5, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 8120b55 Previous: b7c7910 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.6370 ms/op 2.5877 ms/op 0.63
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 60.561 us/op 81.423 us/op 0.74
BLS verify - blst-native 2.1701 ms/op 1.8565 ms/op 1.17
BLS verifyMultipleSignatures 3 - blst-native 4.4956 ms/op 3.8159 ms/op 1.18
BLS verifyMultipleSignatures 8 - blst-native 9.7108 ms/op 8.2060 ms/op 1.18
BLS verifyMultipleSignatures 32 - blst-native 35.265 ms/op 29.733 ms/op 1.19
BLS aggregatePubkeys 32 - blst-native 46.479 us/op 39.504 us/op 1.18
BLS aggregatePubkeys 128 - blst-native 181.89 us/op 153.36 us/op 1.19
getAttestationsForBlock 77.144 ms/op 103.03 ms/op 0.75
isKnown best case - 1 super set check 484.00 ns/op 430.00 ns/op 1.13
isKnown normal case - 2 super set checks 474.00 ns/op 417.00 ns/op 1.14
isKnown worse case - 16 super set checks 470.00 ns/op 411.00 ns/op 1.14
CheckpointStateCache - add get delete 8.6600 us/op 9.5090 us/op 0.91
validate gossip signedAggregateAndProof - struct 5.0791 ms/op 4.2848 ms/op 1.19
validate gossip attestation - struct 2.3827 ms/op 2.0383 ms/op 1.17
pickEth1Vote - no votes 2.2612 ms/op 2.1611 ms/op 1.05
pickEth1Vote - max votes 17.716 ms/op 24.238 ms/op 0.73
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.855 ms/op 11.713 ms/op 1.01
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.022 ms/op 23.254 ms/op 0.82
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4298 ms/op 1.6071 ms/op 0.89
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.110 ms/op 15.907 ms/op 0.76
bytes32 toHexString 911.00 ns/op 1.2620 us/op 0.72
bytes32 Buffer.toString(hex) 704.00 ns/op 715.00 ns/op 0.98
bytes32 Buffer.toString(hex) from Uint8Array 975.00 ns/op 944.00 ns/op 1.03
bytes32 Buffer.toString(hex) + 0x 747.00 ns/op 714.00 ns/op 1.05
Object access 1 prop 0.34400 ns/op 0.38200 ns/op 0.90
Map access 1 prop 0.30700 ns/op 0.29300 ns/op 1.05
Object get x1000 10.957 ns/op 17.018 ns/op 0.64
Map get x1000 1.0150 ns/op 0.97800 ns/op 1.04
Object set x1000 72.055 ns/op 130.67 ns/op 0.55
Map set x1000 47.304 ns/op 79.062 ns/op 0.60
Return object 10000 times 0.43300 ns/op 0.36670 ns/op 1.18
Throw Error 10000 times 6.1709 us/op 5.9525 us/op 1.04
fastMsgIdFn sha256 / 200 bytes 4.8080 us/op 4.1650 us/op 1.15
fastMsgIdFn h32 xxhash / 200 bytes 539.00 ns/op 604.00 ns/op 0.89
fastMsgIdFn h64 xxhash / 200 bytes 765.00 ns/op 854.00 ns/op 0.90
fastMsgIdFn sha256 / 1000 bytes 15.379 us/op 13.251 us/op 1.16
fastMsgIdFn h32 xxhash / 1000 bytes 698.00 ns/op 742.00 ns/op 0.94
fastMsgIdFn h64 xxhash / 1000 bytes 902.00 ns/op 843.00 ns/op 1.07
fastMsgIdFn sha256 / 10000 bytes 134.99 us/op 112.17 us/op 1.20
fastMsgIdFn h32 xxhash / 10000 bytes 2.5740 us/op 2.4360 us/op 1.06
fastMsgIdFn h64 xxhash / 10000 bytes 1.9820 us/op 1.8000 us/op 1.10
enrSubnets - fastDeserialize 64 bits 2.6360 us/op 3.3100 us/op 0.80
enrSubnets - ssz BitVector 64 bits 827.00 ns/op 835.00 ns/op 0.99
enrSubnets - fastDeserialize 4 bits 360.00 ns/op 442.00 ns/op 0.81
enrSubnets - ssz BitVector 4 bits 793.00 ns/op 823.00 ns/op 0.96
prioritizePeers score -10:0 att 32-0.1 sync 2-0 83.614 us/op 103.37 us/op 0.81
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 127.73 us/op 146.94 us/op 0.87
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 197.88 us/op 250.52 us/op 0.79
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 343.22 us/op 393.58 us/op 0.87
prioritizePeers score 0:0 att 64-1 sync 4-1 417.68 us/op 469.36 us/op 0.89
RateTracker 1000000 limit, 1 obj count per request 186.82 ns/op 196.35 ns/op 0.95
RateTracker 1000000 limit, 2 obj count per request 135.18 ns/op 150.19 ns/op 0.90
RateTracker 1000000 limit, 4 obj count per request 110.29 ns/op 128.72 ns/op 0.86
RateTracker 1000000 limit, 8 obj count per request 97.185 ns/op 109.89 ns/op 0.88
RateTracker with prune 3.8370 us/op 4.8460 us/op 0.79
array of 16000 items push then shift 51.598 us/op 2.8415 us/op 18.16
LinkedList of 16000 items push then shift 12.085 ns/op 19.277 ns/op 0.63
array of 16000 items push then pop 201.17 ns/op 260.85 ns/op 0.77
LinkedList of 16000 items push then pop 11.992 ns/op 17.984 ns/op 0.67
array of 24000 items push then shift 77.366 us/op 3.9780 us/op 19.45
LinkedList of 24000 items push then shift 12.365 ns/op 23.653 ns/op 0.52
array of 24000 items push then pop 191.03 ns/op 211.55 ns/op 0.90
LinkedList of 24000 items push then pop 12.166 ns/op 20.118 ns/op 0.60
intersect bitArray bitLen 8 10.742 ns/op 11.766 ns/op 0.91
intersect array and set length 8 133.25 ns/op 185.99 ns/op 0.72
intersect bitArray bitLen 128 57.928 ns/op 62.042 ns/op 0.93
intersect array and set length 128 1.7266 us/op 2.3980 us/op 0.72
Buffer.concat 32 items 1.7380 ns/op 1.9770 ns/op 0.88
pass gossip attestations to forkchoice per slot 3.6027 ms/op 4.0803 ms/op 0.88
computeDeltas 4.7424 ms/op 6.2317 ms/op 0.76
computeProposerBoostScoreFromBalances 803.64 us/op 921.41 us/op 0.87
altair processAttestation - 250000 vs - 7PWei normalcase 3.2775 ms/op 4.5081 ms/op 0.73
altair processAttestation - 250000 vs - 7PWei worstcase 5.7258 ms/op 7.3138 ms/op 0.78
altair processAttestation - setStatus - 1/6 committees join 182.55 us/op 222.00 us/op 0.82
altair processAttestation - setStatus - 1/3 committees join 355.48 us/op 404.62 us/op 0.88
altair processAttestation - setStatus - 1/2 committees join 509.33 us/op 566.28 us/op 0.90
altair processAttestation - setStatus - 2/3 committees join 676.98 us/op 736.59 us/op 0.92
altair processAttestation - setStatus - 4/5 committees join 944.02 us/op 1.0146 ms/op 0.93
altair processAttestation - setStatus - 100% committees join 1.1212 ms/op 1.1839 ms/op 0.95
altair processBlock - 250000 vs - 7PWei normalcase 24.822 ms/op 29.120 ms/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.556 ms/op 37.982 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase 80.146 ms/op 92.568 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase hashState 110.67 ms/op 99.039 ms/op 1.12
phase0 processBlock - 250000 vs - 7PWei normalcase 3.0934 ms/op 4.1782 ms/op 0.74
phase0 processBlock - 250000 vs - 7PWei worstcase 50.603 ms/op 48.293 ms/op 1.05
altair processEth1Data - 250000 vs - 7PWei normalcase 681.90 us/op 1.0606 ms/op 0.64
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.8240 us/op 9.2200 us/op 0.74
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 20.695 us/op 28.183 us/op 0.73
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 9.3090 us/op 12.031 us/op 0.77
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.7660 us/op 9.4450 us/op 0.72
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 90.165 us/op 104.62 us/op 0.86
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 903.05 us/op 1.0622 ms/op 0.85
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 11.749 ms/op 15.718 ms/op 0.75
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 21.049 ms/op 25.608 ms/op 0.82
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 70.459 ms/op 88.093 ms/op 0.80
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 38.229 ms/op 49.227 ms/op 0.78
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 117.32 ms/op 136.89 ms/op 0.86
Tree 40 250000 create 689.73 ms/op 937.19 ms/op 0.74
Tree 40 250000 get(125000) 222.53 ns/op 301.47 ns/op 0.74
Tree 40 250000 set(125000) 2.0605 us/op 3.2061 us/op 0.64
Tree 40 250000 toArray() 25.771 ms/op 36.314 ms/op 0.71
Tree 40 250000 iterate all - toArray() + loop 26.491 ms/op 36.712 ms/op 0.72
Tree 40 250000 iterate all - get(i) 108.60 ms/op 118.45 ms/op 0.92
MutableVector 250000 create 12.427 ms/op 19.028 ms/op 0.65
MutableVector 250000 get(125000) 11.023 ns/op 14.792 ns/op 0.75
MutableVector 250000 set(125000) 555.24 ns/op 763.93 ns/op 0.73
MutableVector 250000 toArray() 5.8076 ms/op 8.4590 ms/op 0.69
MutableVector 250000 iterate all - toArray() + loop 5.9957 ms/op 8.6017 ms/op 0.70
MutableVector 250000 iterate all - get(i) 3.2454 ms/op 3.4432 ms/op 0.94
Array 250000 create 5.6722 ms/op 7.9429 ms/op 0.71
Array 250000 clone - spread 3.2052 ms/op 4.1764 ms/op 0.77
Array 250000 get(125000) 1.4380 ns/op 1.6680 ns/op 0.86
Array 250000 set(125000) 1.4430 ns/op 1.7050 ns/op 0.85
Array 250000 iterate all - loop 153.51 us/op 167.97 us/op 0.91
effectiveBalanceIncrements clone Uint8Array 300000 161.31 us/op 101.08 us/op 1.60
effectiveBalanceIncrements clone MutableVector 300000 685.00 ns/op 1.2870 us/op 0.53
effectiveBalanceIncrements rw all Uint8Array 300000 247.55 us/op 252.63 us/op 0.98
effectiveBalanceIncrements rw all MutableVector 300000 143.76 ms/op 241.39 ms/op 0.60
phase0 afterProcessEpoch - 250000 vs - 7PWei 189.14 ms/op 184.41 ms/op 1.03
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.168 ms/op 86.023 ms/op 0.68
altair processEpoch - mainnet_e81889 486.46 ms/op 605.39 ms/op 0.80
mainnet_e81889 - altair beforeProcessEpoch 116.87 ms/op 158.48 ms/op 0.74
mainnet_e81889 - altair processJustificationAndFinalization 16.213 us/op 31.467 us/op 0.52
mainnet_e81889 - altair processInactivityUpdates 8.7347 ms/op 12.448 ms/op 0.70
mainnet_e81889 - altair processRewardsAndPenalties 77.561 ms/op 99.155 ms/op 0.78
mainnet_e81889 - altair processRegistryUpdates 2.4830 us/op 3.9850 us/op 0.62
mainnet_e81889 - altair processSlashings 572.00 ns/op 803.00 ns/op 0.71
mainnet_e81889 - altair processEth1DataReset 622.00 ns/op 1.0090 us/op 0.62
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.3500 ms/op 2.2845 ms/op 1.03
mainnet_e81889 - altair processSlashingsReset 4.0170 us/op 6.4180 us/op 0.63
mainnet_e81889 - altair processRandaoMixesReset 4.1020 us/op 7.9700 us/op 0.51
mainnet_e81889 - altair processHistoricalRootsUpdate 693.00 ns/op 1.0240 us/op 0.68
mainnet_e81889 - altair processParticipationFlagUpdates 2.2970 us/op 4.9790 us/op 0.46
mainnet_e81889 - altair processSyncCommitteeUpdates 629.00 ns/op 1.1650 us/op 0.54
mainnet_e81889 - altair afterProcessEpoch 201.28 ms/op 193.77 ms/op 1.04
phase0 processEpoch - mainnet_e58758 483.81 ms/op 556.99 ms/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 198.45 ms/op 264.38 ms/op 0.75
mainnet_e58758 - phase0 processJustificationAndFinalization 14.952 us/op 43.321 us/op 0.35
mainnet_e58758 - phase0 processRewardsAndPenalties 67.374 ms/op 149.36 ms/op 0.45
mainnet_e58758 - phase0 processRegistryUpdates 7.8810 us/op 13.455 us/op 0.59
mainnet_e58758 - phase0 processSlashings 611.00 ns/op 1.0440 us/op 0.59
mainnet_e58758 - phase0 processEth1DataReset 599.00 ns/op 1.1900 us/op 0.50
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.9192 ms/op 2.0219 ms/op 0.95
mainnet_e58758 - phase0 processSlashingsReset 3.5460 us/op 6.5420 us/op 0.54
mainnet_e58758 - phase0 processRandaoMixesReset 4.1370 us/op 8.2140 us/op 0.50
mainnet_e58758 - phase0 processHistoricalRootsUpdate 688.00 ns/op 1.2710 us/op 0.54
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.3950 us/op 8.7730 us/op 0.39
mainnet_e58758 - phase0 afterProcessEpoch 166.26 ms/op 158.26 ms/op 1.05
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9771 ms/op 2.7282 ms/op 0.72
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2128 ms/op 3.4648 ms/op 0.64
altair processInactivityUpdates - 250000 normalcase 32.079 ms/op 44.098 ms/op 0.73
altair processInactivityUpdates - 250000 worstcase 41.744 ms/op 51.752 ms/op 0.81
phase0 processRegistryUpdates - 250000 normalcase 6.1280 us/op 8.9190 us/op 0.69
phase0 processRegistryUpdates - 250000 badcase_full_deposits 368.47 us/op 414.94 us/op 0.89
phase0 processRegistryUpdates - 250000 worstcase 0.5 172.99 ms/op 224.83 ms/op 0.77
altair processRewardsAndPenalties - 250000 normalcase 75.818 ms/op 136.97 ms/op 0.55
altair processRewardsAndPenalties - 250000 worstcase 75.391 ms/op 90.593 ms/op 0.83
phase0 getAttestationDeltas - 250000 normalcase 11.959 ms/op 13.331 ms/op 0.90
phase0 getAttestationDeltas - 250000 worstcase 12.338 ms/op 13.855 ms/op 0.89
phase0 processSlashings - 250000 worstcase 5.6405 ms/op 5.8982 ms/op 0.96
altair processSyncCommitteeUpdates - 250000 284.03 ms/op 284.02 ms/op 1.00
BeaconState.hashTreeRoot - No change 550.00 ns/op 489.00 ns/op 1.12
BeaconState.hashTreeRoot - 1 full validator 72.713 us/op 65.841 us/op 1.10
BeaconState.hashTreeRoot - 32 full validator 715.05 us/op 628.26 us/op 1.14
BeaconState.hashTreeRoot - 512 full validator 6.9483 ms/op 8.4741 ms/op 0.82
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 98.291 us/op 78.248 us/op 1.26
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2365 ms/op 1.1797 ms/op 1.05
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.485 ms/op 15.529 ms/op 1.06
BeaconState.hashTreeRoot - 1 balances 67.718 us/op 60.743 us/op 1.11
BeaconState.hashTreeRoot - 32 balances 642.06 us/op 563.28 us/op 1.14
BeaconState.hashTreeRoot - 512 balances 6.2216 ms/op 5.8055 ms/op 1.07
BeaconState.hashTreeRoot - 250000 balances 101.58 ms/op 96.024 ms/op 1.06
aggregationBits - 2048 els - zipIndexesInBitList 28.824 us/op 38.020 us/op 0.76
regular array get 100000 times 60.586 us/op 67.491 us/op 0.90
wrappedArray get 100000 times 60.572 us/op 67.412 us/op 0.90
arrayWithProxy get 100000 times 27.651 ms/op 28.484 ms/op 0.97
ssz.Root.equals 485.00 ns/op 531.00 ns/op 0.91
byteArrayEquals 476.00 ns/op 484.00 ns/op 0.98
shuffle list - 16384 els 11.672 ms/op 10.913 ms/op 1.07
shuffle list - 250000 els 170.81 ms/op 161.08 ms/op 1.06
processSlot - 1 slots 13.371 us/op 11.524 us/op 1.16
processSlot - 32 slots 1.9143 ms/op 1.7281 ms/op 1.11
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 347.25 us/op 395.12 us/op 0.88
getCommitteeAssignments - req 1 vs - 250000 vc 5.3441 ms/op 5.3237 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 7.8675 ms/op 7.3329 ms/op 1.07
getCommitteeAssignments - req 1000 vs - 250000 vc 8.4352 ms/op 7.7879 ms/op 1.08
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 8.0100 ns/op 10.590 ns/op 0.76
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0528 us/op 1.0595 us/op 0.99
computeProposers - vc 250000 16.848 ms/op 16.796 ms/op 1.00
computeEpochShuffling - vc 250000 173.48 ms/op 166.12 ms/op 1.04
getNextSyncCommittee - vc 250000 282.67 ms/op 272.66 ms/op 1.04

by benchmarkbot/action

@dapplion dapplion added the spec-deneb 🐡 Issues targeting the Deneb spec version label Dec 5, 2022
@dapplion dapplion mentioned this pull request Dec 6, 2022
Merged
@twoeths twoeths force-pushed the dapplion/dynamic-reqresp branch from 76b01f7 to e1c8fe3 Compare December 7, 2022 03:12
@twoeths twoeths merged commit eb61572 into unstable Dec 7, 2022
@twoeths twoeths deleted the dapplion/dynamic-reqresp branch December 7, 2022 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@twoeths twoeths twoeths approved these changes

Assignees

No one assigned

Labels

spec-deneb 🐡 Issues targeting the Deneb spec version

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dapplion @twoeths