Skip to content

EIP-4844: Handle BlobsSidecar#4882

Merged
g11tech merged 1 commit intounstablefrom
dapplion/eip-4844-gossip-handler
Dec 7, 2022
Merged

EIP-4844: Handle BlobsSidecar#4882
g11tech merged 1 commit intounstablefrom
dapplion/eip-4844-gossip-handler

Conversation

@dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 7, 2022

Motivation

Description

Complete handling logic for BlobsSidecar

  • On gossip, handle BlockInput and call processBlock
  • On block verify conditionally run validateBlobsSidecar, skipping for gossip since it's already verified
  • On block import persist BlobsSidecar to db

@dapplion dapplion added the spec-deneb 🐡 Issues targeting the Deneb spec version label Dec 7, 2022
@dapplion dapplion requested a review from a team as a code owner December 7, 2022 09:09
@github-actions
Copy link
Contributor

github-actions bot commented Dec 7, 2022

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: ce4d0eb Previous: 4115f67 Ratio
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.4030 us/op 23.186 us/op 0.28
Full benchmark results
Benchmark suite Current: ce4d0eb Previous: 4115f67 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.7175 ms/op 2.3522 ms/op 0.73
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 62.459 us/op 88.916 us/op 0.70
BLS verify - blst-native 2.1686 ms/op 2.7517 ms/op 0.79
BLS verifyMultipleSignatures 3 - blst-native 4.4949 ms/op 5.5217 ms/op 0.81
BLS verifyMultipleSignatures 8 - blst-native 9.7051 ms/op 12.040 ms/op 0.81
BLS verifyMultipleSignatures 32 - blst-native 35.260 ms/op 40.842 ms/op 0.86
BLS aggregatePubkeys 32 - blst-native 46.698 us/op 55.862 us/op 0.84
BLS aggregatePubkeys 128 - blst-native 182.24 us/op 238.52 us/op 0.76
getAttestationsForBlock 79.560 ms/op 119.79 ms/op 0.66
isKnown best case - 1 super set check 504.00 ns/op 508.00 ns/op 0.99
isKnown normal case - 2 super set checks 483.00 ns/op 493.00 ns/op 0.98
isKnown worse case - 16 super set checks 487.00 ns/op 475.00 ns/op 1.03
CheckpointStateCache - add get delete 8.5640 us/op 10.883 us/op 0.79
validate gossip signedAggregateAndProof - struct 5.0448 ms/op 6.0945 ms/op 0.83
validate gossip attestation - struct 2.3712 ms/op 2.8254 ms/op 0.84
pickEth1Vote - no votes 2.1356 ms/op 2.6825 ms/op 0.80
pickEth1Vote - max votes 19.641 ms/op 22.183 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.100 ms/op 14.179 ms/op 0.92
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.494 ms/op 24.148 ms/op 0.85
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5654 ms/op 1.8585 ms/op 0.84
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.629 ms/op 15.190 ms/op 0.83
bytes32 toHexString 968.00 ns/op 1.2230 us/op 0.79
bytes32 Buffer.toString(hex) 730.00 ns/op 799.00 ns/op 0.91
bytes32 Buffer.toString(hex) from Uint8Array 1.0450 us/op 1.0640 us/op 0.98
bytes32 Buffer.toString(hex) + 0x 733.00 ns/op 778.00 ns/op 0.94
Object access 1 prop 0.37300 ns/op 0.42700 ns/op 0.87
Map access 1 prop 0.30900 ns/op 0.32900 ns/op 0.94
Object get x1000 11.225 ns/op 15.740 ns/op 0.71
Map get x1000 0.92400 ns/op 0.91500 ns/op 1.01
Object set x1000 79.004 ns/op 120.88 ns/op 0.65
Map set x1000 48.584 ns/op 79.547 ns/op 0.61
Return object 10000 times 0.43870 ns/op 0.41480 ns/op 1.06
Throw Error 10000 times 5.9919 us/op 7.1478 us/op 0.84
fastMsgIdFn sha256 / 200 bytes 4.9100 us/op 5.2810 us/op 0.93
fastMsgIdFn h32 xxhash / 200 bytes 626.00 ns/op 616.00 ns/op 1.02
fastMsgIdFn h64 xxhash / 200 bytes 790.00 ns/op 890.00 ns/op 0.89
fastMsgIdFn sha256 / 1000 bytes 15.539 us/op 16.590 us/op 0.94
fastMsgIdFn h32 xxhash / 1000 bytes 806.00 ns/op 804.00 ns/op 1.00
fastMsgIdFn h64 xxhash / 1000 bytes 914.00 ns/op 975.00 ns/op 0.94
fastMsgIdFn sha256 / 10000 bytes 134.03 us/op 153.73 us/op 0.87
fastMsgIdFn h32 xxhash / 10000 bytes 2.6310 us/op 2.8630 us/op 0.92
fastMsgIdFn h64 xxhash / 10000 bytes 1.9470 us/op 2.0510 us/op 0.95
enrSubnets - fastDeserialize 64 bits 2.7670 us/op 2.9950 us/op 0.92
enrSubnets - ssz BitVector 64 bits 854.00 ns/op 806.00 ns/op 1.06
enrSubnets - fastDeserialize 4 bits 391.00 ns/op 475.00 ns/op 0.82
enrSubnets - ssz BitVector 4 bits 848.00 ns/op 886.00 ns/op 0.96
prioritizePeers score -10:0 att 32-0.1 sync 2-0 87.660 us/op 109.49 us/op 0.80
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 131.26 us/op 152.52 us/op 0.86
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 208.22 us/op 287.05 us/op 0.73
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 343.98 us/op 580.28 us/op 0.59
prioritizePeers score 0:0 att 64-1 sync 4-1 438.94 us/op 615.01 us/op 0.71
RateTracker 1000000 limit, 1 obj count per request 182.55 ns/op 206.50 ns/op 0.88
RateTracker 1000000 limit, 2 obj count per request 133.96 ns/op 156.96 ns/op 0.85
RateTracker 1000000 limit, 4 obj count per request 109.08 ns/op 131.21 ns/op 0.83
RateTracker 1000000 limit, 8 obj count per request 96.374 ns/op 111.18 ns/op 0.87
RateTracker with prune 3.9410 us/op 5.3770 us/op 0.73
array of 16000 items push then shift 51.610 us/op 5.0871 us/op 10.15
LinkedList of 16000 items push then shift 12.344 ns/op 19.017 ns/op 0.65
array of 16000 items push then pop 204.86 ns/op 248.49 ns/op 0.82
LinkedList of 16000 items push then pop 12.081 ns/op 18.341 ns/op 0.66
array of 24000 items push then shift 77.392 us/op 7.5670 us/op 10.23
LinkedList of 24000 items push then shift 12.855 ns/op 19.743 ns/op 0.65
array of 24000 items push then pop 192.19 ns/op 249.49 ns/op 0.77
LinkedList of 24000 items push then pop 12.409 ns/op 17.892 ns/op 0.69
intersect bitArray bitLen 8 11.825 ns/op 12.097 ns/op 0.98
intersect array and set length 8 145.33 ns/op 199.97 ns/op 0.73
intersect bitArray bitLen 128 55.878 ns/op 69.326 ns/op 0.81
intersect array and set length 128 1.8088 us/op 2.5573 us/op 0.71
Buffer.concat 32 items 1.7850 ns/op 2.6230 ns/op 0.68
pass gossip attestations to forkchoice per slot 3.5909 ms/op 4.5404 ms/op 0.79
computeDeltas 4.5510 ms/op 5.5700 ms/op 0.82
computeProposerBoostScoreFromBalances 803.85 us/op 856.30 us/op 0.94
altair processAttestation - 250000 vs - 7PWei normalcase 3.3779 ms/op 5.2350 ms/op 0.65
altair processAttestation - 250000 vs - 7PWei worstcase 5.1797 ms/op 9.2972 ms/op 0.56
altair processAttestation - setStatus - 1/6 committees join 186.40 us/op 235.31 us/op 0.79
altair processAttestation - setStatus - 1/3 committees join 366.65 us/op 450.41 us/op 0.81
altair processAttestation - setStatus - 1/2 committees join 524.78 us/op 656.47 us/op 0.80
altair processAttestation - setStatus - 2/3 committees join 694.87 us/op 910.97 us/op 0.76
altair processAttestation - setStatus - 4/5 committees join 969.16 us/op 1.1653 ms/op 0.83
altair processAttestation - setStatus - 100% committees join 1.1648 ms/op 1.4386 ms/op 0.81
altair processBlock - 250000 vs - 7PWei normalcase 24.629 ms/op 30.195 ms/op 0.82
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.867 ms/op 41.124 ms/op 0.97
altair processBlock - 250000 vs - 7PWei worstcase 86.091 ms/op 103.73 ms/op 0.83
altair processBlock - 250000 vs - 7PWei worstcase hashState 99.090 ms/op 121.24 ms/op 0.82
phase0 processBlock - 250000 vs - 7PWei normalcase 3.1790 ms/op 4.0784 ms/op 0.78
phase0 processBlock - 250000 vs - 7PWei worstcase 50.523 ms/op 61.153 ms/op 0.83
altair processEth1Data - 250000 vs - 7PWei normalcase 707.15 us/op 1.0219 ms/op 0.69
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.4030 us/op 23.186 us/op 0.28
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 21.103 us/op 48.326 us/op 0.44
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 9.4650 us/op 27.610 us/op 0.34
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 8.6920 us/op 22.435 us/op 0.39
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 98.279 us/op 152.19 us/op 0.65
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 936.36 us/op 1.2901 ms/op 0.73
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 12.312 ms/op 17.251 ms/op 0.71
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 22.569 ms/op 27.341 ms/op 0.83
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 72.049 ms/op 88.937 ms/op 0.81
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 40.797 ms/op 55.331 ms/op 0.74
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 117.59 ms/op 144.32 ms/op 0.81
Tree 40 250000 create 699.23 ms/op 1.0225 s/op 0.68
Tree 40 250000 get(125000) 275.80 ns/op 334.77 ns/op 0.82
Tree 40 250000 set(125000) 2.0799 us/op 3.0285 us/op 0.69
Tree 40 250000 toArray() 26.789 ms/op 33.783 ms/op 0.79
Tree 40 250000 iterate all - toArray() + loop 27.179 ms/op 35.296 ms/op 0.77
Tree 40 250000 iterate all - get(i) 108.31 ms/op 126.14 ms/op 0.86
MutableVector 250000 create 12.786 ms/op 16.614 ms/op 0.77
MutableVector 250000 get(125000) 10.715 ns/op 13.525 ns/op 0.79
MutableVector 250000 set(125000) 567.35 ns/op 845.46 ns/op 0.67
MutableVector 250000 toArray() 5.9861 ms/op 7.3368 ms/op 0.82
MutableVector 250000 iterate all - toArray() + loop 6.3276 ms/op 7.6059 ms/op 0.83
MutableVector 250000 iterate all - get(i) 2.7228 ms/op 3.4709 ms/op 0.78
Array 250000 create 5.9324 ms/op 7.1322 ms/op 0.83
Array 250000 clone - spread 3.3967 ms/op 4.1867 ms/op 0.81
Array 250000 get(125000) 1.5310 ns/op 1.7440 ns/op 0.88
Array 250000 set(125000) 1.5340 ns/op 1.7780 ns/op 0.86
Array 250000 iterate all - loop 153.04 us/op 137.41 us/op 1.11
effectiveBalanceIncrements clone Uint8Array 300000 144.13 us/op 113.17 us/op 1.27
effectiveBalanceIncrements clone MutableVector 300000 655.00 ns/op 1.2770 us/op 0.51
effectiveBalanceIncrements rw all Uint8Array 300000 248.66 us/op 286.55 us/op 0.87
effectiveBalanceIncrements rw all MutableVector 300000 140.44 ms/op 276.22 ms/op 0.51
phase0 afterProcessEpoch - 250000 vs - 7PWei 190.26 ms/op 206.47 ms/op 0.92
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.627 ms/op 76.951 ms/op 0.76
altair processEpoch - mainnet_e81889 549.31 ms/op 660.23 ms/op 0.83
mainnet_e81889 - altair beforeProcessEpoch 119.93 ms/op 156.18 ms/op 0.77
mainnet_e81889 - altair processJustificationAndFinalization 22.786 us/op 72.865 us/op 0.31
mainnet_e81889 - altair processInactivityUpdates 8.7509 ms/op 11.698 ms/op 0.75
mainnet_e81889 - altair processRewardsAndPenalties 77.379 ms/op 105.60 ms/op 0.73
mainnet_e81889 - altair processRegistryUpdates 4.5870 us/op 18.375 us/op 0.25
mainnet_e81889 - altair processSlashings 1.3970 us/op 4.6040 us/op 0.30
mainnet_e81889 - altair processEth1DataReset 1.4190 us/op 4.7720 us/op 0.30
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4395 ms/op 2.4133 ms/op 1.01
mainnet_e81889 - altair processSlashingsReset 7.3020 us/op 23.082 us/op 0.32
mainnet_e81889 - altair processRandaoMixesReset 7.9850 us/op 27.887 us/op 0.29
mainnet_e81889 - altair processHistoricalRootsUpdate 945.00 ns/op 4.1220 us/op 0.23
mainnet_e81889 - altair processParticipationFlagUpdates 3.4290 us/op 18.535 us/op 0.19
mainnet_e81889 - altair processSyncCommitteeUpdates 946.00 ns/op 3.5830 us/op 0.26
mainnet_e81889 - altair afterProcessEpoch 201.89 ms/op 211.89 ms/op 0.95
phase0 processEpoch - mainnet_e58758 482.92 ms/op 626.25 ms/op 0.77
mainnet_e58758 - phase0 beforeProcessEpoch 177.44 ms/op 285.78 ms/op 0.62
mainnet_e58758 - phase0 processJustificationAndFinalization 29.666 us/op 68.988 us/op 0.43
mainnet_e58758 - phase0 processRewardsAndPenalties 71.755 ms/op 157.13 ms/op 0.46
mainnet_e58758 - phase0 processRegistryUpdates 14.548 us/op 36.968 us/op 0.39
mainnet_e58758 - phase0 processSlashings 1.2100 us/op 3.7740 us/op 0.32
mainnet_e58758 - phase0 processEth1DataReset 1.0850 us/op 3.8180 us/op 0.28
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0737 ms/op 2.4555 ms/op 0.84
mainnet_e58758 - phase0 processSlashingsReset 4.6960 us/op 20.997 us/op 0.22
mainnet_e58758 - phase0 processRandaoMixesReset 7.4220 us/op 28.811 us/op 0.26
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2700 us/op 4.5820 us/op 0.28
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.5110 us/op 27.352 us/op 0.13
mainnet_e58758 - phase0 afterProcessEpoch 167.70 ms/op 174.31 ms/op 0.96
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.2058 ms/op 2.3961 ms/op 0.92
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.4152 ms/op 2.8524 ms/op 0.85
altair processInactivityUpdates - 250000 normalcase 38.602 ms/op 52.093 ms/op 0.74
altair processInactivityUpdates - 250000 worstcase 39.681 ms/op 67.390 ms/op 0.59
phase0 processRegistryUpdates - 250000 normalcase 13.208 us/op 33.216 us/op 0.40
phase0 processRegistryUpdates - 250000 badcase_full_deposits 585.11 us/op 510.75 us/op 1.15
phase0 processRegistryUpdates - 250000 worstcase 0.5 176.70 ms/op 245.52 ms/op 0.72
altair processRewardsAndPenalties - 250000 normalcase 108.36 ms/op 151.09 ms/op 0.72
altair processRewardsAndPenalties - 250000 worstcase 72.877 ms/op 98.810 ms/op 0.74
phase0 getAttestationDeltas - 250000 normalcase 11.971 ms/op 15.104 ms/op 0.79
phase0 getAttestationDeltas - 250000 worstcase 12.247 ms/op 15.673 ms/op 0.78
phase0 processSlashings - 250000 worstcase 5.6916 ms/op 7.2596 ms/op 0.78
altair processSyncCommitteeUpdates - 250000 286.60 ms/op 342.69 ms/op 0.84
BeaconState.hashTreeRoot - No change 534.00 ns/op 740.00 ns/op 0.72
BeaconState.hashTreeRoot - 1 full validator 73.206 us/op 81.661 us/op 0.90
BeaconState.hashTreeRoot - 32 full validator 651.67 us/op 840.81 us/op 0.78
BeaconState.hashTreeRoot - 512 full validator 8.4354 ms/op 7.7104 ms/op 1.09
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 85.110 us/op 129.14 us/op 0.66
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2737 ms/op 1.5287 ms/op 0.83
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.464 ms/op 18.943 ms/op 0.87
BeaconState.hashTreeRoot - 1 balances 63.193 us/op 77.637 us/op 0.81
BeaconState.hashTreeRoot - 32 balances 618.17 us/op 784.71 us/op 0.79
BeaconState.hashTreeRoot - 512 balances 6.0828 ms/op 6.9252 ms/op 0.88
BeaconState.hashTreeRoot - 250000 balances 101.12 ms/op 116.06 ms/op 0.87
aggregationBits - 2048 els - zipIndexesInBitList 27.039 us/op 37.999 us/op 0.71
regular array get 100000 times 62.121 us/op 54.712 us/op 1.14
wrappedArray get 100000 times 60.927 us/op 56.565 us/op 1.08
arrayWithProxy get 100000 times 28.047 ms/op 33.007 ms/op 0.85
ssz.Root.equals 472.00 ns/op 562.00 ns/op 0.84
byteArrayEquals 454.00 ns/op 575.00 ns/op 0.79
shuffle list - 16384 els 11.802 ms/op 12.242 ms/op 0.96
shuffle list - 250000 els 172.55 ms/op 173.84 ms/op 0.99
processSlot - 1 slots 13.151 us/op 19.449 us/op 0.68
processSlot - 32 slots 1.9512 ms/op 2.5994 ms/op 0.75
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 402.65 us/op 578.32 us/op 0.70
getCommitteeAssignments - req 1 vs - 250000 vc 5.3754 ms/op 5.5800 ms/op 0.96
getCommitteeAssignments - req 100 vs - 250000 vc 7.8682 ms/op 7.8092 ms/op 1.01
getCommitteeAssignments - req 1000 vs - 250000 vc 8.4041 ms/op 8.7725 ms/op 0.96
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.8900 ns/op 9.4900 ns/op 0.83
state getBlockRootAtSlot - 250000 vs - 7PWei 951.89 ns/op 1.3194 us/op 0.72
computeProposers - vc 250000 16.687 ms/op 21.538 ms/op 0.77
computeEpochShuffling - vc 250000 175.18 ms/op 189.10 ms/op 0.93
getNextSyncCommittee - vc 250000 279.25 ms/op 353.99 ms/op 0.79

by benchmarkbot/action

g11tech
g11tech approved these changes Dec 7, 2022
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@g11tech g11tech merged commit 0b31818 into unstable Dec 7, 2022
@g11tech g11tech deleted the dapplion/eip-4844-gossip-handler branch December 7, 2022 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@g11tech g11tech g11tech approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

spec-deneb 🐡 Issues targeting the Deneb spec version

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dapplion @g11tech