Skip to content

EIP-4844: Force ExternalData arg in state transition fn#4846

Merged
twoeths merged 4 commits intounstablefrom
dapplion/stfn-external-data
Dec 7, 2022
Merged

EIP-4844: Force ExternalData arg in state transition fn#4846
twoeths merged 4 commits intounstablefrom
dapplion/stfn-external-data

Conversation

@dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 5, 2022 •
edited
Loading

Motivation

State transition function will have two external dependencies: Execution Engine, Data Availability. Current way to handle those is obscure, and optional struct that returns "is payload valid?".

I believe it's necessary to be clear and explicit about this external dependencies and think on each state transition call what are their values

Description

  • Explicitly declare external data of a block on each state transition function call

@dapplion dapplion requested a review from a team as a code owner December 5, 2022 08:35
@dapplion dapplion changed the title Dapplion/stfn external data EIP-4844: Force ExternalData arg in state transition fn Dec 5, 2022
@dapplion dapplion added the spec-deneb 🐡 Issues targeting the Deneb spec version label Dec 5, 2022
@dapplion dapplion force-pushed the dapplion/stfn-external-data branch from 6c4ca70 to f9c503c Compare December 6, 2022 08:26
@github-actions
Copy link
Contributor

github-actions bot commented Dec 6, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 4a9cd19 Previous: 120231d Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.6314 ms/op 1.7096 ms/op 0.95
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 61.823 us/op 59.466 us/op 1.04
BLS verify - blst-native 2.1706 ms/op 2.1697 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 4.4970 ms/op 4.4949 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 9.7015 ms/op 9.7005 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 35.277 ms/op 35.248 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 46.737 us/op 46.958 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 182.28 us/op 182.03 us/op 1.00
getAttestationsForBlock 76.508 ms/op 76.291 ms/op 1.00
isKnown best case - 1 super set check 476.00 ns/op 475.00 ns/op 1.00
isKnown normal case - 2 super set checks 469.00 ns/op 463.00 ns/op 1.01
isKnown worse case - 16 super set checks 465.00 ns/op 464.00 ns/op 1.00
CheckpointStateCache - add get delete 8.8760 us/op 8.7350 us/op 1.02
validate gossip signedAggregateAndProof - struct 5.0443 ms/op 5.0359 ms/op 1.00
validate gossip attestation - struct 2.3672 ms/op 2.3707 ms/op 1.00
pickEth1Vote - no votes 2.1093 ms/op 2.1010 ms/op 1.00
pickEth1Vote - max votes 17.781 ms/op 17.334 ms/op 1.03
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.113 ms/op 11.977 ms/op 1.01
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.739 ms/op 19.086 ms/op 1.03
pickEth1Vote - Eth1Data fastSerialize value x2048 1.4258 ms/op 1.3932 ms/op 1.02
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.168 ms/op 11.384 ms/op 1.07
bytes32 toHexString 953.00 ns/op 962.00 ns/op 0.99
bytes32 Buffer.toString(hex) 703.00 ns/op 739.00 ns/op 0.95
bytes32 Buffer.toString(hex) from Uint8Array 972.00 ns/op 966.00 ns/op 1.01
bytes32 Buffer.toString(hex) + 0x 727.00 ns/op 747.00 ns/op 0.97
Object access 1 prop 0.34800 ns/op 0.37700 ns/op 0.92
Map access 1 prop 0.31300 ns/op 0.31100 ns/op 1.01
Object get x1000 10.944 ns/op 11.029 ns/op 0.99
Map get x1000 0.94900 ns/op 0.93200 ns/op 1.02
Object set x1000 70.435 ns/op 71.212 ns/op 0.99
Map set x1000 47.421 ns/op 45.450 ns/op 1.04
Return object 10000 times 0.43310 ns/op 0.43970 ns/op 0.98
Throw Error 10000 times 5.9276 us/op 5.9739 us/op 0.99
fastMsgIdFn sha256 / 200 bytes 4.8930 us/op 4.8120 us/op 1.02
fastMsgIdFn h32 xxhash / 200 bytes 553.00 ns/op 550.00 ns/op 1.01
fastMsgIdFn h64 xxhash / 200 bytes 821.00 ns/op 741.00 ns/op 1.11
fastMsgIdFn sha256 / 1000 bytes 15.459 us/op 15.477 us/op 1.00
fastMsgIdFn h32 xxhash / 1000 bytes 731.00 ns/op 701.00 ns/op 1.04
fastMsgIdFn h64 xxhash / 1000 bytes 868.00 ns/op 846.00 ns/op 1.03
fastMsgIdFn sha256 / 10000 bytes 134.91 us/op 133.70 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 2.5610 us/op 2.6520 us/op 0.97
fastMsgIdFn h64 xxhash / 10000 bytes 1.9550 us/op 1.8990 us/op 1.03
enrSubnets - fastDeserialize 64 bits 2.6670 us/op 2.6030 us/op 1.02
enrSubnets - ssz BitVector 64 bits 774.00 ns/op 802.00 ns/op 0.97
enrSubnets - fastDeserialize 4 bits 377.00 ns/op 346.00 ns/op 1.09
enrSubnets - ssz BitVector 4 bits 762.00 ns/op 784.00 ns/op 0.97
prioritizePeers score -10:0 att 32-0.1 sync 2-0 81.024 us/op 79.648 us/op 1.02
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 132.50 us/op 119.30 us/op 1.11
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 203.26 us/op 191.11 us/op 1.06
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 359.32 us/op 389.27 us/op 0.92
prioritizePeers score 0:0 att 64-1 sync 4-1 435.84 us/op 440.56 us/op 0.99
RateTracker 1000000 limit, 1 obj count per request 183.09 ns/op 181.21 ns/op 1.01
RateTracker 1000000 limit, 2 obj count per request 133.55 ns/op 132.53 ns/op 1.01
RateTracker 1000000 limit, 4 obj count per request 109.45 ns/op 108.39 ns/op 1.01
RateTracker 1000000 limit, 8 obj count per request 96.511 ns/op 95.898 ns/op 1.01
RateTracker with prune 3.8440 us/op 3.8540 us/op 1.00
array of 16000 items push then shift 51.590 us/op 51.601 us/op 1.00
LinkedList of 16000 items push then shift 12.296 ns/op 12.595 ns/op 0.98
array of 16000 items push then pop 203.26 ns/op 207.67 ns/op 0.98
LinkedList of 16000 items push then pop 11.991 ns/op 11.968 ns/op 1.00
array of 24000 items push then shift 77.359 us/op 77.365 us/op 1.00
LinkedList of 24000 items push then shift 12.401 ns/op 12.771 ns/op 0.97
array of 24000 items push then pop 191.26 ns/op 191.53 ns/op 1.00
LinkedList of 24000 items push then pop 11.899 ns/op 12.670 ns/op 0.94
intersect bitArray bitLen 8 11.405 ns/op 10.884 ns/op 1.05
intersect array and set length 8 131.47 ns/op 151.52 ns/op 0.87
intersect bitArray bitLen 128 57.958 ns/op 64.999 ns/op 0.89
intersect array and set length 128 1.7163 us/op 2.0588 us/op 0.83
Buffer.concat 32 items 1.7750 ns/op 1.9030 ns/op 0.93
pass gossip attestations to forkchoice per slot 5.0332 ms/op 3.9752 ms/op 1.27
computeDeltas 4.6461 ms/op 4.9558 ms/op 0.94
computeProposerBoostScoreFromBalances 803.80 us/op 806.57 us/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 3.2228 ms/op 3.3488 ms/op 0.96
altair processAttestation - 250000 vs - 7PWei worstcase 5.0019 ms/op 5.4231 ms/op 0.92
altair processAttestation - setStatus - 1/6 committees join 174.76 us/op 185.19 us/op 0.94
altair processAttestation - setStatus - 1/3 committees join 343.36 us/op 364.24 us/op 0.94
altair processAttestation - setStatus - 1/2 committees join 495.88 us/op 529.22 us/op 0.94
altair processAttestation - setStatus - 2/3 committees join 650.82 us/op 693.92 us/op 0.94
altair processAttestation - setStatus - 4/5 committees join 913.74 us/op 970.60 us/op 0.94
altair processAttestation - setStatus - 100% committees join 1.1067 ms/op 1.1655 ms/op 0.95
altair processBlock - 250000 vs - 7PWei normalcase 24.766 ms/op 25.893 ms/op 0.96
altair processBlock - 250000 vs - 7PWei normalcase hashState 34.384 ms/op 38.722 ms/op 0.89
altair processBlock - 250000 vs - 7PWei worstcase 83.882 ms/op 77.125 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase hashState 99.876 ms/op 111.67 ms/op 0.89
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2820 ms/op 3.2721 ms/op 1.00
phase0 processBlock - 250000 vs - 7PWei worstcase 50.542 ms/op 51.908 ms/op 0.97
altair processEth1Data - 250000 vs - 7PWei normalcase 793.46 us/op 714.94 us/op 1.11
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 11.092 us/op 6.4860 us/op 1.71
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 22.112 us/op 22.248 us/op 0.99
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 11.549 us/op 8.9660 us/op 1.29
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.9150 us/op 9.4490 us/op 0.73
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 106.17 us/op 112.83 us/op 0.94
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 959.53 us/op 906.89 us/op 1.06
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 12.130 ms/op 12.145 ms/op 1.00
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 22.066 ms/op 21.587 ms/op 1.02
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 80.501 ms/op 71.959 ms/op 1.12
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 37.151 ms/op 37.651 ms/op 0.99
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 123.86 ms/op 121.38 ms/op 1.02
Tree 40 250000 create 705.92 ms/op 689.02 ms/op 1.02
Tree 40 250000 get(125000) 245.26 ns/op 221.97 ns/op 1.10
Tree 40 250000 set(125000) 2.1040 us/op 2.3417 us/op 0.90
Tree 40 250000 toArray() 26.602 ms/op 26.987 ms/op 0.99
Tree 40 250000 iterate all - toArray() + loop 26.856 ms/op 26.858 ms/op 1.00
Tree 40 250000 iterate all - get(i) 108.93 ms/op 109.75 ms/op 0.99
MutableVector 250000 create 12.672 ms/op 12.675 ms/op 1.00
MutableVector 250000 get(125000) 11.008 ns/op 10.640 ns/op 1.03
MutableVector 250000 set(125000) 545.24 ns/op 589.54 ns/op 0.92
MutableVector 250000 toArray() 6.0418 ms/op 6.3417 ms/op 0.95
MutableVector 250000 iterate all - toArray() + loop 5.9955 ms/op 5.5521 ms/op 1.08
MutableVector 250000 iterate all - get(i) 2.6185 ms/op 2.5974 ms/op 1.01
Array 250000 create 5.6749 ms/op 5.3036 ms/op 1.07
Array 250000 clone - spread 3.0941 ms/op 2.4347 ms/op 1.27
Array 250000 get(125000) 1.4320 ns/op 1.1660 ns/op 1.23
Array 250000 set(125000) 1.4320 ns/op 1.1550 ns/op 1.24
Array 250000 iterate all - loop 152.68 us/op 151.66 us/op 1.01
effectiveBalanceIncrements clone Uint8Array 300000 141.12 us/op 164.89 us/op 0.86
effectiveBalanceIncrements clone MutableVector 300000 612.00 ns/op 602.00 ns/op 1.02
effectiveBalanceIncrements rw all Uint8Array 300000 247.25 us/op 247.56 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 139.88 ms/op 140.90 ms/op 0.99
phase0 afterProcessEpoch - 250000 vs - 7PWei 199.50 ms/op 198.92 ms/op 1.00
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.252 ms/op 57.459 ms/op 1.01
altair processEpoch - mainnet_e81889 547.43 ms/op 547.07 ms/op 1.00
mainnet_e81889 - altair beforeProcessEpoch 109.71 ms/op 133.18 ms/op 0.82
mainnet_e81889 - altair processJustificationAndFinalization 27.764 us/op 17.333 us/op 1.60
mainnet_e81889 - altair processInactivityUpdates 8.7642 ms/op 9.0697 ms/op 0.97
mainnet_e81889 - altair processRewardsAndPenalties 121.14 ms/op 79.597 ms/op 1.52
mainnet_e81889 - altair processRegistryUpdates 4.5520 us/op 2.6930 us/op 1.69
mainnet_e81889 - altair processSlashings 1.2550 us/op 642.00 ns/op 1.95
mainnet_e81889 - altair processEth1DataReset 1.1910 us/op 1.0540 us/op 1.13
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.0647 ms/op 2.2102 ms/op 0.93
mainnet_e81889 - altair processSlashingsReset 7.5190 us/op 5.1490 us/op 1.46
mainnet_e81889 - altair processRandaoMixesReset 7.6370 us/op 4.2280 us/op 1.81
mainnet_e81889 - altair processHistoricalRootsUpdate 1.3940 us/op 832.00 ns/op 1.68
mainnet_e81889 - altair processParticipationFlagUpdates 4.9670 us/op 2.5550 us/op 1.94
mainnet_e81889 - altair processSyncCommitteeUpdates 1.1850 us/op 914.00 ns/op 1.30
mainnet_e81889 - altair afterProcessEpoch 221.69 ms/op 200.23 ms/op 1.11
phase0 processEpoch - mainnet_e58758 614.41 ms/op 483.36 ms/op 1.27
mainnet_e58758 - phase0 beforeProcessEpoch 225.05 ms/op 176.78 ms/op 1.27
mainnet_e58758 - phase0 processJustificationAndFinalization 28.399 us/op 20.799 us/op 1.37
mainnet_e58758 - phase0 processRewardsAndPenalties 128.04 ms/op 111.59 ms/op 1.15
mainnet_e58758 - phase0 processRegistryUpdates 13.784 us/op 8.2180 us/op 1.68
mainnet_e58758 - phase0 processSlashings 1.1610 us/op 1.0420 us/op 1.11
mainnet_e58758 - phase0 processEth1DataReset 1.2550 us/op 1.1310 us/op 1.11
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0093 ms/op 2.0597 ms/op 0.98
mainnet_e58758 - phase0 processSlashingsReset 6.9500 us/op 6.0030 us/op 1.16
mainnet_e58758 - phase0 processRandaoMixesReset 8.1310 us/op 7.7930 us/op 1.04
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2900 us/op 1.1210 us/op 1.15
mainnet_e58758 - phase0 processParticipationRecordUpdates 6.1050 us/op 7.0190 us/op 0.87
mainnet_e58758 - phase0 afterProcessEpoch 164.56 ms/op 163.67 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.2139 ms/op 2.0783 ms/op 1.07
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.3373 ms/op 2.3178 ms/op 1.01
altair processInactivityUpdates - 250000 normalcase 52.884 ms/op 39.352 ms/op 1.34
altair processInactivityUpdates - 250000 worstcase 53.894 ms/op 39.571 ms/op 1.36
phase0 processRegistryUpdates - 250000 normalcase 12.480 us/op 12.260 us/op 1.02
phase0 processRegistryUpdates - 250000 badcase_full_deposits 448.39 us/op 582.23 us/op 0.77
phase0 processRegistryUpdates - 250000 worstcase 0.5 225.64 ms/op 172.84 ms/op 1.31
altair processRewardsAndPenalties - 250000 normalcase 137.51 ms/op 78.662 ms/op 1.75
altair processRewardsAndPenalties - 250000 worstcase 138.85 ms/op 74.990 ms/op 1.85
phase0 getAttestationDeltas - 250000 normalcase 11.521 ms/op 11.803 ms/op 0.98
phase0 getAttestationDeltas - 250000 worstcase 11.759 ms/op 11.879 ms/op 0.99
phase0 processSlashings - 250000 worstcase 5.4741 ms/op 5.6509 ms/op 0.97
altair processSyncCommitteeUpdates - 250000 297.25 ms/op 283.53 ms/op 1.05
BeaconState.hashTreeRoot - No change 505.00 ns/op 545.00 ns/op 0.93
BeaconState.hashTreeRoot - 1 full validator 72.072 us/op 66.158 us/op 1.09
BeaconState.hashTreeRoot - 32 full validator 812.59 us/op 690.90 us/op 1.18
BeaconState.hashTreeRoot - 512 full validator 6.8896 ms/op 6.9787 ms/op 0.99
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 103.15 us/op 87.955 us/op 1.17
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2873 ms/op 1.2671 ms/op 1.02
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 18.506 ms/op 17.848 ms/op 1.04
BeaconState.hashTreeRoot - 1 balances 69.498 us/op 66.249 us/op 1.05
BeaconState.hashTreeRoot - 32 balances 716.83 us/op 661.10 us/op 1.08
BeaconState.hashTreeRoot - 512 balances 6.9047 ms/op 6.2152 ms/op 1.11
BeaconState.hashTreeRoot - 250000 balances 109.51 ms/op 93.772 ms/op 1.17
aggregationBits - 2048 els - zipIndexesInBitList 22.250 us/op 27.042 us/op 0.82
regular array get 100000 times 61.358 us/op 62.064 us/op 0.99
wrappedArray get 100000 times 62.967 us/op 61.425 us/op 1.03
arrayWithProxy get 100000 times 27.830 ms/op 28.542 ms/op 0.98
ssz.Root.equals 460.00 ns/op 469.00 ns/op 0.98
byteArrayEquals 449.00 ns/op 458.00 ns/op 0.98
shuffle list - 16384 els 11.335 ms/op 11.637 ms/op 0.97
shuffle list - 250000 els 168.07 ms/op 168.31 ms/op 1.00
processSlot - 1 slots 13.250 us/op 12.521 us/op 1.06
processSlot - 32 slots 1.9314 ms/op 1.9172 ms/op 1.01
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 369.37 us/op 369.14 us/op 1.00
getCommitteeAssignments - req 1 vs - 250000 vc 5.3753 ms/op 5.3451 ms/op 1.01
getCommitteeAssignments - req 100 vs - 250000 vc 7.8948 ms/op 7.8475 ms/op 1.01
getCommitteeAssignments - req 1000 vs - 250000 vc 8.4206 ms/op 8.4429 ms/op 1.00
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 8.4500 ns/op 7.5900 ns/op 1.11
state getBlockRootAtSlot - 250000 vs - 7PWei 952.66 ns/op 993.91 ns/op 0.96
computeProposers - vc 250000 17.631 ms/op 16.747 ms/op 1.05
computeEpochShuffling - vc 250000 174.63 ms/op 172.08 ms/op 1.01
getNextSyncCommittee - vc 250000 288.71 ms/op 289.05 ms/op 1.00

by benchmarkbot/action

wemeetagain
wemeetagain previously approved these changes Dec 6, 2022
View reviewed changes
@twoeths twoeths force-pushed the dapplion/stfn-external-data branch from f9c503c to fc3fe61 Compare December 7, 2022 01:48
@twoeths twoeths merged commit b7c7910 into unstable Dec 7, 2022
@twoeths twoeths deleted the dapplion/stfn-external-data branch December 7, 2022 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@twoeths twoeths twoeths approved these changes

@wemeetagain wemeetagain wemeetagain left review comments

Assignees

No one assigned

Labels

spec-deneb 🐡 Issues targeting the Deneb spec version

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dapplion @wemeetagain @twoeths