Skip to content

Fix CONTRIBUTING.md spec tests instructions#4840

Merged
wemeetagain merged 2 commits intounstablefrom
dapplion/contributing-test-instructions
Dec 5, 2022
Merged

Fix CONTRIBUTING.md spec tests instructions#4840
wemeetagain merged 2 commits intounstablefrom
dapplion/contributing-test-instructions

Conversation

@dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 5, 2022

Motivation

Description

  • Fix CONTRIBUTING.md spec tests instructions

@dapplion dapplion requested a review from a team as a code owner December 5, 2022 06:45
@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 52aee20 Previous: 1fde79d Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.4626 ms/op 2.3952 ms/op 1.03
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 77.941 us/op 79.363 us/op 0.98
BLS verify - blst-native 1.8514 ms/op 1.9986 ms/op 0.93
BLS verifyMultipleSignatures 3 - blst-native 3.8194 ms/op 4.2332 ms/op 0.90
BLS verifyMultipleSignatures 8 - blst-native 8.2060 ms/op 8.8027 ms/op 0.93
BLS verifyMultipleSignatures 32 - blst-native 29.796 ms/op 34.314 ms/op 0.87
BLS aggregatePubkeys 32 - blst-native 39.454 us/op 48.821 us/op 0.81
BLS aggregatePubkeys 128 - blst-native 153.52 us/op 163.45 us/op 0.94
getAttestationsForBlock 102.80 ms/op 102.00 ms/op 1.01
isKnown best case - 1 super set check 457.00 ns/op 436.00 ns/op 1.05
isKnown normal case - 2 super set checks 434.00 ns/op 421.00 ns/op 1.03
isKnown worse case - 16 super set checks 433.00 ns/op 442.00 ns/op 0.98
CheckpointStateCache - add get delete 9.0710 us/op 9.4070 us/op 0.96
validate gossip signedAggregateAndProof - struct 4.2930 ms/op 4.7969 ms/op 0.89
validate gossip attestation - struct 2.0370 ms/op 2.0977 ms/op 0.97
pickEth1Vote - no votes 2.1153 ms/op 2.0058 ms/op 1.05
pickEth1Vote - max votes 23.263 ms/op 21.064 ms/op 1.10
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.747 ms/op 11.254 ms/op 1.04
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 22.998 ms/op 21.036 ms/op 1.09
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5931 ms/op 1.6766 ms/op 0.95
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.959 ms/op 15.477 ms/op 0.97
bytes32 toHexString 1.1370 us/op 1.1290 us/op 1.01
bytes32 Buffer.toString(hex) 709.00 ns/op 746.00 ns/op 0.95
bytes32 Buffer.toString(hex) from Uint8Array 965.00 ns/op 1.0470 us/op 0.92
bytes32 Buffer.toString(hex) + 0x 704.00 ns/op 688.00 ns/op 1.02
Object access 1 prop 0.37300 ns/op 0.34100 ns/op 1.09
Map access 1 prop 0.28500 ns/op 0.27500 ns/op 1.04
Object get x1000 16.979 ns/op 13.368 ns/op 1.27
Map get x1000 0.96700 ns/op 0.75700 ns/op 1.28
Object set x1000 124.36 ns/op 85.592 ns/op 1.45
Map set x1000 77.258 ns/op 63.109 ns/op 1.22
Return object 10000 times 0.37630 ns/op 0.39670 ns/op 0.95
Throw Error 10000 times 5.9317 us/op 6.4669 us/op 0.92
fastMsgIdFn sha256 / 200 bytes 4.1940 us/op 5.1520 us/op 0.81
fastMsgIdFn h32 xxhash / 200 bytes 556.00 ns/op 579.00 ns/op 0.96
fastMsgIdFn h64 xxhash / 200 bytes 700.00 ns/op 721.00 ns/op 0.97
fastMsgIdFn sha256 / 1000 bytes 13.111 us/op 14.306 us/op 0.92
fastMsgIdFn h32 xxhash / 1000 bytes 712.00 ns/op 667.00 ns/op 1.07
fastMsgIdFn h64 xxhash / 1000 bytes 778.00 ns/op 881.00 ns/op 0.88
fastMsgIdFn sha256 / 10000 bytes 112.12 us/op 120.15 us/op 0.93
fastMsgIdFn h32 xxhash / 10000 bytes 2.3280 us/op 2.2780 us/op 1.02
fastMsgIdFn h64 xxhash / 10000 bytes 1.7360 us/op 1.7690 us/op 0.98
enrSubnets - fastDeserialize 64 bits 2.8850 us/op 2.7610 us/op 1.04
enrSubnets - ssz BitVector 64 bits 788.00 ns/op 861.00 ns/op 0.92
enrSubnets - fastDeserialize 4 bits 394.00 ns/op 364.00 ns/op 1.08
enrSubnets - ssz BitVector 4 bits 788.00 ns/op 695.00 ns/op 1.13
prioritizePeers score -10:0 att 32-0.1 sync 2-0 97.168 us/op 90.494 us/op 1.07
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 136.84 us/op 122.51 us/op 1.12
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 237.71 us/op 228.55 us/op 1.04
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 385.63 us/op 498.16 us/op 0.77
prioritizePeers score 0:0 att 64-1 sync 4-1 458.54 us/op 482.54 us/op 0.95
RateTracker 1000000 limit, 1 obj count per request 196.01 ns/op 163.35 ns/op 1.20
RateTracker 1000000 limit, 2 obj count per request 148.19 ns/op 116.16 ns/op 1.28
RateTracker 1000000 limit, 4 obj count per request 124.63 ns/op 97.823 ns/op 1.27
RateTracker 1000000 limit, 8 obj count per request 111.88 ns/op 89.142 ns/op 1.26
RateTracker with prune 4.6270 us/op 4.0900 us/op 1.13
array of 16000 items push then shift 2.7983 us/op 4.2745 us/op 0.65
LinkedList of 16000 items push then shift 17.794 ns/op 14.993 ns/op 1.19
array of 16000 items push then pop 240.04 ns/op 212.11 ns/op 1.13
LinkedList of 16000 items push then pop 16.692 ns/op 13.661 ns/op 1.22
array of 24000 items push then shift 3.9948 us/op 6.3622 us/op 0.63
LinkedList of 24000 items push then shift 21.092 ns/op 16.140 ns/op 1.31
array of 24000 items push then pop 243.66 ns/op 203.90 ns/op 1.20
LinkedList of 24000 items push then pop 18.207 ns/op 14.571 ns/op 1.25
intersect bitArray bitLen 8 11.399 ns/op 9.7590 ns/op 1.17
intersect array and set length 8 178.91 ns/op 152.06 ns/op 1.18
intersect bitArray bitLen 128 71.904 ns/op 57.488 ns/op 1.25
intersect array and set length 128 2.3438 us/op 1.7687 us/op 1.33
Buffer.concat 32 items 2.2670 ns/op 1.8720 ns/op 1.21
pass gossip attestations to forkchoice per slot 7.4902 ms/op 3.5583 ms/op 2.11
computeDeltas 6.0419 ms/op 5.1914 ms/op 1.16
computeProposerBoostScoreFromBalances 907.71 us/op 712.25 us/op 1.27
altair processAttestation - 250000 vs - 7PWei normalcase 4.1249 ms/op 3.7455 ms/op 1.10
altair processAttestation - 250000 vs - 7PWei worstcase 6.7607 ms/op 5.8901 ms/op 1.15
altair processAttestation - setStatus - 1/6 committees join 218.72 us/op 200.75 us/op 1.09
altair processAttestation - setStatus - 1/3 committees join 417.99 us/op 402.99 us/op 1.04
altair processAttestation - setStatus - 1/2 committees join 586.02 us/op 575.29 us/op 1.02
altair processAttestation - setStatus - 2/3 committees join 749.66 us/op 757.32 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 1.0436 ms/op 1.0110 ms/op 1.03
altair processAttestation - setStatus - 100% committees join 1.2360 ms/op 1.3092 ms/op 0.94
altair processBlock - 250000 vs - 7PWei normalcase 29.997 ms/op 26.738 ms/op 1.12
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.708 ms/op 43.711 ms/op 0.91
altair processBlock - 250000 vs - 7PWei worstcase 87.151 ms/op 79.531 ms/op 1.10
altair processBlock - 250000 vs - 7PWei worstcase hashState 102.89 ms/op 98.296 ms/op 1.05
phase0 processBlock - 250000 vs - 7PWei normalcase 4.0994 ms/op 3.5202 ms/op 1.16
phase0 processBlock - 250000 vs - 7PWei worstcase 47.813 ms/op 43.190 ms/op 1.11
altair processEth1Data - 250000 vs - 7PWei normalcase 929.50 us/op 734.92 us/op 1.26
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 11.380 us/op 20.075 us/op 0.57
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 28.604 us/op 45.087 us/op 0.63
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 13.854 us/op 25.600 us/op 0.54
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 10.360 us/op 21.210 us/op 0.49
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 116.78 us/op 134.01 us/op 0.87
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 1.1000 ms/op 1.1139 ms/op 0.99
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 14.569 ms/op 13.075 ms/op 1.11
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 25.789 ms/op 23.415 ms/op 1.10
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 85.530 ms/op 76.872 ms/op 1.11
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 47.134 ms/op 44.208 ms/op 1.07
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 129.56 ms/op 123.31 ms/op 1.05
Tree 40 250000 create 853.28 ms/op 757.50 ms/op 1.13
Tree 40 250000 get(125000) 290.81 ns/op 257.99 ns/op 1.13
Tree 40 250000 set(125000) 2.5491 us/op 2.1617 us/op 1.18
Tree 40 250000 toArray() 32.123 ms/op 32.374 ms/op 0.99
Tree 40 250000 iterate all - toArray() + loop 32.532 ms/op 32.960 ms/op 0.99
Tree 40 250000 iterate all - get(i) 112.02 ms/op 108.83 ms/op 1.03
MutableVector 250000 create 17.391 ms/op 13.369 ms/op 1.30
MutableVector 250000 get(125000) 14.812 ns/op 11.575 ns/op 1.28
MutableVector 250000 set(125000) 733.48 ns/op 545.94 ns/op 1.34
MutableVector 250000 toArray() 8.5142 ms/op 6.2579 ms/op 1.36
MutableVector 250000 iterate all - toArray() + loop 8.2602 ms/op 6.2043 ms/op 1.33
MutableVector 250000 iterate all - get(i) 3.4405 ms/op 2.6749 ms/op 1.29
Array 250000 create 7.2442 ms/op 5.6554 ms/op 1.28
Array 250000 clone - spread 5.5391 ms/op 3.1076 ms/op 1.78
Array 250000 get(125000) 1.9190 ns/op 1.3990 ns/op 1.37
Array 250000 set(125000) 1.8520 ns/op 1.3240 ns/op 1.40
Array 250000 iterate all - loop 167.86 us/op 119.56 us/op 1.40
effectiveBalanceIncrements clone Uint8Array 300000 109.85 us/op 64.298 us/op 1.71
effectiveBalanceIncrements clone MutableVector 300000 1.3630 us/op 934.00 ns/op 1.46
effectiveBalanceIncrements rw all Uint8Array 300000 252.47 us/op 242.30 us/op 1.04
effectiveBalanceIncrements rw all MutableVector 300000 266.02 ms/op 192.29 ms/op 1.38
phase0 afterProcessEpoch - 250000 vs - 7PWei 187.52 ms/op 165.34 ms/op 1.13
phase0 beforeProcessEpoch - 250000 vs - 7PWei 70.957 ms/op 99.949 ms/op 0.71
altair processEpoch - mainnet_e81889 600.84 ms/op 538.90 ms/op 1.11
mainnet_e81889 - altair beforeProcessEpoch 163.50 ms/op 130.98 ms/op 1.25
mainnet_e81889 - altair processJustificationAndFinalization 40.520 us/op 69.371 us/op 0.58
mainnet_e81889 - altair processInactivityUpdates 11.227 ms/op 9.3178 ms/op 1.20
mainnet_e81889 - altair processRewardsAndPenalties 96.077 ms/op 89.595 ms/op 1.07
mainnet_e81889 - altair processRegistryUpdates 6.7800 us/op 14.945 us/op 0.45
mainnet_e81889 - altair processSlashings 1.9780 us/op 4.1410 us/op 0.48
mainnet_e81889 - altair processEth1DataReset 2.0660 us/op 4.3760 us/op 0.47
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.5175 ms/op 1.9670 ms/op 1.28
mainnet_e81889 - altair processSlashingsReset 10.257 us/op 27.487 us/op 0.37
mainnet_e81889 - altair processRandaoMixesReset 12.075 us/op 25.159 us/op 0.48
mainnet_e81889 - altair processHistoricalRootsUpdate 1.9870 us/op 3.9940 us/op 0.50
mainnet_e81889 - altair processParticipationFlagUpdates 5.9850 us/op 15.278 us/op 0.39
mainnet_e81889 - altair processSyncCommitteeUpdates 659.00 ns/op 3.5080 us/op 0.19
mainnet_e81889 - altair afterProcessEpoch 196.81 ms/op 172.46 ms/op 1.14
phase0 processEpoch - mainnet_e58758 552.22 ms/op 493.85 ms/op 1.12
mainnet_e58758 - phase0 beforeProcessEpoch 248.31 ms/op 260.13 ms/op 0.95
mainnet_e58758 - phase0 processJustificationAndFinalization 33.509 us/op 63.352 us/op 0.53
mainnet_e58758 - phase0 processRewardsAndPenalties 147.52 ms/op 112.07 ms/op 1.32
mainnet_e58758 - phase0 processRegistryUpdates 16.813 us/op 34.207 us/op 0.49
mainnet_e58758 - phase0 processSlashings 1.5190 us/op 3.9930 us/op 0.38
mainnet_e58758 - phase0 processEth1DataReset 2.0260 us/op 3.8360 us/op 0.53
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0491 ms/op 1.9498 ms/op 1.05
mainnet_e58758 - phase0 processSlashingsReset 9.9720 us/op 19.124 us/op 0.52
mainnet_e58758 - phase0 processRandaoMixesReset 11.813 us/op 26.058 us/op 0.45
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.5670 us/op 4.3150 us/op 0.36
mainnet_e58758 - phase0 processParticipationRecordUpdates 10.823 us/op 24.125 us/op 0.45
mainnet_e58758 - phase0 afterProcessEpoch 163.68 ms/op 134.75 ms/op 1.21
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.6346 ms/op 1.9822 ms/op 1.33
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.6112 ms/op 2.2041 ms/op 1.64
altair processInactivityUpdates - 250000 normalcase 39.957 ms/op 47.645 ms/op 0.84
altair processInactivityUpdates - 250000 worstcase 47.690 ms/op 55.703 ms/op 0.86
phase0 processRegistryUpdates - 250000 normalcase 13.884 us/op 29.695 us/op 0.47
phase0 processRegistryUpdates - 250000 badcase_full_deposits 451.45 us/op 448.64 us/op 1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5 223.08 ms/op 228.74 ms/op 0.98
altair processRewardsAndPenalties - 250000 normalcase 90.747 ms/op 135.80 ms/op 0.67
altair processRewardsAndPenalties - 250000 worstcase 136.65 ms/op 80.118 ms/op 1.71
phase0 getAttestationDeltas - 250000 normalcase 13.987 ms/op 11.512 ms/op 1.22
phase0 getAttestationDeltas - 250000 worstcase 13.609 ms/op 11.690 ms/op 1.16
phase0 processSlashings - 250000 worstcase 6.3545 ms/op 5.8091 ms/op 1.09
altair processSyncCommitteeUpdates - 250000 286.92 ms/op 267.40 ms/op 1.07
BeaconState.hashTreeRoot - No change 498.00 ns/op 460.00 ns/op 1.08
BeaconState.hashTreeRoot - 1 full validator 64.953 us/op 61.382 us/op 1.06
BeaconState.hashTreeRoot - 32 full validator 657.08 us/op 624.11 us/op 1.05
BeaconState.hashTreeRoot - 512 full validator 7.0083 ms/op 6.7505 ms/op 1.04
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 86.478 us/op 84.040 us/op 1.03
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2466 ms/op 1.1719 ms/op 1.06
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 17.050 ms/op 14.657 ms/op 1.16
BeaconState.hashTreeRoot - 1 balances 63.539 us/op 58.336 us/op 1.09
BeaconState.hashTreeRoot - 32 balances 689.97 us/op 561.78 us/op 1.23
BeaconState.hashTreeRoot - 512 balances 6.1132 ms/op 5.7629 ms/op 1.06
BeaconState.hashTreeRoot - 250000 balances 89.496 ms/op 88.042 ms/op 1.02
aggregationBits - 2048 els - zipIndexesInBitList 37.840 us/op 21.679 us/op 1.75
regular array get 100000 times 67.412 us/op 45.457 us/op 1.48
wrappedArray get 100000 times 67.432 us/op 46.295 us/op 1.46
arrayWithProxy get 100000 times 28.459 ms/op 27.715 ms/op 1.03
ssz.Root.equals 509.00 ns/op 466.00 ns/op 1.09
byteArrayEquals 514.00 ns/op 413.00 ns/op 1.24
shuffle list - 16384 els 11.336 ms/op 9.1569 ms/op 1.24
shuffle list - 250000 els 165.85 ms/op 137.88 ms/op 1.20
processSlot - 1 slots 13.890 us/op 13.045 us/op 1.06
processSlot - 32 slots 1.9356 ms/op 1.8176 ms/op 1.06
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 411.11 us/op 357.18 us/op 1.15
getCommitteeAssignments - req 1 vs - 250000 vc 5.3149 ms/op 4.2755 ms/op 1.24
getCommitteeAssignments - req 100 vs - 250000 vc 7.3456 ms/op 6.1270 ms/op 1.20
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7671 ms/op 6.4460 ms/op 1.20
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 10.880 ns/op 7.8900 ns/op 1.38
state getBlockRootAtSlot - 250000 vs - 7PWei 1.2327 us/op 924.44 ns/op 1.33
computeProposers - vc 250000 17.855 ms/op 15.780 ms/op 1.13
computeEpochShuffling - vc 250000 169.58 ms/op 144.12 ms/op 1.18
getNextSyncCommittee - vc 250000 284.54 ms/op 268.06 ms/op 1.06

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 337f2fc into unstable Dec 5, 2022
@wemeetagain wemeetagain deleted the dapplion/contributing-test-instructions branch December 5, 2022 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dapplion @wemeetagain