L1 Info tree sync speed

[arnaubennassar]

The L1 Info tree syncing takes too long (1h at Bali). We need to go faster! Some things that we can do to improve syncing times:

• A better default for block ranges (currently set at 100)
  • Note that some providers limit the amount of logs returned by query, this makes it tricky to set a default... But 100 is very likely to be too small
  • Since a year has ~3M blocks, and the block range only makes sense in the context of historical syncing (once we are up to date we go block by block), we can be clever here: get the total missing blocks to sync and do a binary search until the query is accepted by the provider, have a good default (like 500k) and decrease if error, ... Overall it looks like we should be able to get rid as config param and handle it in the code. A decent provider should be able to return all the events in less than a minute, and since this is only needed to bootstrap the node or to catch up after reboot it makes sense to handle this on code
• Unify all L1 syncers to reduce overhead
• Review if we could optimize the usage of the reorg detector when syncing blocks that are already finalized
• Review if we can get rid of the get header call for all the blocks that contain relevant events, and use it only for the events where is actually needed
• Other improvements and optimizations
• Detect start syncing block to avoid lazy configs where this is set to 0
• Batch the processor requests so it's not one block per DB tx

Reference code for benchmarking (note that this represents bali network with 500k block range)

package l1infotreesync

import (
	"context"
	"path"
	"testing"
	"time"

	"github.com/0xPolygon/cdk-contracts-tooling/contracts/elderberry/polygonzkevmglobalexitrootv2"
	"github.com/0xPolygon/cdk/etherman"
	mocks_l1infotreesync "github.com/0xPolygon/cdk/l1infotreesync/mocks"
	"github.com/0xPolygon/cdk/log"
	"github.com/0xPolygon/cdk/reorgdetector"
	"github.com/ethereum/go-ethereum/accounts/abi/bind"
	"github.com/ethereum/go-ethereum/common"
	"github.com/ethereum/go-ethereum/ethclient"
	"github.com/stretchr/testify/mock"
	"github.com/stretchr/testify/require"
)

func TestBenchBali(t *testing.T) {
	rdm := mocks_l1infotreesync.NewReorgDetectorMock(t)
	rdm.On("Subscribe", mock.Anything).Return(&reorgdetector.Subscription{}, nil)
	rdm.On("AddBlockToTrack", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
	ctx := context.Background()
	dbPath := path.Join(t.TempDir(), "bench.sqlite")
	gerAddr := common.HexToAddress("0x2968d6d736178f8fe7393cc33c87f29d9c287e78")
	verifyAddr := common.HexToAddress("0xe2ef6215adc132df6913c8dd16487abf118d1764")
	client, err := ethclient.Dial("https://sepolia.gateway.tenderly.co/*********")
	require.NoError(t, err)
	syncer, err := New(ctx, dbPath, gerAddr, verifyAddr, 500_000, etherman.LatestBlock, rdm, client, time.Millisecond, 4794471, 100*time.Millisecond, 3,
		FlagAllowWrongContractsAddrs)
	require.NoError(t, err)
	now := time.Now()
	go syncer.Start(ctx)
	currentBlock, err := client.BlockNumber(ctx)
	require.NoError(t, err)
	for {
		lpb, err := syncer.GetLastProcessedBlock(ctx)
		require.NoError(t, err)
		if lpb >= currentBlock {
			break
		}
		log.Infof("syncer last processed block %d", lpb)
		time.Sleep(time.Second)
	}
	elapsed := time.Since(now)

	gerSc, err := polygonzkevmglobalexitrootv2.NewPolygonzkevmglobalexitrootv2(gerAddr, client)
	require.NoError(t, err)
	expectedRoot, err := gerSc.GetRoot(&bind.CallOpts{Pending: false})
	require.NoError(t, err)
	actualRoot, err := syncer.GetLastL1InfoTreeRoot(ctx)
	require.NoError(t, err)
	require.Equal(t, common.Hash(expectedRoot), actualRoot.Hash)
	depositCount, err := gerSc.DepositCount(nil)
	require.NoError(t, err)
	log.Infof("processed %d deposits in %f seconds", depositCount.Int64(), elapsed.Seconds())
}