add centralized sequencer doc by Manav-Aggarwal · Pull Request #59 · evstack/centralized-sequencer

Manav-Aggarwal · 2025-03-05T12:46:29Z

Overview

coderabbitai · 2025-03-05T12:46:38Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

tac0turtle · 2025-03-05T13:37:21Z

adr-11-centralized-sequencer.md

+We implement a standalone centralized sequencer that:
+
+1. Implements the Generic Sequencer interface from the `go-sequencing` package
+2. Exposes a gRPC service for rollup clients to submit transactions


this isnt the case with execution api anymore right?

Yes, not the case once we move this code back into Rollkit

tac0turtle · 2025-03-05T13:37:56Z

adr-11-centralized-sequencer.md

+       batchTime       time.Duration
+       bq              *BatchQueue
+       lastBatchHash   []byte
+       seenBatches     map[string]struct{}


what is the reason to keep seen batches here?

The seenBatches map is more so relevant to a verifying full node to know whether a batch was successfully published to DA or not. My understanding is this structure is equivalent to the dataCache type in Rollkit. We can get rid of it in favor of dataCache which seems like isn't currently used in Rollkit

we dont verify in the sequencer as well. we just pull from the queue and pass it to be executed. is there an open issue for verification?

tac0turtle · 2025-03-05T13:38:17Z

adr-11-centralized-sequencer.md

+       seenBatches     map[string]struct{}
+       seenBatchesMutex sync.RWMutex
+       metrics         *Metrics
+       db              *badger.DB


do we need this to have storage?

The only reason to have storage currently is to keep track of the last batch hash, which is needed when a rollkit node is restarted. We can store this in Rollkit's store instead to make the centralized sequencer stateless

make sense, in the current design we store lots of other things, would these things be removed as well and stored in rollkit?

adr-11-centralized-sequencer.md

tac0turtle

left some comments, it would be good to understand how the sequencer can move faster than the DA layer or in tandem

adr-11-centralized-sequencer.md

Co-authored-by: Marko <marko@baricevic.me>

tac0turtle · 2025-03-05T14:53:52Z

adr-11-centralized-sequencer.md

+
+       rollupId sequencing.RollupId // Identifier for the rollup this sequencer serves
+
+       tq                 *TransactionQueue  // Queue for storing pending transactions


do we need this if we fire off as soon as we get the data? also should rollkit construct batches instead of the sequencer, well they are the same thing or will be with the changes

I don't see a strong reason to have this either. Thinking that with the new changes, for a centralized sequencer setting, we should just default to the pre-separation code.

cc: @gupadhyaya

rollkit relays transactions as they gets into mempool. centralized sequencer needs a place to collect transactions and prepare batch at the batch interval (could be 1s or 2s, etc). hence the transaction queue. while the centralized sequencer and rollkit node are same but different processes, you don't want the sequencer to directly access mempool right? if you don't store it, will you create batch for every transaction? (i know with api change of submitting []Tx instead of single Tx via SubmitRollupTxs api, still you could fit in multiple sets of txs (relayed via multiple SubmitRollupTxs calls) into a single batch. queue is convenient, but open to any other ways to handle this.

Would it not work that ordering is done by the aggregator execution env and we just getTxs() and submit that? what extra ordering is there?

tac0turtle · 2025-03-05T14:54:55Z

adr-11-centralized-sequencer.md

+       lastBatchHash      []byte             // Hash of the last processed batch
+       lastBatchHashMutex sync.RWMutex       // Mutex for thread-safe access to lastBatchHash


are there multiple callers updating this? what is the reason for the mutex? seems expensive if there is only one updater

cc: @gupadhyaya

you can get rid of mutex. only one caller (header producer of rollkit)

tac0turtle · 2025-03-05T14:55:13Z

adr-11-centralized-sequencer.md

+       seenBatches      map[string]struct{}  // Map to track batches that have been processed
+       seenBatchesMutex sync.Mutex           // Mutex for thread-safe access to seenBatches


ditto on the mutex

for the batch producer the verifybatch is basically return true, hence you can get rid of this alltogether.

gupadhyaya · 2025-03-05T15:05:05Z

adr-11-centralized-sequencer.md

+   - Similar to the other methods, it first validates the rollup ID.
+   - It then checks if the provided batch hash exists in the internal data structure that tracks seen batches.
+   - If the batch hash is found, it returns a response indicating that the batch is valid. If not, it returns a response indicating that the batch is invalid.
+


should we add a TODO here? Currently VerifyBatch only does a quick verification of the batch hash exist in the seenBatches, however in the future, it is expected to perform the full DA verification where the batch submitted to DA as referenced using a DA height and VerifyBatch would query the DA at the referenced height to verify that the batch exists in DA.

would this be what the pending header system would work off of.

pending headers in rollkit is different. the verifybatch is for transaction batches.
verifyBatch for the batch producer (which is rollkit's header producer) is basically return true
for fullnodes, it needs to provide a stateless verification, meaning the verification is done using the DA

gupadhyaya · 2025-03-05T15:09:49Z

adr-11-centralized-sequencer.md

+
+   This method should be looking at the DA layer to update its view of which batches have been published. 
+
+These methods work together to ensure that the centralized sequencer can effectively manage transaction submissions, retrievals, and verifications, providing a reliable interface for rollup clients to interact with the sequencer.


should we add something about the batch submission loop?
The centralized sequencer runs a batch submission loop at the batch timer interval which prepares a batch and submits it to DA. The batch timer is configurable.

tac0turtle · 2025-03-14T13:37:15Z

closing this and bringing it into my pr, ill update it there

evstack/ev-node#2016

add centralized sequencer doc

dc24709

RollkitBot added this to Evolve Mar 5, 2025

github-actions bot assigned Manav-Aggarwal Mar 5, 2025

Manav-Aggarwal added 2 commits March 5, 2025 17:10

elaborate

ede2e1b

delete whitespace

e94c7f4

tac0turtle reviewed Mar 5, 2025

View reviewed changes

adr-11-centralized-sequencer.md Outdated Show resolved Hide resolved

tac0turtle reviewed Mar 5, 2025

View reviewed changes

Manav-Aggarwal added 2 commits March 5, 2025 18:32

add more comments

4d3debf

add more info on VerifyBatch

3731b11

tac0turtle reviewed Mar 5, 2025

View reviewed changes

adr-11-centralized-sequencer.md Outdated Show resolved Hide resolved

tac0turtle reviewed Mar 5, 2025

View reviewed changes

adr-11-centralized-sequencer.md Show resolved Hide resolved

Update adr-11-centralized-sequencer.md

84e2ee7

Co-authored-by: Marko <marko@baricevic.me>

tac0turtle reviewed Mar 5, 2025

View reviewed changes

update interface method description

c329151

gupadhyaya reviewed Mar 5, 2025

View reviewed changes

tac0turtle closed this Mar 14, 2025

github-project-automation bot moved this to Done in Evolve Mar 14, 2025

tac0turtle removed this from Evolve Apr 24, 2025


		rollupId sequencing.RollupId // Identifier for the rollup this sequencer serves

		tq *TransactionQueue // Queue for storing pending transactions

		lastBatchHash []byte // Hash of the last processed batch
		lastBatchHashMutex sync.RWMutex // Mutex for thread-safe access to lastBatchHash

		seenBatches map[string]struct{} // Map to track batches that have been processed
		seenBatchesMutex sync.Mutex // Mutex for thread-safe access to seenBatches


		This method should be looking at the DA layer to update its view of which batches have been published.

		These methods work together to ensure that the centralized sequencer can effectively manage transaction submissions, retrievals, and verifications, providing a reliable interface for rollup clients to interact with the sequencer.

Conversation

Manav-Aggarwal commented Mar 5, 2025

Overview

Uh oh!

coderabbitai bot commented Mar 5, 2025

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Manav-Aggarwal Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Manav-Aggarwal Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tac0turtle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tac0turtle commented Mar 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CodeRabbit Configuration File (`.coderabbit.yaml`)

Manav-Aggarwal Mar 5, 2025 •

edited

Loading

Manav-Aggarwal Mar 5, 2025 •

edited

Loading