feat(l2): guest program modularization core#6
Merged
Zena-park merged 41 commits intofeat/app-customized-frameworkfrom Mar 2, 2026
Merged
feat(l2): guest program modularization core#6Zena-park merged 41 commits intofeat/app-customized-frameworkfrom
Zena-park merged 41 commits intofeat/app-customized-frameworkfrom
Conversation
Design documents for Phase 2 of the ZK optimization plan — making the guest program a pluggable module for app-specific L2s. Covers current architecture analysis, target architecture with GuestProgram trait, implementation phases, and risk analysis.
Define the GuestProgram trait abstraction with backend-agnostic interface for ELF loading, VK retrieval, and I/O serialization. Implement three programs: EvmL2GuestProgram (wraps existing monolithic guest program), ZkDexGuestProgram and TokammonGuestProgram (app-specific stubs). Add GUEST_PROGRAMS env var support to build.rs for multi-program builds. Includes 14 unit tests covering all program implementations.
Add backend_name(), execute_with_elf(), prove_with_elf() and their timed variants to the ProverBackend trait with default NotImplemented fallback. Implement backend_name() for all five backends (SP1, RISC0, ZisK, OpenVM, Exec). Exec backend provides full execute_with_elf/prove_with_elf implementation with rkyv deserialization. Add BackendType::as_backend_name() for enum-to-string conversion and BackendError::NotImplemented/Serialization error variants.
Implement GuestProgramRegistry for runtime program lookup by ID with default program support. Extend ProofData protocol with program_id and supported_programs fields (backward compatible via serde defaults). Update Proof Coordinator to dispatch batches with program_id and filter by prover capabilities. Integrate dual-path proving in the prover main loop: attempt ELF-based proving via registry, fall back to legacy path. Includes 5 registry unit tests.
…stry Upgrade OnChainProposer VK mapping from 2D to 3D (commitHash → programTypeId → verifierId → vk) to support multiple guest program types. Add programTypeId to BatchCommitmentInfo and commitBatch() signature with backward-compatible default (0 maps to DEFAULT_PROGRAM_TYPE_ID=1). Add GuestProgramRegistry.sol (UUPS upgradeable) for on-chain program registration with auto-assigned typeIds (1: EVM-L2, 2-9: official, 10+: community). Integrate with OnChainProposer via isProgramActive() validation. Update l1_committer to include programTypeId in commitment calldata. Add registry deployment and initialization to L1 deployer.
Implement GPTs-style marketplace for guest programs with Express.js backend and Next.js 15 frontend. Server: SQLite-backed REST API with Google/Naver/Kakao OAuth, session auth, program CRUD with approval workflow, ELF/VK upload (SHA-256), deployment management, admin dashboard API, input validation, and rate limiting. Client: 16-route Next.js app with store browsing, creator portal, deployment configurator with L2 TOML export, admin panel, and responsive Tailwind CSS UI.
Add new-guest-program.sh that generates boilerplate for custom guest programs including Rust module, trait implementation, unit tests, and module registration in programs/mod.rs.
Update Phase 2.1-2.4 and Phase 3 completion status with actual implementation details, deviations from plan, and test results. Add new design documents: - 05: Improvements proposal for future optimization - 06: App-specific templates (ZK-DEX, Tokamon) design - 07: Multi-role platform architecture (Guest Program Store)
Centralize OpenVM ELF binary into the guest-program crate with cfg-gated ZKVM_OPENVM_PROGRAM_ELF constant, matching the existing SP1/ZisK pattern. Remove local include_bytes! from the OpenVM backend and add ELF lookup test for the OpenVM backend.
Add SHA-256 based caching for client.setup(elf) in prove_with_elf to avoid re-running the expensive setup on every call. The cache uses a static Mutex<HashMap<[u8;32], (PK, VK)>> keyed by the ELF digest. Legacy prove_with_stdin path remains unchanged.
Add 5 tests covering the ELF-based execution path: backend name check, invalid rkyv input returns Serialization error, and empty input returns error for both execute_with_elf and prove_with_elf.
Replace hardcoded program_type_id=1 in l1_committer with a storage- backed lookup. Add store_program_id_by_batch / get_program_id_by_batch to the storage API (in-memory + SQL), persist program_id in proof_coordinator on proof submission, and resolve it in send_commitment via resolve_program_type_id(). Falls back to "evm-l2" (type_id=1) when no program_id is stored for backward compatibility.
Update three design documents to reflect the completed stabilization work (S1-S4): SP1 setup caching, prove_with_elf tests, OpenVM ELF registry integration, and dynamic programTypeId. - 03-implementation-phases: add S1-S4 section, update test count (19→25), update dependency diagram and remaining work table - 04-risk-analysis: mark R6 as resolved, R8 as mitigated with ELF hash caching implementation - 05-improvements-proposal: mark §1.4, §2.4, §3.3, §5.3 as resolved, update status and remaining work priorities
Add serialize_raw(&ProgramInput) -> Result<Vec<u8>> to ProverBackend trait with default rkyv implementation. All backends now call serialize_raw() inside serialize_input() instead of duplicating rkyv::to_bytes. prover.rs prove_batch() uses backend.serialize_raw() instead of inline rkyv call. Removes direct rkyv imports from SP1, RISC0, OpenVM, and ZisK backends. Also fixes OpenVM import bug: ZKVM_OPENVM_ZKVM_OPENVM_PROGRAM_ELF (doubled prefix) -> ZKVM_OPENVM_PROGRAM_ELF.
Verify that pre-modularization JSON (without supported_programs and program_id fields) correctly deserializes via serde defaults. Tests cover BatchRequest, BatchResponse, and ProofSubmit variants in both old-format and new-format roundtrip scenarios. Adds serde_json as dev-dependency to ethrex-l2-common.
Update improvements proposal: serialize_raw() standardization (§2.3) resolved, multi-ELF build tool already implemented. Reorganize remaining work list from 10 to 8 items.
Restructure ZK-DEX guest program from flat file to module directory with dedicated types, execution, and trait implementation files. - Define DexProgramInput, DexTransfer, DexProgramOutput with rkyv - Implement chained keccak-256 state transition model - Validate empty batch, zero amount, and self-transfer - Add 10 tests covering execution, serialization, and error cases
Restructure Tokamon guest program from flat file to module directory with dedicated types, execution, and trait implementation files. - Define ActionType enum, GameAction, TokammonProgramInput/Output with rkyv - Implement chained keccak-256 state transition with payload validation - Validate empty batch, short CreateSpot/Battle payloads - Add 11 tests covering execution, serialization, and error cases
Create dedicated SP1 guest binaries for each custom program: - bin/sp1-zk-dex/: reads DexProgramInput, executes transfers, commits output - bin/sp1-tokamon/: reads TokammonProgramInput, executes game actions, commits output Build infrastructure updates: - Make program modules public for cross-crate access - Add ZKVM_SP1_ZK_DEX_ELF and ZKVM_SP1_TOKAMON_ELF constants in lib.rs - Add build_sp1_zk_dex() and build_sp1_tokamon() in build.rs - Wire GUEST_PROGRAMS=zk-dex,tokamon to trigger SP1 builds - Update GuestProgram::elf() to return SP1 ELFs - Add per-program Makefile targets (sp1-zk-dex, sp1-tokamon)
The new-guest-program.sh now creates: - Module directory with types.rs, execution.rs, and mod.rs - SP1 binary directory with Cargo.toml and main.rs entry point - 8 unit tests (trait, execution, rkyv roundtrip, output encoding) - Auto-assigns program_type_id and registers in mod.rs Fixed macOS bash 3 compatibility (PascalCase via awk, no bash 4 features).
Add 9 integration tests using real GuestProgram implementations: - Verify 3 programs registered with correct IDs and type IDs - Verify unique type IDs across all programs - Cross-crate execution: ZK-DEX transfers and Tokamon game actions - Output encoding length verification (72 bytes ZK-DEX, 88 bytes Tokamon) - rkyv serialization roundtrip for both program input types
Replace the volatile in-memory session Map with persistent SQLite storage: - Add sessions table with user_id FK and ON DELETE CASCADE - Create db/sessions.js with CRUD operations and TTL validation - Update auth middleware to use SQLite session lookups - Add hourly cleanup interval for expired sessions Sessions now survive server restarts and are automatically cleaned up when the associated user is deleted.
Add validate_elf() method to GuestProgram trait with default impl: - Checks ELF magic number, class (32/64-bit), and RISC-V machine type - Backend-aware class expectations (SP1/RISC0/OpenVM=32-bit, ZisK=64-bit) - New GuestProgramError::InvalidElf variant Add 12 tests: - 9 ELF validation tests (valid, too short, bad magic, wrong class, etc.) - 3 fuzz-style property tests ensuring serialize_input, encode_output, and validate_elf never panic on arbitrary byte inputs
Implement DynamicGuestProgram that loads ELF binaries from the filesystem at runtime, eliminating the need to recompile the prover when adding or updating guest programs. - Builder API: elf_from_file(), elf_from_bytes(), vk_from_file(), vk_from_bytes() - from_dir(): auto-scans <dir>/<backend>/elf for known backends - ELF header validation on load (skip_validation() to disable) - Implements GuestProgram trait for Arc<dyn GuestProgram> registry use - 15 tests covering builder, file I/O, directory scan, validation
Add 08-developer-guide.md covering quick start (scaffold script), step-by-step manual creation, dynamic ELF loading, GuestProgram trait reference, testing checklist, and existing programs reference. Update 05-improvements-proposal.md to mark dynamic ELF loading and developer guide as completed — all planned tasks are now done.
Seven production hardening tasks for the guest program modularization: 1. Resource Limits — ResourceLimits struct with max_input_bytes and max_proving_duration, enforced in prove_batch() 2. L1 Public Input Dispatcher — publicValuesHash for custom programs (programTypeId > 1) in OnChainProposer commit/verify flow 3. Cross-language encoding tests — byte layout verification for L2, ZK-DEX, Tokamon, and L1 output encodings 4. Guest Program versioning — version() and elf_hash() methods on GuestProgram trait 5. Runtime config — TOML-based ProgramsConfig for filtering active programs without recompilation 6. ProofData backward compat tests — roundtrip and edge case coverage 7. zkVM integration test scaffolding — ignored SP1/RISC0 tests for CI Also includes Korean translation of the developer guide.
🤖 Kimi Code ReviewAutomated review by Kimi (Moonshot AI) |
5 tasks
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request lays the foundational infrastructure for modularizing guest programs within the ZK proving system. It introduces a flexible Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Zena-park
changed the title
…ock.commitBatch The IOnChainProposer.commitBatch interface expects 11 parameters but Timelock.commitBatch was only passing 9, missing programTypeId and publicValuesHash. This caused the Solidity compilation to fail.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This is a massive and impressive pull request that introduces a foundational change to modularize guest programs. The core of the change is the GuestProgram trait, which successfully abstracts away the specifics of each program, and the corresponding refactoring of the ProverBackend to be guest-agnostic. The introduction of a 3D verification key mapping in the L1 contracts to support multiple program types is well-thought-out, and the backward compatibility considerations using #[serde(default)] and default program IDs are excellent.
The addition of the full-stack "Guest Program Store" platform is a significant feature that greatly enhances the developer experience. The code is well-structured, and the inclusion of extensive tests, including backward compatibility and fuzz-style robustness tests, is commendable.
My review includes a few suggestions to improve maintainability in the build script, enhance on-chain error reporting for better debuggability, and improve the portability of the new scaffolding script. Overall, this is a high-quality contribution that sets a strong foundation for future expansion.
…d clippy lints - Add missing programTypeId and publicValuesHash params to ITimelock.commitBatch - Update all Cargo.lock files (sp1, risc0, zisk, openvm, tee, tooling, sp1-zk-dex, sp1-tokamon) - Add clippy allow attributes for test modules in exec.rs, programs_config.rs, registry.rs
Add @Custom:oz-upgrades-unsafe-allow state-variable-type-change to both OnChainProposer contracts. The verificationKeys mapping was intentionally changed from 2D to 3D to support programTypeId, and this is a fresh deployment on tokamak-dev (not an in-place upgrade from main).
…rade Replace the invalid @Custom:oz-upgrades-unsafe-allow state-variable-type-change annotation with @Custom:oz-retyped-from, which is the correct OpenZeppelin annotation for intentional storage variable type changes. This tells the upgradeability checker that the mapping change from 2D to 3D was intentional.
This was referenced Mar 1, 2026
- Add clippy allow for expect/unwrap in zkvm_integration test modules - Collapse nested if-let patterns in prover.rs (collapsible_if) - Add missing programs_config_path field in ProverConfig construction
When GUEST_PROGRAMS doesn't include zk-dex or tokamon, the build.rs now creates empty placeholder ELF files so that include_bytes! in lib.rs doesn't fail during cargo check in CI.
…al impl Clippy's `derivable_impls` lint flags the manual `Default` implementation since all fields are `Option` types that default to `None`. Use derive instead.
The Lint L2 CI job runs clippy with `-D clippy::panic` and `-D clippy::expect_used` even on test code. Add the standard `#[allow(...)]` block to the test module.
After initialize(), the OnChainProposer owner is set to the Timelock contract via __Ownable_init(). The deployer's subsequent call to setGuestProgramRegistry() reverted because the deployer is not the owner. Fix by accepting guestProgramRegistry as a parameter in initialize() and setting it before ownership transfer. - Add guestProgramRegistry parameter to initialize() in both based and non-based OnChainProposer contracts - Update deployer.rs to pass registry address in initialize calldata - Remove separate setGuestProgramRegistry() call from deployer - Remove unused SET_GUEST_PROGRAM_REGISTRY_SIGNATURE constant
Remove the uniswap-swap job and its references from the all-tests aggregator. This test is not relevant to the Tokamak L2 and adds unnecessary CI time.
This reverts commit 488490b.
Zena-park
changed the base branch from
tokamak-dev
to
feat/app-customized-framework
Zena-park
added a commit
that referenced
this pull request
Mar 16, 2026
…ntainability - CORS: restrict origin to Tauri dev/prod allowlist (Copilot #1) - open-url: use execFile with arg arrays instead of shell exec (Copilot #2) - fs browse: restrict path traversal to home directory (Copilot #3) - test-e2e-fork: move RPC URL to SEPOLIA_RPC_URL env var (Copilot #4) - docker-remote: clear timeout on stream close, close stream on timeout (Copilot #5) - docker-remote: add shell quoting (q()) and assertSafeName for all interpolated shell args to prevent injection (Copilot #6-8) - genesis.rs: add ChainConfig::validate() for pre-startup checks (Copilot #9) - listings.js: use named params (@id, @name, ...) instead of 30 positional ? args for upsertListing (Gemini #1)
Zena-park
added a commit
that referenced
this pull request
Mar 17, 2026
- Return 503 instead of {exists:false} on check endpoint errors (#1)
- Sanitize all error messages — log internally, return generic to client (#2,#3,#4,#5)
- Add serverless rate limit limitation comment (#6)
- Add console.warn to all empty catch blocks in github-pr.ts (#9,#10,#11)
- Note: params as Promise is correct for Next.js 15 (#7,#8)
Zena-park
added a commit
that referenced
this pull request
Mar 17, 2026
* feat(platform): add appchain registry API routes to Next.js client Port Express server appchain-registry endpoints to Next.js API routes for Vercel deployment: - GET /api/appchain-registry/check/[l1ChainId]/[stackType]/[identityAddress] - POST /api/appchain-registry/submit - GET /api/appchain-registry/status/[prNumber] Shared logic in lib/appchain-registry.ts and lib/github-pr.ts. * fix: address PR #67 code review feedback - Return 503 instead of {exists:false} on check endpoint errors (#1) - Sanitize all error messages — log internally, return generic to client (#2,#3,#4,#5) - Add serverless rate limit limitation comment (#6) - Add console.warn to all empty catch blocks in github-pr.ts (#9,#10,#11) - Note: params as Promise is correct for Next.js 15 (#7,#8) * fix: address additional Copilot PR #67 review feedback - Add AbortSignal.timeout(15s) to all GitHub API fetch calls - Fix authHeaders error message to list both accepted env vars - Distinguish RPC errors (502) from permission denied (403) in ownership check - Add typeof validation for metadata.signedBy - Add unit test suggestion acknowledged (future work)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Core infrastructure for guest program modularization in the ZK proving system:
ProverBackendprogramTypeIdfield andGuestProgramRegistry.solfor on-chain guest program managementprogramTypeIdresolutionDependencies
This is the first PR in a 4-part stacked PR series (1/4).
Merge order: #6 → #7 → #8 → #9
Test plan
prove_with_elfintegration tests for exec backend