Replace bundled command

[sergerad]

Replace bundled command with local multi-process deployment

Summary

Remove the miden-node bundled command and replace it with a proper local deployment mechanism that runs each node component as a separate (containerized) process. This ensures that local testing and development imitates production-style deployment as closely as possible.

Motivation

The bundled command runs all node components (Store, RPC, Block Producer, Validator, NTX Builder) as tasks within a single process. While convenient, this approach has several drawbacks:

• Hides the real architecture. Developers don't validate aspects of the system that are affected by a distributed deployment topology.
• Masks failure modes. A single process with fail-fast semantics (JoinSet::join_next) doesn't exercise the partial-failure and reconnection scenarios that occur in production.
• Adds unnecessary complexity and maintenance overhead. Developers have to consider both bundled and non-bundled configurations when implementing new features.
• Prevents independent scaling/restart. You can't restart or replace a single component without tearing down the entire node.
• Diverges from production. The bundled mode uses 127.0.0.1:0 for internal endpoints and passes TcpListener handles directly, which is fundamentally different from how components connect in a distributed deployment. We also deploy containerized versions of the stack in production.

A local multi-process deployment tool would provide a more realistic and useful development/testing experience while still being easy to use.

Current state

The bundled command

Located in bin/node/src/commands/bundled.rs, it provides two subcommands:

• bundled bootstrap — initializes the store database with a genesis block.
• bundled start — spawns all components as tokio tasks in a JoinSet, binds internal gRPC endpoints to random localhost ports, and crashes if any component exits.

Individual component commands

Each component already has its own CLI subcommand under miden-node:

Component	Command	Key flags
Store	miden-node store start	--rpc.url, --block-producer.url, --ntx-builder.url, --data-directory
RPC	miden-node rpc start	--url, --store.url, --block-producer.url, --validator.url
Block Producer	miden-node block-producer start	--store.url, --validator.url
Validator	miden-node validator start	--data-directory
NTX Builder	none (embedded in bundled only)	--store.url, --block-producer.url, --validator.url, --ntx-builder.data-directory

Note: The NTX Builder currently has no standalone CLI subcommand — it is only launched as part of bundled start. A new miden-node ntx-builder start subcommand will need to be added before it can run as its own process.

Running these individually today requires manually coordinating all URLs and ensuring correct startup order (store must be available before other components connect).

Existing Docker support

• A Dockerfile exists at bin/node/Dockerfile that builds the miden-node binary.
• make docker-build-node and make docker-run-node targets exist but only run a single container.
• No docker-compose.yml exists. There is no multi-container orchestration.

Proposal

1. Remove the bundled command

Delete the bundled subcommand and its associated code:

• bin/node/src/commands/bundled.rs
• Related CLI variants, config structs (BundledValidatorConfig), and wiring in bin/node/src/commands/mod.rs
• Update the systemd service file (packaging/node/miden-node.service) which currently runs bundled start
• Update operator documentation (docs/external/src/operator/) to remove bundled references

2. Add a local multi-process deployment tool

Replace bundled with a tool that launches each component as a separate OS process with pre-configured, well-known ports. Two main options are considered:

---

Option A: docker-compose

Add a docker-compose.yml at the repo root that defines one service per component.

Sketch:

services:
  store:
    build:
      context: .
      dockerfile: bin/node/Dockerfile
    command: >
      miden-node store start
        --rpc.url http://0.0.0.0:50051
        --block-producer.url http://0.0.0.0:50052
        --ntx-builder.url http://0.0.0.0:50053
        --data-directory /data
    volumes:
      - store-data:/data
    healthcheck:
      test: ["CMD", "grpc_health_probe", "-addr=:50051"]
      interval: 5s

  block-producer:
    build:
      context: .
      dockerfile: bin/node/Dockerfile
    command: >
      miden-node block-producer start http://0.0.0.0:50054
        --store.url http://store:50052
        --validator.url http://validator:50055
    depends_on:
      store:
        condition: service_healthy

  validator:
    build:
      context: .
      dockerfile: bin/node/Dockerfile
    command: >
      miden-node validator start http://0.0.0.0:50055
        --data-directory /data
    volumes:
      - validator-data:/data

  rpc:
    build:
      context: .
      dockerfile: bin/node/Dockerfile
    command: >
      miden-node rpc start
        --url http://0.0.0.0:57291
        --store.url http://store:50051
        --block-producer.url http://block-producer:50054
        --validator.url http://validator:50055
    ports:
      - "57291:57291"
    depends_on:
      store:
        condition: service_healthy

  ntx-builder:
    build:
      context: .
      dockerfile: bin/node/Dockerfile
    command: >
      miden-node ntx-builder start
        --store.url http://store:50053
        --block-producer.url http://block-producer:50054
        --validator.url http://validator:50055
        --ntx-builder.data-directory /data
    volumes:
      - ntx-builder-data:/data
    depends_on:
      store:
        condition: service_healthy

volumes:
  store-data:
  validator-data:
  ntx-builder-data:

Pros:

• Industry standard, most developers already know it
• Zero additional tooling — Docker is the only dependency
• Simple YAML, easy to read and modify
• docker compose up / docker compose down is familiar
• Existing Dockerfile can be reused as-is
• Supports depends_on with health checks for startup ordering

Cons:

• Rebuild cycle for code changes is slower (must rebuild container image)
• Debugging is slightly less convenient than native processes (need docker exec or remote debugger)
• Doesn't support running a mix of local binaries and containers easily

---

Option B: Kurtosis

Use Kurtosis to define a package (Starlark) that orchestrates the node stack.

Sketch:

def run(plan):
    store = plan.add_service(
        name="store",
        config=ServiceConfig(
            image=ImageBuildSpec(image_name="miden-node", build_context_dir="."),
            cmd=["miden-node", "store", "start", ...],
            ports={"rpc": PortSpec(50051), "bp": PortSpec(50052), "ntx": PortSpec(50053)},
        ),
    )

    validator = plan.add_service(
        name="validator",
        config=ServiceConfig(
            image="miden-node",
            cmd=["miden-node", "validator", "start", ...],
            ports={"grpc": PortSpec(50055)},
        ),
    )

    block_producer = plan.add_service(
        name="block-producer",
        config=ServiceConfig(
            image="miden-node",
            cmd=["miden-node", "block-producer", "start",
                 "--store.url", "http://{}:{}".format(store.ip_address, 50052),
                 "--validator.url", "http://{}:{}".format(validator.ip_address, 50055)],
            ports={"grpc": PortSpec(50054)},
        ),
    )

    plan.add_service(
        name="rpc",
        config=ServiceConfig(
            image="miden-node",
            cmd=["miden-node", "rpc", "start",
                 "--url", "http://0.0.0.0:57291",
                 "--store.url", "http://{}:{}".format(store.ip_address, 50051),
                 "--block-producer.url", "http://{}:{}".format(block_producer.ip_address, 50054),
                 "--validator.url", "http://{}:{}".format(validator.ip_address, 50055)],
            ports={"rpc": PortSpec(57291, transport_protocol="TCP")},
        ),
    )

    plan.add_service(
        name="ntx-builder",
        config=ServiceConfig(
            image="miden-node",
            cmd=["miden-node", "ntx-builder", "start",
                 "--store.url", "http://{}:{}".format(store.ip_address, 50053),
                 "--block-producer.url", "http://{}:{}".format(block_producer.ip_address, 50054),
                 "--validator.url", "http://{}:{}".format(validator.ip_address, 50055),
                 "--ntx-builder.data-directory", "/data"],
        ),
    )

Pros:

• Programmable topology (Starlark) — can parameterize number of validators, toggle NTX builder, etc.
• Built-in service dependency resolution and health checking
• Designed for multi-service dev/test environments
• Enclave isolation — multiple instances can run side-by-side without port conflicts
• Good for CI — reproducible, hermetic environments
• Can be extended for multi-node / network testing scenarios

Cons:

• Requires installing Kurtosis CLI — an additional dependency most developers won't have
• Starlark is less familiar than YAML for most developers
• Smaller community and ecosystem compared to Docker Compose
• Adds a layer of abstraction that may complicate debugging
• Kurtosis itself is a relatively young tool — risk of API changes or reduced maintenance

---

Work items (assuming docker-compose approach)

• Add a standalone miden-node ntx-builder start CLI subcommand
• Add docker-compose.yml with all node components as separate services
• Add a bootstrap init-container or script that runs miden-node store bootstrap before the stack starts
• Add make compose-up / make compose-down targets to the Makefile
• Update operator documentation to describe the new deployment workflow
• Remove bundled.rs and all associated CLI/config code
• Remove all system service files apart from validator. Also consider deploying validator with Docker instead.

[Mirko-von-Leipzig]

Thank you! I had heard of Kurtosis but hadn't ever looked into it. It looks and sounds pretty useful.

I think given the use case for the "bundled" variant as a local dev environemtn, docker compose makes the most sense to me just because it at least would be familiar to the users. This isn't a strong opinion though.

---

Add a bootstrap init-container or script that runs miden-node store bootstrap before the stack starts

I think an alternative is to have separate bootstrap and run commands? I might feel better about it that way; but maybe a script is simpler. Lets see.

---

ensuring correct startup order (store must be available before other components connect).

This shouldn't be a requirement ^^ but rather something we should fix. Components should be startable in any order and if a dependency is unavailable, then they should idle, retry or at worst exit and be restarted. They don't currently all work like this, but they should :)

Some prerequisites imo before we can begin this process, is that each component should

• start independently
• correctly specify healthy based on dependent components being up/down
• respond to sigkill etc, ideally with a proper shutdown