PoC: OpenShell integration

[josh-pritchard]

Description

Proof-of-concept integration of OpenShell as a third deployment platform alongside local (Docker Compose) and kubernetes. OpenShell provides secure, sandboxed agent execution with defense-in-depth isolation (Landlock LSM, seccomp-bpf, network namespaces, inference routing).

This PoC validates the end-to-end flow: register agent → build image → deploy to OpenShell sandbox → sandbox reaches READY. It is not merge-ready — see "Not working / Known gaps" below.

Change Type

/kind feature

Changelog

OpenShell integration

What's working

• gRPC client (client.go): Connects to an OpenShell gateway with mTLS authentication. Supports endpoint discovery via env vars (OPENSHELL_GATEWAY_ENDPOINT) or filesystem config (~/.config/openshell/gateways/). Lazy client initialization — AR server starts fine without OpenShell installed.
• Deployment adapter (deployment_adapter.go): Full DeploymentPlatformAdapter implementation — Deploy, Undeploy, GetLogs, Cancel. Polls GetSandbox until the sandbox reaches READY phase (120s timeout).
• Provider registration: openshell platform registered in provider adapters and deployment platform map. DB migration seeds an openshell-default provider.
• Proto vendoring: Makefile target (make sync-openshell-proto) fetches protos from NVIDIA/OpenShell at a pinned version and generates Go stubs. Both protos and generated code are checked in so builds don't require protoc.
• UI deploy target selector: Deploy dialog fetches available providers and shows a dropdown instead of hardcoding providerId: "local". Platform display names map openshell → "OpenShell".
• Docker Compose config: AR server container configured with OpenShell gateway endpoint and mTLS cert mount for local dev.
• Unit tests: Client and deployment adapter have full mock-based test coverage.

Not working / Known gaps

• E2E tests: Test scaffolding is in place (skip when OpenShell unavailable, image loading into K3s, sandbox verification/cleanup helpers) but tests have not been run end-to-end in CI. They require an OpenShell gateway running locally.
• Image compatibility: OpenShell's supervisor requires images to include iproute2 (for ip binary) and a sandbox user/group. Standard agent images will fail without these. See "Image requirements" below.
• CLI timeout: arctl deploy create blocks synchronously for up to 120s while the sandbox provisions. The arctl HTTP client can timeout before the server-side deploy completes. Needs async deploy pattern (return immediately, poll status).
• Agent invocation: No way to send prompts to deployed agents. arctl agent run only works with the local Docker Compose platform. There is no deploy invoke or deploy chat command.
• SPA routing fix: Includes an unrelated fix to server.go for Next.js static export routing (.html suffix resolution, SPA fallback). Should be split into a separate PR before merge.

Image requirements for OpenShell

OpenShell's supervisor binary is sideloaded into user containers at runtime. It requires:

1. iproute2 package — supervisor shells out to ip for network namespace creation (veth pairs, netns). Without it, sandbox creation fails with ENOENT.
2. iptables package (optional) — enables network bypass detection.
3. sandbox user and group — supervisor drops privileges to this user after setup.

For Alpine/Wolfi-based images (like kagent-adk):

USER root
RUN apk add --no-cache iproute2 iptables && \
    addgroup -S sandbox && adduser -S -G sandbox -s /bin/sh sandbox

These requirements should eventually be handled automatically by arctl agent build when targeting OpenShell, rather than requiring manual Dockerfile changes.

Why OpenShell is a good fit for AR

• Secure local alternative: The local platform (Docker Compose) has zero isolation. OpenShell provides Landlock, seccomp, network namespaces, and inference routing out of the box — same UX, defense-in-depth security.
• Purpose-built API: gRPC gateway with mTLS is designed for programmatic sandbox management. Cleaner integration surface than shelling out to Docker/kubectl.
• Runtime, not infrastructure: K3s is an implementation detail hidden from users. They interact with sandboxes, not pods. AR manages the registry; OpenShell manages execution.
• Multi-gateway architecture: One AR instance can target multiple OpenShell gateways (dev laptop, staging server, production). Each provider maps to a gateway endpoint. This naturally could extend AR to multi-environment deployment.
• Inference routing opportunity: OpenShell can enforce which LLM providers an agent can reach. AR could integrate this for provider-level access control — a capability neither local nor kubernetes offers.

Local testing steps

# 1. Start OpenShell gateway
openshell gateway start --name ar-dev

# 2. Start AgentRegistry
make docker-compose-up

# 3. Build an agent with OpenShell requirements
arctl agent init adk python my-agent
# Edit Dockerfile to add iproute2 + sandbox user (see above)
arctl agent build my-agent

# 4. Load image into OpenShell's K3s
docker save my-agent:latest | \
  docker exec -i $(docker ps --filter name=openshell-cluster- --format '{{.Names}}') \
  ctr -n k8s.io images import --all-platforms -

# 5. Set image pull policy (needed for :latest tags)
kubectl set env statefulset/openshell -n openshell \
  OPENSHELL_SANDBOX_IMAGE_PULL_POLICY=IfNotPresent

# 6. Deploy
arctl deploy create my-agent --type agent --provider-id openshell-default

# 7. Verify
openshell sandbox list
arctl deploy show <deployment-id>

[Copilot]

Pull request overview

Proof-of-concept integration of OpenShell as a third deployment platform (alongside local and kubernetes), including vendored OpenShell gRPC protos/clients, adapter wiring, provider seeding, UI provider selection, and E2E scaffolding. The PR also includes an unrelated UI static-export routing change in the API server.

Changes:

• Add openshell deployment adapter + gRPC client (mTLS + gateway discovery) with unit tests.
• Vendor OpenShell protos + generated Go stubs, plus a Makefile sync target and a DB seed migration for openshell-default.
• Update UI deploy dialog to select a provider dynamically; extend E2E deploy targets and local docker-compose dev config.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
internal/registry/platforms/openshell/client.go	OpenShell gRPC client (endpoint discovery + mTLS) and basic sandbox operations
internal/registry/platforms/openshell/client_test.go	Unit tests for gateway metadata / TLS loading and small helpers
internal/registry/platforms/openshell/deployment_adapter.go	OpenShell DeploymentPlatformAdapter implementation (deploy, undeploy, logs, cancel)
internal/registry/platforms/openshell/deployment_adapter_test.go	Mock-based unit tests for adapter behavior (deploy polling, etc.)
internal/registry/platforms/openshell/provider_config.go	Provider-level config type for the OpenShell platform
internal/registry/platforms/openshell/proto/OPENSHELL_PROTO_VERSION	Pinned upstream proto version marker
internal/registry/platforms/openshell/proto/openshell.proto	Vendored OpenShell service proto
internal/registry/platforms/openshell/proto/datamodel.proto	Vendored OpenShell datamodel proto
internal/registry/platforms/openshell/proto/sandbox.proto	Vendored sandbox policy proto
internal/registry/platforms/openshell/proto/inference.proto	Vendored inference proto
internal/registry/platforms/openshell/proto/gen/openshell.pb.go	Generated Go stubs for openshell.proto
internal/registry/platforms/openshell/proto/gen/openshell_grpc.pb.go	Generated Go gRPC stubs for OpenShell service
internal/registry/platforms/openshell/proto/gen/datamodel.pb.go	Generated Go stubs for datamodel.proto
internal/registry/platforms/openshell/proto/gen/sandbox.pb.go	Generated Go stubs for sandbox.proto
internal/registry/platforms/openshell/proto/gen/inference.pb.go	Generated Go stubs for inference.proto
internal/registry/platforms/openshell/proto/gen/inference_grpc.pb.go	Generated Go gRPC stubs for Inference service
internal/registry/registry_app.go	Wire openshell into the deployment platform adapter map
internal/registry/api/handlers/v0/provider_adapters.go	Register openshell provider adapter
internal/registry/database/migrations/011_seed_openshell_provider.sql	Seed openshell-default provider
internal/daemon/docker-compose.yml	Local dev env vars + mTLS mount for OpenShell gateway access
Makefile	Add sync-openshell-proto target to fetch protos and regenerate stubs
e2e/deploy_test.go	Add OpenShell deploy targets + helpers for image loading / sandbox verification
ui/components/deploy-server-dialog.tsx	UI deploy dialog: provider dropdown (instead of hardcoded local)
ui/lib/platform-display.ts	Platform display mapping for UI labels/descriptions
internal/registry/api/server.go	SPA/static-export routing logic for embedded UI assets

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[peterj]

I was able to partially get it to work:

• made changes to the dockerfile to use the sandbox user and install iproute/iptables
• automatically creating the provider based on the model set in the agent (note that for gemini, we have to use generic as openshell doesn't have a dedicated type for that yet)
• creating the policy that allows egress to the model + allows executing the python binary

Deployment work. You can deploy an agent to openshell. However, for whatever reason I wasn't able to set the OPENSHELL_SANDBOX_COMMAND to the actual kagent-adk command we need to launch the agent. Openshell keeps changing that to sleep inifinity (might have to look at this with the fresh eyes).

Anyway, so you can still make it work manually after that:

# getting a shell inside the sandbox
RUST_LOG=debug openshell sandbox connect myagent

# in the shell
kagent-adk run --host 0.0.0.0 --port 9999 myagent --local

Then, start port forward to the sandbox:

RUST_LOG=debug openshell forward start 9999 myagent

And then you can send the A2A request to localhost:9999 and that will work.

Interestingly, if I provide the dockerfile directly to the sandbox create command, that works without issues:

RUST_LOG=debug openshell sandbox create \
    --from Dockerfile \
    --forward 9999 \
    -- kagent-adk run --host 0.0.0.0 --port 8080 myagent --local

[peterj]

adding the "how to":

Try the OpenShell integration (end-to-end)

In this walkthrough you'll do the following:

• install the OpenShell CLI
• scaffold and publish an agent to agent registry
• deploy the agent to OpenShell
• port-forwards to the agent running in OpenShell and interact with the agent

Prerequisites

• Docker (e.g. Docker Desktop) running.
• AgentRegistry running with the API reachable from your machine (default: http://localhost:12121). The arctl CLI uses that URL unless you set --registry-url or ARCTL_API_BASE_URL.
• You will need a model API key in the environment for the provider you choose (for example GOOGLE_API_KEY for the default Gemini scaffold).

For image pulls, local setups often use the dev registry from Docker Compose (localhost:5001). Use an image name your OpenShell gateway can pull (same host/network as the gateway).

---

1. Install the OpenShell CLI

curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh

2. Start a gateway

openshell gateway start
openshell status

This creates credentials under ~/.config/openshell/gateways/<name>/ (default name is often openshell).

---

3. Create a sample agent

AGENT_IMAGE=docker.io/yourusername/demoagent:latest
arctl agent init adk python demoagent --image $AGENT_IMAGE
cd demoagent

---

4. Build the image and publish to your registry

Build and push an image the gateway can pull. Example for a local registry on port 5001:

arctl agent build . --image $AGENT_IMAGE --push

Publish the agent to the agentregistry:

arctl agent publish .

---

5. Deploy to OpenShell

Use the built-in provider openshell-default. Pass model credentials (and any overrides) with --env:

arctl deployments create demo-agent \
  --type agent \
  --provider-id openshell-default \
  --env GOOGLE_API_KEY="$GOOGLE_API_KEY"

Wait until status is deployed and use the forward command to port-forward to the agent. Example output from the create deployment command:

✓ OpenShell: forward this port from your machine to reach the agent
  openshell forward start 9999 myagent-f5a2367a-0284-4003-84fe-2e230fa128c5

---

6. Forward the agent port to your laptop

The workload listens on 9999 inside the sandbox. From your machine:

openshell forward start 9999 <sandbox-name>

If the CLI fails with ssh exited with status … 255, try this:

RUST_LOG=debug openshell forward start 9999 <sandbox-name>

(You can add --background to avoid blocking the terminal; see openshell forward --help.)

---

7. Send a request to the agent

Agent card (A2A discovery)

curl -sS http://127.0.0.1:9999/.well-known/agent-card.json | head

Send a sample request:

curl -X POST \
  "http://localhost:9999" \
  -H "Content-Type: application/json" \
  -d '{
  "jsonrpc": "2.0",
  "id": "1",
  "method": "message/send",
  "params": {
   "message": {
    "kind": "message",
    "role": "user",
    "messageId": "msg-1",
    "contextId": "f5bd2a40-74b6-4f7a-b649-ea3f09890003",
    "parts": [
        { "kind": "text", "text": "what can you do?" }
    ]
    }
  }
}'