Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
8c36239
run-manager: wip automatic run selection
dsocolobsky Jan 21, 2026
02f03a8
run-manager: filter halted runs from joining
dsocolobsky Jan 22, 2026
308a0fa
run-manager: more refactor
dsocolobsky Jan 22, 2026
9be6a25
run-manager change warn! to debug! to reduce verbosity in some places
dsocolobsky Jan 23, 2026
49a6f18
run-manager: refactor halted() to simplify code
dsocolobsky Jan 23, 2026
e57f68f
docs: update run-manager related docs to talk about automatic run
dsocolobsky Jan 23, 2026
2c89ed1
Merge branch 'main' into dy/run-manager-automatic-run-selection
dsocolobsky Jan 23, 2026
19155a2
Revert "run-manager: refactor halted() to simplify code"
dsocolobsky Jan 26, 2026
8098ebd
Merge branch 'main' into dy/run-manager-automatic-run-selection
dsocolobsky Jan 26, 2026
67d3f07
run-manager: prioritize joining waiting for compute runs
dsocolobsky Jan 27, 2026
f4d0910
run-manager: handle authorization in run selection
dsocolobsky Jan 27, 2026
85f35e4
run-manager: fix empty run_id thing
dsocolobsky Jan 27, 2026
6e10578
Merge branch 'main' into dy/run-manager-automatic-run-selection
dsocolobsky Jan 28, 2026
eedb03b
dev: fix just commands for creating authorized test runs
dsocolobsky Jan 28, 2026
7e3d852
run-manager: --authorizer param to filter runs authorized by a certai…
dsocolobsky Jan 28, 2026
5850113
docs: update with new run-manager --authorizer flag
dsocolobsky Jan 28, 2026
645d3c7
run-manager: improve base58 wallet key handling
dsocolobsky Jan 30, 2026
393b909
Merge branch 'main' into dy/run-manager-automatic-run-selection
dsocolobsky Jan 30, 2026
951fdb8
run-manager: fix authorization on join
dsocolobsky Jan 30, 2026
3017048
run-manager: handle delegate keys in can_user_join_run
dsocolobsky Feb 3, 2026
75f1d65
run-manager: list-runs command
dsocolobsky Feb 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions architectures/decentralized/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,16 @@ setup-solana-localnet-light-test-run-treasurer run_id="test" *args='':
RUN_ID={{ run_id }} CONFIG_FILE=./config/solana-test/light-config.toml PERMISSIONLESS=true ./scripts/setup-and-deploy-solana-test.sh --treasurer {{ args }}

setup-solana-localnet-permissioned-test-run run_id="test" *args='':
RUN_ID={{ run_id }} ./scripts/deploy-solana-test.sh {{ args }}
RUN_ID={{ run_id }} ./scripts/setup-and-deploy-solana-test.sh {{ args }}

setup-solana-localnet-permissioned-light-test-run run_id="test" *args='':
RUN_ID={{ run_id }} CONFIG_FILE=./config/solana-test/light-config.toml ./scripts/deploy-solana-test.sh {{ args }}
RUN_ID={{ run_id }} CONFIG_FILE=./config/solana-test/light-config.toml ./scripts/setup-and-deploy-solana-test.sh {{ args }}

setup-solana-localnet-permissioned-test-run-treasurer run_id="test" *args='':
RUN_ID={{ run_id }} ./scripts/deploy-solana-test.sh --treasurer {{ args }}
RUN_ID={{ run_id }} ./scripts/setup-and-deploy-solana-test.sh --treasurer {{ args }}

setup-solana-localnet-permissioned-light-test-run-treasurer run_id="test" *args='':
RUN_ID={{ run_id }} CONFIG_FILE=./config/solana-test/light-config.toml ./scripts/deploy-solana-test.sh --treasurer {{ args }}
RUN_ID={{ run_id }} CONFIG_FILE=./config/solana-test/light-config.toml ./scripts/setup-and-deploy-solana-test.sh --treasurer {{ args }}

start-training-localnet-client run_id="test" *args='':
AUTHORIZER={{ AUTHORIZER }} RUN_ID={{ run_id }} ./scripts/train-solana-test.sh {{ args }}
Expand Down
2 changes: 1 addition & 1 deletion architectures/decentralized/solana-authorizer/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions architectures/decentralized/solana-coordinator/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion architectures/decentralized/solana-mining-pool/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions architectures/decentralized/solana-treasurer/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 26 additions & 2 deletions psyche-book/src/enduser/join-run.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ WALLET_PATH=/path/to/your/keypair.json
RPC=https://your-primary-rpc-provider.com
WS_RPC=wss://your-primary-rpc-provider.com

# Required: Which run id to join
# Optional: Which run id to join
# If not set, the client will automatically discover and join an available run
RUN_ID=your_run_id_here

# Recommended: Fallback RPC Endpoints (for reliability)
Expand All @@ -78,6 +79,19 @@ Then, you can start training through the run manager running:
./run-manager --env-file /path/to/your/.env
```

### Automatic Run Selection

If you don't specify a `RUN_ID` in your `.env` file, the run-manager will automatically query the Solana coordinator to find a suitable run to join.
This makes it easier to join training without needing to know the specific run ID in advance. The run-manager will display which run it selected in the logs:

```
INFO RUN_ID not set, discovering available runs...
INFO Found 2 available run(s):
INFO - run_abc123 (state: Waiting for members)
INFO - run_def456 (state: Training)
INFO Selected run: run_abc123 (state: Waiting for members)
```

After the initial setup, you'll see the Psyche client logs streaming in real-time. These logs show training progress, network status, and other important information.

To stop the client, press `Ctrl+C` in the terminal.
Expand All @@ -86,9 +100,19 @@ To stop the client, press `Ctrl+C` in the terminal.

We recommend using a dedicated RPC service such as [Helius](https://www.helius.dev/), [QuickNode](https://www.quicknode.com/), [Triton](https://triton.one/), or self-hosting your own Solana RPC node.

## Filtering by Authorizer

If you want to only join runs authorized by a specific entity, you can use the `--authorizer` flag:

```bash
./run-manager --env-file /path/to/your/.env --authorizer <AUTHORIZER_PUBKEY>
```

This is useful when you want to ensure you only join runs from a trusted coordinator.

## Additional config variables

In general it's not neccesary to change these variables to join a run since we provide sensible defaults,
In general it's not necessary to change these variables to join a run since we provide sensible defaults,
though you might need to.

**`NVIDIA_DRIVER_CAPABILITIES`** - An environment variable that the NVIDIA Container Toolkit uses to determine which compute capabilities should be provided to your container. It is recommended to set it to 'all', e.g. `NVIDIA_DRIVER_CAPABILITIES=all`.
Expand Down
16 changes: 11 additions & 5 deletions scripts/setup-and-deploy-solana-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,19 @@ sleep 3

solana airdrop 10 --url ${RPC} --keypair ${WALLET_FILE}

# Pass treasurer flag to deploy script if set
if [[ "$DEPLOY_TREASURER" == "true" && "$PERMISSIONLESS" == "true" ]]; then

if [[ "$DEPLOY_TREASURER" == "true" ]]; then
WALLET_FILE=${WALLET_FILE} ./scripts/deploy-solana-test.sh --treasurer "${EXTRA_ARGS[@]}"
CONFIG_FILE=${CONFIG_FILE} WALLET_FILE=${WALLET_FILE} ./scripts/create-permissionless-run.sh --treasurer "${EXTRA_ARGS[@]}"
elif [[ "$PERMISSIONLESS" == "true" ]]; then
else
WALLET_FILE=${WALLET_FILE} ./scripts/deploy-solana-test.sh "${EXTRA_ARGS[@]}"
CONFIG_FILE=${CONFIG_FILE} WALLET_FILE=${WALLET_FILE} ./scripts/create-permissionless-run.sh "${EXTRA_ARGS[@]}"
fi

if [[ "$PERMISSIONLESS" == "true" ]]; then
if [[ "$DEPLOY_TREASURER" == "true" ]]; then
CONFIG_FILE=${CONFIG_FILE} WALLET_FILE=${WALLET_FILE} ./scripts/create-permissionless-run.sh --treasurer "${EXTRA_ARGS[@]}"
else
CONFIG_FILE=${CONFIG_FILE} WALLET_FILE=${WALLET_FILE} ./scripts/create-permissionless-run.sh "${EXTRA_ARGS[@]}"
fi
fi
echo -e "\n[+] Testing Solana setup ready, starting Solana logs...\n"

Expand Down
1 change: 1 addition & 0 deletions tools/rust-tools/run-manager/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ rand.workspace = true
rand_chacha.workspace = true
time.workspace = true
solana-client = "=2.1.4"
solana-account-decoder-client-types = "=2.1.4"

[dev-dependencies]
serial_test = "3.0"
Expand Down
Loading
Loading