Skip to content

Full End-To-End Inference Flow and Gateway Node Implementation #487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
samherring99 wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Full End-To-End Inference Flow and Gateway Node Implementation #487

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
efe54c0
Updating protocol with InferenceMessage and InferenceGossipMessage ty…
samherring99 Dec 23, 2025
7247636
Adding gateway node binary file source and adding protocol handler to…
samherring99 Dec 29, 2025
ab84ca0
Adding additional protocols vec to P2PNetwork init, updating protocol…
samherring99 Jan 6, 2026
f619a9e
Bootstrapping from file and env variable, adding direct connection P2…
samherring99 Jan 6, 2026
a4834ed
Adding stop tokens and chat templating to rust bridge, updating Cargo…
samherring99 Jan 7, 2026
a025430
Migrating all code to OpenAI API compatible chat completions endpoint
samherring99 Jan 13, 2026
9851fa1
Updating init method changes for shared network to have an optional P…
samherring99 Jan 14, 2026
84557ad
Moving bootstrap logic to new lib file in inference crate and changin…
samherring99 Jan 21, 2026
8d55cd0
Removing gateway feature flag, adding task set for tracking spawned t…
samherring99 Jan 22, 2026
c6f4439
Updating scripts and justfile to use nix run
samherring99 Feb 3, 2026
87be8d1
Changing relay and discovery mode to be defaulted CLI args
samherring99 Feb 4, 2026
4dbfac2
fix weird bash devshell
arilotter Feb 4, 2026
02e71b5
Fixing merge conflicts
samherring99 Feb 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,771 changes: 965 additions & 806 deletions Cargo.lock
View file
Open in desktop

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ iroh = { version = "0.96.0", features = ["metrics"] }
iroh-relay = { version = "0.96.0", features = ["metrics"] }
iroh-blobs = { version = "0.98", features = ["metrics"] }
iroh-gossip = { version = "0.96.0" }
axum = "0.7"
tower = "0.5"
tower-http = { version = "0.6", features = ["trace"] }
flume = "0.11.1"
memmap2 = { version = "0.9.3", features = ["stable_deref_trait"] }
indicatif = "0.17.5"
Expand Down Expand Up @@ -106,6 +109,10 @@ ts-rs = { git = "https://github.com/arilotter/ts-rs.git", rev = "92ce1752227fec9
] }
rayon = "1.10.0"

# with iroh update need to specify digest and crypto-common versions
digest = { version = "=0.11.0-rc.10", default-features = false }
crypto-common = "0.2"

# dev only
test-log = { version = "0.2.16", features = ["trace"] }
pretty_assertions = "1.4.1"
Expand Down
12 changes: 12 additions & 0 deletions architectures/inference-only/inference-node/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ path = "src/main.rs"
name = "test-network"
path = "src/bin/test-network.rs"

[[bin]]
name = "gateway-node"
path = "src/bin/gateway-node.rs"

[dependencies]
psyche-inference.workspace = true
psyche-network.workspace = true
Expand All @@ -33,3 +37,11 @@ iroh-blobs.workspace = true
iroh-gossip.workspace = true

serde_json.workspace = true
serde.workspace = true
uuid = { version = "1", features = ["v4"] }
pyo3.workspace = true
postcard.workspace = true

axum = { version = "0.7", features = ["macros"] }
tower = { version = "0.4" }
tower-http = { version = "0.5", features = ["cors"] }
Loading
Loading