diff --git a/docs/guides/01-getting-started-cli.md b/docs/guides/01-getting-started-cli.md index 0c8a641..873f6dc 100644 --- a/docs/guides/01-getting-started-cli.md +++ b/docs/guides/01-getting-started-cli.md @@ -46,8 +46,6 @@ Authentication in `mump2p-cli` is not just about logging in, it enables: * **Usage Tracking**: Monitor your publish/subscription stats. * **Account Linking**: Associate activity with your user or team. -Without authentication, you can only use **open/public topics** with strict limits. - ## How It Fits into the Network ![mump2p CLI Architecture](../../static/img/mump2p.png) @@ -87,21 +85,10 @@ If you prefer manual installation: ./mump2p version ``` -**Output:** - -```bash -Version: v0.0.1-rc5 -Commit: d5b774a -``` - -You can visit [mump2p-cli release page](https://github.com/getoptimum/mump2p-cli/releases) for the latest version. - ---- +> Always use the latest version from [mump2p-cli releases](https://github.com/getoptimum/mump2p-cli/releases) ### 2. Authenticate -*mump2p-cli currently uses [auth0](https://auth0.com/) to manage authentication/authorization*. - Login via device authorization flow: ```sh @@ -162,6 +149,29 @@ Daily Quota: 5120.00 MB ./mump2p logout # Logout ``` +#### Development Mode (No Authentication) + +For testing without authentication: + +```sh +./mump2p --disable-auth --client-id="test-client" publish --topic=demo --message="Hello" --service-url="http://proxy_1:8081" # localhost +./mump2p --disable-auth --client-id="test-client" subscribe --topic=demo --service-url="http://proxy_1:8081" # localhost +``` + +> **Complete Guide:** [Development mode documentation](https://github.com/getoptimum/mump2p-cli/blob/main/docs/guide.md#developmenttesting-mode) - covers all flags, usage, and examples + +#### Custom Authentication Path + +For production deployments or custom storage locations: + +```sh +./mump2p --auth-path /opt/mump2p/auth/token.yml login +# Or use environment variable +export MUMP2P_AUTH_PATH="/opt/mump2p/auth/token.yml" +``` + +> **Complete Guide:** [Custom auth path documentation](https://github.com/getoptimum/mump2p-cli/blob/main/docs/guide.md#custom-authentication-file-location) - use cases, security, deployment + **Refresh Output:** ```bash @@ -196,7 +206,6 @@ Use a custom location --service-url="http://34.142.205.26:8080" ``` ---- ### 4. Subscribe to a Topic @@ -243,22 +252,27 @@ Listening for messages on topic 'demo'... Press Ctrl+C to exit #### Forward to webhook +Basic webhook forwarding: + ```sh ./mump2p subscribe --topic=demo --webhook=https://your-server.com/webhook ``` -**Output:** +**Webhook with custom schema templates:** -```bash -Forwarding messages to webhook: https://your-server.com/webhook -claims is &{google-oauth2|100677750055416883405 2025-08-17 13:15:07 +0530 IST 2025-08-18 13:15:07 +0530 IST true 4194304 1000 8 5368709120 google-oauth2|100677750055416883405 1755416706719} -claims is google-oauth2|100677750055416883405 -Sending HTTP POST subscription request... -HTTP POST subscription successful: {"status":"subscribed","topic":"demo"} -Opening WebSocket connection... -Listening for messages on topic 'demo'... Press Ctrl+C to exit +```sh +# Discord +./mump2p subscribe --topic=alerts --webhook="https://discord.com/api/webhooks/..." --webhook-schema='{"content":"{{.Message}}"}' + +# Slack +./mump2p subscribe --topic=notifications --webhook="https://hooks.slack.com/services/..." --webhook-schema='{"text":"{{.Message}}"}' + +# Custom JSON with metadata +./mump2p subscribe --topic=logs --webhook="https://your-server.com/webhook" --webhook-schema='{"message":"{{.Message}}","timestamp":"{{.Timestamp}}","topic":"{{.Topic}}"}' ``` +> **Complete Guide:** [Webhook formatting documentation](https://github.com/getoptimum/mump2p-cli/blob/main/docs/guide.md#webhook-formatting) - Discord, Slack, Telegram templates, variables, queue options + #### gRPC Subscription For high-performance streaming, use gRPC mode: @@ -277,7 +291,6 @@ HTTP POST subscription successful: {"client":"google-oauth2|10067775005541688340 Listening for messages on topic 'demo' via gRPC... Press Ctrl+C to exit ``` ---- ### 5. Publish a Message @@ -327,7 +340,6 @@ For high-performance publishing, use gRPC mode: ./mump2p publish --topic=demo --message="High reliability" --threshold=0.9 ``` ---- ### 6. Check Usage & Limits @@ -352,9 +364,19 @@ Shows: * Time until reset * Token expiry ---- -### 7. Check Proxy Health +### 7. List Active Topics + +Check which topics you're currently subscribed to: + +```sh +./mump2p list-topics +``` + +> **Complete Guide:** [List topics documentation](https://github.com/getoptimum/mump2p-cli/blob/main/docs/guide.md#list-your-active-topics) - multi-proxy support, output format + + +### 8. Check Proxy Health Monitor the health and system metrics of the proxy server: @@ -379,9 +401,57 @@ Disk Used: 44.91% ./mump2p health --service-url="http://35.221.118.95:8080" ``` ---- -### 8. Common Issues +### 9. Debug Mode + +The `--debug` flag provides detailed timing and proxy information for troubleshooting and performance analysis. + +**What it shows:** + +* Message timestamps (send/receive times in nanoseconds) +* Proxy IP addresses (source and destination) +* Message metadata (size, hash, protocol) +* Sequential message numbering + +**Basic usage:** + +```sh +# Debug publish operations +./mump2p --debug publish --topic=test-topic --message='Hello World' + +# Debug subscribe operations +./mump2p --debug subscribe --topic=test-topic + +# Debug with gRPC +./mump2p --debug publish --topic=test-topic --message='Hello World' --grpc +./mump2p --debug subscribe --topic=test-topic --grpc +``` + +**Example output:** + +Publish: + +```text +Publish: sender_info:34.146.222.111, [send_time, size]:[1757606701424811000, 2010] topic:test-topic msg_hash:4bbac12f protocol:HTTP +``` + +Subscribe: + +```text +Recv: [1] receiver_addr:34.146.222.111 [recv_time, size]:[1757606701424811000, 2082] sender_addr:34.146.222.111 [send_time, size]:[1757606700160514000, 2009] topic:test-topic hash:8da69366 protocol:WebSocket +``` + +**Understanding the output:** + +* `[1]` - Message sequence number +* `[send_time, size]` - Unix timestamp (nanoseconds) and message size +* `msg_hash/hash` - First 8 characters of SHA256 hash for tracking +* Calculate latency: `recv_time - send_time` + +> **Complete Guide:** [Debug mode documentation](https://github.com/getoptimum/mump2p-cli/blob/main/docs/guide.md#debug-mode) - blast testing, load testing, performance analysis + + +### 10. Common Issues #### Unauthorized @@ -439,7 +509,8 @@ Error: authentication required: token has expired, please login again → Run `./mump2p login` to authenticate. -### 9. Important Tips + +### 11. Important Tips * Use descriptive topic names per team. * Keep `whoami` and `usage` handy. @@ -449,12 +520,12 @@ Error: authentication required: token has expired, please login again * Use the `--service-url` flag to connect to different gateways for better performance. * Use `--grpc` flag for high-performance streaming and publishing. * Monitor proxy health with `./mump2p health` for troubleshooting. +* Use `--debug` flag for performance monitoring and troubleshooting. +* Check active topics with `./mump2p list-topics` to manage subscriptions. ---- - -#### Important Links +## Complete Documentation -* [mump2p CLI Source Code](https://github.com/getoptimum/mump2p-cli) -* [Developer Guide](https://github.com/getoptimum/mump2p-cli/blob/main/docs/guide.md) -* [Release Page](https://github.com/getoptimum/mump2p-cli/releases) -* [Available Service URLs](https://github.com/getoptimum/mump2p-cli?tab=readme-ov-file#3-service-url--connectivity-issues) +* **[Complete User Guide](https://github.com/getoptimum/mump2p-cli/blob/main/docs/guide.md)** - Advanced features, authentication, webhooks, debug mode +* **[CLI Repository](https://github.com/getoptimum/mump2p-cli)** - Source code and documentation +* **[FAQ & Troubleshooting](https://github.com/getoptimum/mump2p-cli#faq---common-issues--troubleshooting)** - Common issues and solutions +* **[Latest Releases](https://github.com/getoptimum/mump2p-cli/releases)** - Download latest version diff --git a/docs/guides/02-getting-started-docker.md b/docs/guides/02-getting-started-docker.md index 3dfce53..77e800f 100644 --- a/docs/guides/02-getting-started-docker.md +++ b/docs/guides/02-getting-started-docker.md @@ -40,8 +40,8 @@ Choose the deployment mode that best fits your use case: **Quick Decision:** -* **Want simpler setup and client code?** → **[Start with Mode A](#4-mode-a--optimumproxy--mump2p-recommended)** -* **Need maximum performance and control?** → **[Jump to Mode B](#5-mode-b--direct-mump2p-advanced--lower-latency)** +* **Want simpler setup and client code?** → **[Start with Mode A](#5-mode-a--optimumproxy--mump2p-recommended)** +* **Need maximum performance and control?** → **[Jump to Mode B](#6-mode-b--direct-mump2p-advanced--lower-latency)** ## 1. Before You Start @@ -88,192 +88,95 @@ We’ll keep identity in `./identity` folder so you can reuse keys across restar | **Direct mumP2P** | Fewer hops, you control connection/retry logic and node selection | -## 3. Generate a Bootstrap Identity (once) +## 3. Environment Configuration -mumP2P nodes need a **P2P identity** (cryptographic keypair) for peer-to-peer communication. The bootstrap node needs a persistent identity so other nodes can discover it reliably. +Before starting, create your `.env` file: -**What is P2P Identity?** - -* A cryptographic private key stored in `identity/p2p.key` -* Used for peer authentication and discovery -* Generates a unique **Peer ID** (like `12D3KooW...`) that other nodes use to connect +```bash +cp .env.example .env +``` -### Quick One-Command Setup +Edit with your values: ```bash -curl -sSL https://raw.githubusercontent.com/getoptimum/optimum-dev-setup-guide/main/script/generate-identity.sh | bash +BOOTSTRAP_PEER_ID= +CLUSTER_ID=my-cluster +PROXY_VERSION=v0.0.1-rc7 +P2P_NODE_VERSION=v0.0.1-rc6 ``` -This script: +> **Complete Guide:** [Environment configuration](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#environment-variables-env) -* Creates `./identity/` directory -* Generates P2P keypair using the existing keygen utility -* Saves to `identity/p2p.key` with proper checksum format -* Exports `BOOTSTRAP_PEER_ID` environment variable -* Handles existing identity gracefully -* Uses the correct file format expected by mumP2P nodes +## 4. Generate a Bootstrap Identity -**Output:** +Generate P2P identity for node discovery: ```bash -[INFO] Generating P2P Bootstrap Identity... -[INFO] Creating identity directory... -[INFO] Using existing keygen script... -[INFO] Generating P2P keypair... -[SUCCESS] Generated P2P identity successfully! -[SUCCESS] Identity saved to: ./identity/p2p.key -[SUCCESS] Peer ID: 12D3KooWLsSmLLoE2T7JJ3ZyPqoXEusnBhsBA1ynJETsziCKGsBw -[INFO] To use in docker-compose: -export BOOTSTRAP_PEER_ID=12D3KooWLsSmLLoE2T7JJ3ZyPqoXEusnBhsBA1ynJETsziCKGsBw -[SUCCESS] Done! Your mumP2P peer ID: 12D3KooWLsSmLLoE2T7JJ3ZyPqoXEusnBhsBA1ynJETsziCKGsBw +make generate-identity ``` -**What this creates:** +This creates `./identity/p2p.key` with your unique Peer ID. -* `./identity/p2p.key` — JSON file containing the private key and peer ID -* `BOOTSTRAP_PEER_ID` environment variable for use in docker-compose files +> **Complete Guide:** [Identity generation and Makefile commands](https://github.com/getoptimum/optimum-dev-setup-guide#quick-start) - all make commands, direct binary usage -> **Note:** This process creates a persistent identity that will be reused across container restarts. The `identity/` folder is mounted into containers so the same keypair is shared by the bootstrap node. +## 5. Mode A — OptimumProxy + mumP2P (Recommended) -This guide covers: +### Docker Compose Setup -* Setting up Docker Compose for both approach. -* Running and verifying the network. -* Connecting via CLI (`mump2p-cli`) or `gRPC clients` (Go examples included). -* Adjusting key parameters for your environment. +**Key points:** -## 4. Mode A — OptimumProxy + mumP2P (Recommended) +* Use `.env` variables for versions and cluster ID +* Network uses static IPs for deterministic bootstrap addresses +* Bootstrap node (p2pnode-1) needs identity volume mount +* Production setup uses 2 proxies and 4 P2P nodes -### Create the docker-compose file - -Save as `./proxy-p2p/docker-compose.yml`: +**Simplified example:** ```yaml services: - proxy: - image: 'getoptimum/proxy:v0.0.1-rc3' - platform: linux/amd64 + proxy-1: + image: 'getoptimum/proxy:${PROXY_VERSION-latest}' environment: - - PROXY_PORT=:8080 # internal port, mapped below - - CLUSTER_ID=proxy-1 - - ENABLE_AUTH=false # set true in prod; see "Auth" below - - LOG_LEVEL=debug - # list of P2P node sidecar (gRPC) addresses (container DNS names) + - CLUSTER_ID=${CLUSTER_ID} - P2P_NODES=p2pnode-1:33212,p2pnode-2:33212 - depends_on: - - p2pnode-1 - - p2pnode-2 ports: - - "8081:8080" # HTTP/WebSocket API (POST /publish, WS /ws, GET /health) - - "50051:50051" # (optional) proxy gRPC, if exposed by your proxy build - networks: - testnet: - ipv4_address: 172.28.0.10 + - "8081:8080" + - "50051:50051" p2pnode-1: - image: 'getoptimum/p2pnode:v0.0.1-rc2' - platform: linux/amd64 + image: 'getoptimum/p2pnode:${P2P_NODE_VERSION-latest}' volumes: - - ../identity:/identity - environment: - - LOG_LEVEL=debug - - CLUSTER_ID=p2pnode-1 - - NODE_MODE=optimum # or gossipsub - - IDENTITY_DIR=/identity - # Ports - - SIDECAR_PORT=33212 # client/proxy gRPC port - - API_PORT=9090 # node REST API - - OPTIMUM_PORT=7070 # inter-node P2P - # Mesh & RLNC - - OPTIMUM_MESH_TARGET=6 - - OPTIMUM_MESH_MIN=3 - - OPTIMUM_MESH_MAX=12 - - OPTIMUM_MAX_MSG_SIZE=1048576 - - OPTIMUM_SHARD_FACTOR=4 - - OPTIMUM_SHARD_MULT=1.5 - - OPTIMUM_THRESHOLD=0.75 - ports: - - "33221:33212" # sidecar gRPC (host) - - "9091:9090" # node API (host) - - "7071:7070" # P2P (host; optional if single host only) - networks: - testnet: - ipv4_address: 172.28.0.11 - - p2pnode-2: - image: 'getoptimum/p2pnode:latest' - platform: linux/amd64 + - ./identity:/identity environment: - - LOG_LEVEL=debug - - CLUSTER_ID=p2pnode-2 + - CLUSTER_ID=${CLUSTER_ID} - NODE_MODE=optimum - - SIDECAR_PORT=33212 - - API_PORT=9090 - - OPTIMUM_PORT=7070 - - OPTIMUM_MAX_MSG_SIZE=1048576 - - OPTIMUM_MESH_TARGET=6 - - OPTIMUM_MESH_MIN=3 - - OPTIMUM_MESH_MAX=12 - - OPTIMUM_SHARD_FACTOR=4 - - OPTIMUM_SHARD_MULT=1.5 - - OPTIMUM_THRESHOLD=0.75 - # Bootstrap to node-1 (libp2p multiaddr) - - BOOTSTRAP_PEERS=/ip4/172.28.0.11/tcp/7070/p2p/${BOOTSTRAP_PEER_ID} - ports: - - "33222:33212" - - "9092:9090" - - "7072:7070" - networks: - testnet: - ipv4_address: 172.28.0.12 - -networks: - testnet: - driver: bridge - ipam: - config: - - subnet: 172.28.0.0/16 + - IDENTITY_DIR=/identity ``` -*Why fixed IPs?* - -It makes the bootstrap address deterministic `(/ip4/172.28.0.11/tcp/7070/p2p/)`. -You can use the default bridge and service names, but then set `BOOTSTRAP_PEERS` to a reachable address. - -See the [Parameter Section](./03-parameters.md) for config options and port usage. +> **Complete Docker Compose:** +> +> * [Full configuration with 2 proxies + 4 nodes](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docker-compose-optimum.yml) +> * [All environment variables](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#p2p-node-variables) ### Start the Network ```sh -cd ~/optimum-local/proxy-p2p -export BOOTSTRAP_PEER_ID= +export BOOTSTRAP_PEER_ID= docker compose up -d ``` -### Validate everything is healthy +### Verify Health ```sh +# Check containers docker compose ps -docker compose logs -f proxy | sed -n '1,120p -``` - -#### Health checks - -```sh -# Proxy -curl -s http://localhost:8081/api/v1/version -curl -s http://localhost:8081/api/v1/health -# Nodes -curl -s http://localhost:9091/api/v1/health -curl -s http://localhost:9092/api/v1/health - -# Peers seen by node-2 (should include node-1) -curl -s http://localhost:9092/api/v1/node-state | jq '.peers' +# Test endpoints +curl http://localhost:8081/api/v1/health # Proxy +curl http://localhost:9091/api/v1/health # P2P node ``` -You should see the mesh forming and node-2 discovering node-1 via the bootstrap address. - +> **Complete Testing Guide:** [Health checks and validation](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#3-verification) ### Send & receive (Proxy mode) using mump2p-cli @@ -294,349 +197,92 @@ If you haven't already installed `mump2p-cli`, see the [**Getting Started with m You should see your subscriber print the message immediately. -### Use Proxy using REST API and WebSocket (optional) +### Use Proxy via REST API (Optional) -**Publish a message:** +**Basic commands:** ```sh +# Publish curl -X POST http://localhost:8081/api/v1/publish \ -H "Content-Type: application/json" \ - -d '{ - "client_id": "your-client-id", - "topic": "example-topic", - "message": "Hello, world!" -}' -``` - -**Parameters:** - -* `client_id` – Unique identifier for the client (required) -* `topic` – The topic to publish the message to -* `message` – The content to broadcast to subscribers - -> **Important:** The `client_id` field is required for all publish requests. This should be the same ID used when subscribing to topics. If you're using WebSocket connections, use the same `client_id` for consistency. + -d '{"client_id":"test","topic":"demo","message":"Hello"}' -**Subscribe to a topic:** - -```sh +# Subscribe curl -X POST http://localhost:8081/api/v1/subscribe \ -H "Content-Type: application/json" \ - -d '{ - "client_id": "unique-client-id", - "topic": "example-topic", - "threshold": 0.7 -}' -``` + -d '{"client_id":"test","topic":"demo","threshold":0.1}' -**Parameters:** +# WebSocket +wscat -c "ws://localhost:8081/api/v1/ws?client_id=test" +``` -* `client_id` – Unique identifier for the client (required) -* `topic` – The topic to subscribe to -* `threshold` (optional, float) – Minimum percentage (0.1–1.0) of active nodes that must report a message before it's forwarded to this client - * Default: 0.1 (10% confirmation) +> **Complete API Reference:** [Proxy REST and WebSocket API](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#proxy-rest-api) - parameters, rate limits, authentication -**Connect via WebSocket:** +### Use Proxy via gRPC (Optional) -```sh -wscat -c "ws://localhost:8081/api/v1/ws?client_id=unique-client-id" -``` +For gRPC bidirectional streaming (higher performance than WebSocket): -> **Important:** WebSocket has limitations, and you may experience unreliable delivery when publishing message bursts. A gRPC connection (shown below) provides more reliable streaming. - -**Rate Limits:** - -Rate limits are enforced based on client configuration. Exceeding limits results in 429 responses. - -> **Note:** Since authentication is disabled in our local setup (`ENABLE_AUTH=false`), no JWT tokens are required for these requests. - -### Use Proxy using gRPC Stream - -For a complete working proxy client with both REST subscription and gRPC streaming, see the full implementation: - -**[Complete Proxy Client Example](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#grpc-proxy-client-implementation)** - -**[Complete Code](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_proxy_client/proxy_client.go)** - -The proxy client provides: - -* **REST API subscription** for topic registration and threshold configuration -* **gRPC bidirectional streaming** for real-time message delivery -* **Message publishing** via REST API endpoints -* **Configurable parameters** for topic, threshold, and message count -* **Flow control settings** for robust connections - -```go -// Basic proxy client implementation (see full version in GitHub link above) -package main - -import ( - "bytes" - "context" - "encoding/json" - "fmt" - "io" - "log" - "math" - "net/http" - "time" - - "google.golang.org/grpc" - "google.golang.org/grpc/credentials/insecure" - - protobuf "proxy_client/grpc" // Generated from gateway_stream.proto -) - -const ( - proxyREST = "http://localhost:8081" // REST API for subscription/publishing - proxyGRPC = "localhost:50051" // gRPC endpoint for streaming -) - -func main() { - clientID := "client_demo123" - topic := "demo" - threshold := 0.1 - - // 1. Subscribe via REST API - body := map[string]interface{}{ - "client_id": clientID, - "topic": topic, - "threshold": threshold, - } - data, _ := json.Marshal(body) - resp, err := http.Post(proxyREST+"/api/v1/subscribe", "application/json", bytes.NewReader(data)) - if err != nil { - log.Fatalf("subscription failed: %v", err) - } - resp.Body.Close() - - // 2. Connect to gRPC stream - conn, err := grpc.NewClient(proxyGRPC, - grpc.WithTransportCredentials(insecure.NewCredentials()), - grpc.WithInitialWindowSize(1024*1024*1024), // 1GB per-stream receive window - grpc.WithInitialConnWindowSize(1024*1024*1024), // 1GB connection-level receive window - grpc.WithDefaultCallOptions( - grpc.MaxCallRecvMsgSize(math.MaxInt), - grpc.MaxCallSendMsgSize(math.MaxInt), - ), - ) - if err != nil { - log.Fatalf("gRPC connection failed: %v", err) - } - defer conn.Close() - - client := protobuf.NewProxyStreamClient(conn) - stream, err := client.ClientStream(context.Background()) - if err != nil { - log.Fatalf("stream creation failed: %v", err) - } - - // 3. Send client ID to establish stream - if err := stream.Send(&protobuf.ProxyMessage{ClientId: clientID}); err != nil { - log.Fatalf("client ID send failed: %v", err) - } - - // 4. Handle incoming messages - go func() { - for { - resp, err := stream.Recv() - if err == io.EOF { - log.Println("Stream closed by server") - return - } - if err != nil { - log.Printf("Stream receive error: %v", err) - return - } - log.Printf("Received: Topic=%s, Message=%s", resp.Topic, string(resp.Message)) - } - }() - - // 5. Publish messages via REST API - for i := 0; i < 3; i++ { - msg := fmt.Sprintf("Hello message %d @ %s", i+1, time.Now().Format("15:04:05")) - publishBody := map[string]interface{}{ - "client_id": clientID, - "topic": topic, - "message": msg, - } - publishData, _ := json.Marshal(publishBody) - - log.Printf("Publishing: %s", msg) - resp, err := http.Post(proxyREST+"/api/v1/publish", "application/json", bytes.NewReader(publishData)) - if err != nil { - log.Printf("Publish error: %v", err) - } else { - resp.Body.Close() - } - - time.Sleep(2 * time.Second) - } - - // Keep client running to receive messages - time.Sleep(10 * time.Second) -} -``` +> **Complete Implementation:** +> +> * [Proxy gRPC Client Source](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_proxy_client/proxy_client.go) +> * [Setup and Usage Guide](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#grpc-proxy-client-implementation) +> * REST subscription + gRPC streaming + flow control settings -## 5. Mode B — Direct mumP2P (Advanced / Lower Latency) +## 6. Mode B — Direct mumP2P (Advanced / Lower Latency) -In this mode, clients connect `straight to node sidecar gRPC`. You’ll manage client-side reconnection, backoff, and which node to hit. +In this mode, clients connect directly to node sidecar gRPC (no proxy). -### Create the docker-compose file +### Docker Compose Setup -Save as `./direct-p2p/docker-compose.yml`: +**Simplified example:** ```yaml services: p2pnode-1: - image: 'getoptimum/p2pnode:latest' - platform: linux/amd64 + image: 'getoptimum/p2pnode:${P2P_NODE_VERSION-latest}' volumes: - - ../identity:/identity + - ./identity:/identity environment: - - LOG_LEVEL=debug - - CLUSTER_ID=p2pnode-1 + - CLUSTER_ID=${CLUSTER_ID} - NODE_MODE=optimum - IDENTITY_DIR=/identity - - SIDECAR_PORT=33212 - - API_PORT=9090 - - OPTIMUM_PORT=7070 - - OPTIMUM_MESH_TARGET=6 - - OPTIMUM_MESH_MIN=3 - - OPTIMUM_MESH_MAX=12 - - OPTIMUM_MAX_MSG_SIZE=1048576 - - OPTIMUM_SHARD_FACTOR=4 - - OPTIMUM_SHARD_MULT=1.5 - - OPTIMUM_THRESHOLD=0.75 ports: - "33221:33212" - - "9091:9090" - - "7071:7070" - networks: - testnet: - ipv4_address: 172.28.0.11 - - p2pnode-2: - image: 'getoptimum/p2pnode:latest' - platform: linux/amd64 - environment: - - LOG_LEVEL=debug - - CLUSTER_ID=p2pnode-2 - - NODE_MODE=optimum - - SIDECAR_PORT=33212 - - API_PORT=9090 - - OPTIMUM_PORT=7070 - - OPTIMUM_MESH_TARGET=6 - - OPTIMUM_MESH_MIN=3 - - OPTIMUM_MESH_MAX=12 - - OPTIMUM_MAX_MSG_SIZE=1048576 - - OPTIMUM_SHARD_FACTOR=4 - - OPTIMUM_SHARD_MULT=1.5 - - OPTIMUM_THRESHOLD=0.75 - - BOOTSTRAP_PEERS=/ip4/172.28.0.11/tcp/7070/p2p/${BOOTSTRAP_PEER_ID} - ports: - - "33222:33212" - - "9092:9090" - - "7072:7070" - networks: - testnet: - ipv4_address: 172.28.0.12 - -networks: - testnet: - driver: bridge - ipam: - config: - - subnet: 172.28.0.0/16 ``` -### Start and validate +> **Complete Docker Compose:** [Full configuration for direct P2P mode](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docker-compose-optimum.yml) + +### Start and Verify ```sh -cd ~/optimum-local/direct-p2p -export BOOTSTRAP_PEER_ID= +export BOOTSTRAP_PEER_ID= docker compose up -d -docker compose ps -curl -s http://localhost:9091/api/v1/health -curl -s http://localhost:9092/api/v1/health +curl http://localhost:9091/api/v1/health ``` -### Minimal Direct P2P sidecar gRPC stream client - -For a complete working P2P client that connects directly to nodes, see the full implementation with trace handling: - -**[Complete P2P Client Example](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#using-p2p-nodes-directly-optional--no-proxy)** - -**[Complete Code](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/p2p_client.go)** - -The client includes: - -* **Message publishing and subscribing** with gRPC streaming -* **Protocol trace handling** for both GossipSub and mumP2P -* **Metrics collection** via `MessageTraceGossipSub` and `MessageTraceOptimumP2P` responses -* **Stress testing capabilities** with batch message publishing - -```go -// Basic client skeleton (see full implementation in GitHub link above) -package main - -import ( - "context" - "fmt" - "log" - "math" - - "google.golang.org/grpc" - "google.golang.org/grpc/credentials/insecure" - - protobuf "p2p_client/grpc" // Generated from p2p_stream.proto -) - -func main() { - conn, err := grpc.NewClient("localhost:33221", - grpc.WithTransportCredentials(insecure.NewCredentials()), - grpc.WithDefaultCallOptions( - grpc.MaxCallRecvMsgSize(math.MaxInt), - grpc.MaxCallSendMsgSize(math.MaxInt), - ), - ) - if err != nil { log.Fatal(err) } - defer conn.Close() - - client := protobuf.NewCommandStreamClient(conn) - stream, _ := client.ListenCommands(context.Background()) - - // Subscribe to topic - stream.Send(&protobuf.Request{ - Command: int32(CommandSubscribeToTopic), - Topic: "demo", - }) - - // Handle responses including trace data - for { - resp, _ := stream.Recv() - switch resp.GetCommand() { - case protobuf.ResponseType_Message: - fmt.Printf("MSG: %s\n", string(resp.GetData())) - case protobuf.ResponseType_MessageTraceGossipSub: - fmt.Printf("[TRACE] GossipSub trace: %d bytes\n", len(resp.GetData())) - case protobuf.ResponseType_MessageTraceOptimumP2P: - fmt.Printf("[TRACE] mumP2P trace: %d bytes\n", len(resp.GetData())) - } - } -} -``` +### Use P2P Client Directly -For all available configuration variables, observability and validations check the [Parameters Section](./03-parameters.md). +Connect using the P2P client with trace handling and metrics: -## Troubleshooting +```bash +# Subscribe +./p2p-client -mode=subscribe -topic=testtopic --addr=127.0.0.1:33221 -### No peers / mesh empty +# Publish +./p2p-client -mode=publish -topic=testtopic -msg="Hello" --addr=127.0.0.1:33222 +``` + +> **Complete Implementation:** +> +> * [P2P Client Source](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/grpc_p2p_client/p2p_client.go) +> * [Usage Guide with Makefile commands](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#using-p2p-nodes-directly-optional--no-proxy) +> * [Message format explanation](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#understanding-message-output-format) -* Check `BOOTSTRAP_PEERS` uses a reachable address of a running node. -* If using static IPs, confirm the `testnet` subnet and container IPs match the compose file. -* Look for `bootstrap` lines in docker compose logs `p2pnode`. +For all configuration variables, see the [Parameters Section](./03-parameters.md). + +## Troubleshooting -### “connection refused” from client +### "Connection refused" from client * Ensure you’re pointing to the host-mapped ports (e.g., 33221, 8081). * Run docker compose ps and confirm port bindings. diff --git a/docs/guides/03-parameters.md b/docs/guides/03-parameters.md index 48d5b9a..118cfa1 100644 --- a/docs/guides/03-parameters.md +++ b/docs/guides/03-parameters.md @@ -1,5 +1,7 @@ # Understanding Key Parameters in mumP2P +> **Complete Reference:** [Full parameter guide and defaults](https://github.com/getoptimum/optimum-dev-setup-guide/blob/main/docs/guide.md#configuration) - all variables, `.env` workflow, production configs + mumP2P nodes can operate in **two distinct protocol modes**, configured via: ```sh @@ -23,20 +25,14 @@ Example Docker service: ```yaml p2pnode-1: - image: 'getoptimum/p2pnode:latest' + image: 'getoptimum/p2pnode:${P2P_NODE_VERSION-latest}' environment: - NODE_MODE=optimum - - LOG_LEVEL=debug - - CLUSTER_ID=p2pnode-1 + - CLUSTER_ID=${CLUSTER_ID} - OPTIMUM_PORT=7070 - - OPTIMUM_MAX_MSG_SIZE=1048576 - OPTIMUM_MESH_TARGET=6 - - OPTIMUM_MESH_MIN=3 - - OPTIMUM_MESH_MAX=12 - OPTIMUM_SHARD_FACTOR=4 - - OPTIMUM_SHARD_MULT=1.5 - OPTIMUM_THRESHOLD=0.75 - - BOOTSTRAP_PEERS=/ip4/172.28.0.12/tcp/7070/p2p/${BOOTSTRAP_PEER_ID} ``` ### Parameters @@ -66,7 +62,7 @@ Example Docker service: ```yaml p2pnode-1: - image: 'getoptimum/p2pnode:latest' + image: 'getoptimum/p2pnode:${P2P_NODE_VERSION-latest}' environment: - NODE_MODE=gossipsub - LOG_LEVEL=debug @@ -113,7 +109,7 @@ Example Docker service: ```yaml proxy-1: - image: 'getoptimum/proxy:latest' + image: 'getoptimum/proxy:${PROXY_VERSION-latest}' environment: - PROXY_HTTP_PORT=:8080 - PROXY_GRPC_PORT=:50051