# Nodetalk – Minimal 3-Node Prototype

Nodetalk is a small distributed systems prototype for the UH Distributed Systems course.

The current version does two main things:

1. Runs the same Python server on three UH servers so that all three nodes
   connect to each other over WebSockets and exchange small “hello” messages.
2. Accepts `chat` messages from a separate Python client on any node and
   replicates those chat messages to all three nodes (in memory).

There is still no leader, no global ordering, no database, and no Redis yet.
This is a minimal communication + replication skeleton.

---

## 1. What the code does

We assume exactly three fixed nodes:

- Node **A** on `svm-11.cs.helsinki.fi`
- Node **B** on `svm-11-2.cs.helsinki.fi`
- Node **C** on `svm-11-3.cs.helsinki.fi`

Every **server node**:

- runs a WebSocket **server** on port `9000`
- opens outgoing WebSocket **connections** to the other two nodes (for `hello` messages)
- keeps an in-memory list `MESSAGES` of all `chat` messages it has seen
- accepts WebSocket connections from a **client** (Python CLI, later React)
- when it receives a `chat` from a client:
  - appends it to its local `MESSAGES`
  - broadcasts it once to the other two nodes
- when it receives a **replicated** `chat` from another node:
  - appends it to its local `MESSAGES`
  - does not forward it further (to avoid loops)

So after one client sends a `chat` to any node, **all three nodes** have that message in memory.

### 1.1 Server-to-server “hello” connections

```text
        ws://svm-11:9000             ws://svm-11-2:9000             ws://svm-11-3:9000

  +-------------------+         +-------------------+         +-------------------+
  |   Node A (svm-11) |         |  Node B (svm-11-2)|         |  Node C (svm-11-3)|
  |                   |         |                   |         |                   |
  |  WS server :9000  |         |  WS server :9000  |         |  WS server :9000  |
  |        ^          |         |        ^          |         |        ^          |
  |        |          |         |        |          |         |        |          |
  |   +----+------+   |         |   +----+------+   |         |   +----+------+   |
  |   | outgoing  |---+-------->|   | outgoing  |---+-------->|   | outgoing  |   |
  |   | to B,C    |<------------+   | to A,C    |<------------+   | to A,B    |   |
  |   +-----------+   |         |   +-----------+   |         |   +-----------+   |
  +-------------------+         +-------------------+         +-------------------+

Legend:
- Each node runs a WebSocket server on port 9000.
- Each node also opens outgoing WebSocket connections to the other two nodes.
- When a connection is established, the client side sends JSON like {"type": "hello", ...}.
- The server side prints all received messages.

1.2 Client-to-server chat + replication

Logical flow:

[Client] -- chat --> [Node X] -- replicated chat --> [Node Y]
                               \-- replicated chat --> [Node Z]

• Client sends:

  { "type": "chat", "from": "sender-name", "text": "some message" }

• Receiving node stores it and sends a copy with "replicated": true to the other nodes.

• Replicated messages are stored but not re-broadcast.

All nodes converge to the same sequence of messages in MESSAGES, but there is no strict global ordering or durability yet.

---

2. Code layout

main.py                # entrypoint; parses --node-id and starts the server node

nodetalk/
  __init__.py          # empty, marks this as a Python package
  config.py            # static mapping of node IDs to svm servers and port
  network.py           # WebSocket server, peer connections, in-memory MESSAGES, chat replication

client/
  client.py            # simple CLI client: connect to a node, send and receive messages (for testing purpose)
  clientUI.py          # streamlit based webui for the client

---

3. Dependencies

Python packages:

• websockets
• python-dotenv
• streamlit (optional)

Regular dependencies will get installed and synced automatically when using uv run
Install new dependencies with uv (recommended in this repo)

For example:

uv add websockets

Or with pip:

pip install websockets

Streamlit is an optional dependency and needs to be installed with:

uv sync --extra webui

---

4. How to run on the UH course servers

All three VMs are reachable only through the UH gateway melkki.cs.helsinki.fi. Example username here is kkumar.

4.1 Clone the repo (first time only)

From your laptop:

ssh kkumar@melkki.cs.helsinki.fi

Then, from melkki, for each server:

ssh kkumar@svm-11.cs.helsinki.fi
git clone https://github.com/hojahoja/Nodetalk.git
cd Nodetalk
uv sync   # or ensure 'websockets' is installed

Repeat the same on:

ssh kkumar@svm-11-2.cs.helsinki.fi
ssh kkumar@svm-11-3.cs.helsinki.fi

4.2 Start the three nodes (three terminals)

Open three separate SSH sessions:

Node A – svm-11

ssh kkumar@melkki.cs.helsinki.fi
ssh kkumar@svm-11.cs.helsinki.fi
cd Nodetalk
uv run main.py --node-id A    # or: python main.py --node-id A

Node B – svm-11-2

ssh kkumar@melkki.cs.helsinki.fi
ssh kkumar@svm-11-2.cs.helsinki.fi
cd Nodetalk
uv run main.py --node-id B    # or: python main.py --node-id B

Node C – svm-11-3

ssh kkumar@melkki.cs.helsinki.fi
ssh kkumar@svm-11-3.cs.helsinki.fi
cd Nodetalk
uv run main.py --node-id C    # or: python main.py --node-id C

4.3 Expected output (servers)

On each node you should see logs similar to:

[A] Starting WebSocket server on port 9000...
[A] Ready. Peers: ['B', 'C']
[A] Trying to connect to peer B at ws://svm-11-2.cs.helsinki.fi:9000
[A] Trying to connect to peer C at ws://svm-11-3.cs.helsinki.fi:9000
[A] Sent hello to B
[A] Received from B: {'type': 'hello', 'from': 'B', 'to': 'A'}
...

Once you start sending chat messages (explained below), you will also see lines like:

[A] CHAT from kamlesh: 'hello from client' (total messages here: 1)
[A] Broadcasting chat to B at ws://svm-11-2.cs.helsinki.fi:9000
[A] Broadcasting chat to C at ws://svm-11-3.cs.helsinki.fi:9000

[B] CHAT from kamlesh: 'hello from client' (total messages here: 1)
[C] CHAT from kamlesh: 'hello from client' (total messages here: 1)

If all three nodes show matching CHAT logs with the same text and increasing total messages here, basic replication is working.

---

4.4 With Shell Scripts

Run serverpush.sh to update the server with local changes from your files. On first run it will ask for your username and add them to a local .env file in the root directory. On subsequent runs it will check for your username in the .env file

start.sh will start the main process on each server with unique node ID and leaves them running. It will create a .txt file to track the process id on each server.

stop.sh will check all the create .txt files and close the running processes on each server. After closing each server the corresponding.txt file will be deleted.

4.5 Environmental variables

• REMOTE_USER
  • serverpush saves this variable on first use and will use this for future ssh commands
• NT_PORT
  • Set this manually and the server ports will default to this variable.
  • e.g. NT_PORT=9001

---

5. How to send chat messages (CLI client on svm)

The Python client sends a single chat message to one server node.

From any of the three svm nodes (e.g. svm-11), with the servers already running:

ssh kkumar@melkki.cs.helsinki.fi
ssh kkumar@svm-11.cs.helsinki.fi
cd Nodetalk

Send a chat to node A (svm-11):

uv run client/client.py \
  --server-host svm-11.cs.helsinki.fi \
  --sender kamlesh \
  --text "hello from svm-11"

Send a chat to node B (svm-11-2):

uv run client/client.py \
  --server-host svm-11-2.cs.helsinki.fi \
  --sender kamlesh \
  --text "hello to node B"

Send a chat to node C (svm-11-3):

uv run client/client.py \
  --server-host svm-11-3.cs.helsinki.fi \
  --sender kamlesh \
  --text "hello to node C"

After any of these commands:

• The target node logs the CHAT and broadcasts it.
• The other two nodes log the replicated CHAT.
• Each node’s MESSAGES list grows by one.

---

6. Optional: running the client from your own laptop (SSH tunnel)

The svm machines are on an internal UH network, so direct WebSocket from your laptop to svm-11:9000 will usually time out.

You can use SSH port forwarding to expose a node locally:

On your local machine:

ssh -L 9000:svm-11.cs.helsinki.fi:9000 kkumar@melkki.cs.helsinki.fi

Leave this terminal open. Now ws://localhost:9000 on your laptop is forwarded to svm-11:9000.

In another local terminal (with a local clone of the repo):

cd Nodetalk
uv run client/client.py \
  --server-host localhost \
  --port 9000 \
  --sender laptop \
  --text "hello from my laptop"

Node A (svm-11) will see the CHAT and replicate it to B and C exactly as if the client were running on one of the svm servers.

You can also forward all three nodes if needed:

ssh \
  -L 9000:svm-11.cs.helsinki.fi:9000 \
  -L 9001:svm-11-2.cs.helsinki.fi:9000 \
  -L 9002:svm-11-3.cs.helsinki.fi:9000 \
  kkumar@melkki.cs.helsinki.fi

Then:

• Node A: localhost:9000
• Node B: localhost:9001
• Node C: localhost:9002

---

7. Streamlit chat UI (optional)

If you prefer a browser UI instead of the CLI client:

1. Install the optional extras once:

   uv sync --extra webui

2. Make sure at least one Nodetalk server is already running (locally or via the SSH tunnel above).
3. Launch the UI from your laptop or any machine that can reach the server:

   uv run streamlit run client/clientUI.py

   (Plain streamlit run client/clientUI.py works too if your env is activated.)
4. In the Streamlit sidebar, set the target host/port (localhost:9000 when using SSH forwarding) and enter your display name, then click Connect / Reconnect.
5. Type messages using the chat input at the bottom. They are sent through the Nodetalk server and replicated just like the CLI client messages.

Keep the Streamlit page open; it auto-refreshes every second so new remote messages appear without manual interaction.