From 3c8880333c99483f7e61eca8b19786cf2e5187ea Mon Sep 17 00:00:00 2001 From: vsoch Date: Wed, 25 Jun 2025 21:05:09 -0600 Subject: [PATCH] docs: note on order of starting components flannel requires an annotation to use a host external ip for a multi-node setup. If the ip addresses that are in the private space can be routed between nodes (possible in some clouds) this is not an issue. It is only an issue in an HPC or similar environment where the private 10.x address might go to a router and not be understood (and dropped). We ran into this issue on our HPC system, and I realized it was because of the order of operations - we should make sync-external-ip first (adding the annotation) and then make install-flannel to use it. This would only be a bug for specific, multi-node environments. Signed-off-by: vsoch --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index db1eccce..0907ff09 100644 --- a/README.md +++ b/README.md @@ -103,6 +103,7 @@ pasta does not seem to work well Use scripts in [`./init-host`](./init-host) for automating these steps. ## Usage + See `make help`. ```bash @@ -170,7 +171,7 @@ make up ### Multi-tenancy -Multiple users on the hosts may create their own instances of Usernetes, but the port numbers have to be changed to avoid conflicts. +Multiple users on the hosts may create their own instances of Usernetes. For systems that do not allow the lower port range, or for multiple usernetes deployments on the same physical node (experimental), the port numbers can be changed. ```bash # Default: 2379 @@ -187,6 +188,8 @@ make up ![docs/images/multi-tenancy.png](./docs/images/multi-tenancy.png) +In addition, for multi-host, you will want to `make install-flannel` after `make sync-external-ip` when worker pods are up. The sync command adds an annotation `flannel.alpha.coreos.com/public-ip-overwrite` for flannel to direct the nodes to use the physical node host IP. If the flannel pod has already been created for a node, it would need to be restarted to recheck the annotation. The easiest approach is to install flannel after the annotations have been applied. + ### Rootful mode Although Usernetes (Gen2) is designed to be used with Rootless Docker, it should work with the regular "rootful" Docker too. This might be useful for some people who are looking for "multi-host" version of [`kind`](https://kind.sigs.k8s.io/) and [minikube](https://minikube.sigs.k8s.io/).