Create working setup for Kubernetes

[vniche]

I will start soon working again on a working cete cluster on Kubernetes and was wondering if anyone here already achieved something of a sort 🤔

[mosuka]

I'd like to run a Cete cluster on Kubernetes, but I haven't started anything yet.

[xaocon]

Just started looking at the project. Sounds pretty neat. I have some helm experience and could make something that would make deployment pretty easy but I don't really understand the requirements yet. What state information do nodes need to join a cluster? Seems like maybe you just need one up and then the rest get --peer-grpc-address with the address of the "leader"? Everything else looks like it's pretty standard across each node except for some concept of ID which I think just needs to be different for each node in the cluster?

[vniche]

I've tried a few setups myself @xaocon, i guess the current implementation does not provide a way to let raft itself elect the leader but only to use a previously chosen one (as you mentioned with --peer-grpc-address), for a initial version you could try working with an OrderedReady StatefulSet, pointing to the first node as the leader.

Maybe we could fetch a few ideas from:

• https://github.com/etcd-io/etcd/tree/master/contrib/raftexample
• https://medium.com/@kjanshair_4991/using-etcd-raftexample-for-go-b469aad2eadc
  and reach a dynamic consensus, wether the first healthy node is 0, 1, 2, etc...