Allow node leaves to happen when still joining #208
Description
It would be great if a user could leave a node even if it is still in the process of joining the cluster. As it is now, a node that joins a cluster must wait to be handed all of its data before it can leave. In other words: wait until a bunch of data is transfered to the node, just so the node can hand all the data off to other nodes and exit. Not particularly efficient.
The key to this change is to ensure that data already transfered to the node is handed off before leaving. Ideally, we should hand it back to the original owner if that's safe. The hard part is ensuring that the ring continues to meet guarantees (ie. all replicas on distinct nodes) without falling back to rediagonializing the ring. Which largely explains why this issue was punted on thus far. Some of the newer/planned claim work should help address this.