Confusing error - setting location before replace

[martinsumner]

When running a replace of an existing node, the new node should first be joined. However, it is not possible to set the location before the replace command - if the replace is prompted after the location has been set, the following error is returned:

riak admin cluster replace dev@192.168.53.150 dev@192.168.53.151
Failed: 'dev@192.168.53.151' is not a valid replacement candidate.
Only newly joining nodes can be used for replacement.
error

The setting of location is considered a change, and so as there is a change to perform between the join and the replace - the joining node is then not considered to be "newly joined" even though the join has not yet been planned or committed.

When using locations, and replacing a node - do not set the location prior to the replace. This will cause the expected behaviour of all vnodes simply being transferred from the replaced not to the joined node - regardless of location.

If the new node is the same location as the old, setting the location after the replace has completed, should lead to a null cluster change. If the new node is in a different location, and the change is permanent - then a separate cluster change to set the location (and reshuffle the vnodes if necessary) should be staged once the replace is complete.

[martinsumner]

Raised as an issue - a clearer error should be returned, and the expected process should be documented.