Support for AddMetadataString before instance registration

[cquinn]

It seems like it would be cleaner for some use cases if the instance metadata could be setup first, and then just registered as payload in the instance registration. Adding additional metadata later would be fine as well, but in our case, we have a bunch of static values that we want to include with the instance for its whole lifespan.

I think the main task for this would be to track the instance registration state, and if not registered then skip the PUT part of AddMetadataString()

(I am also seeing some flakiness with the individual PUTs, and some metadata gets lost. Maybe Eureka propagating is sporadic, or setting individual values for an instance using different Eureka nodes is problematic. I need to debug this.)

[cquinn]

I tracked down the problem with metadata not sticking: fargo picks random Eurekas to PUT to, and it seems that Eureka doesn't sync the metadata at a fine grain level--it's all or nothing. So if fargo PUTs the metadata in individual steps to different Eurekas, then only the metadata PUT to the last Eureka will survive.

Another approach to fixing this is to make the Eureka selection sticky or at least stickier. Pick one at random and then keep it until there's a problem, or maybe after a timeout period and then step through to the next one. This is more like what Netflix's Java client does.

I'll submit a PR for this.

[ryansb]

Yeah, availability zone stickiness is on the todo list.

Adding timeout-based stickiness would be the way to go, I think.

[cquinn]

I talked to another guy that used to work at Netflix and worked on Eureka. At Netflix they do all their metadata registration at the initial POST, and rarely use the PUT method.

I'll try to put together a PR with marshaling of the metadata for POST and tracking the registration state so that AddMetadataString() can be used pre-registration. Probably mid week I should be able to get to it.

I think stickiness is a nice feature distinct from this one.

[seh]

I see tests fail regularly due to the replication lag, where the test first posts to one Eureka server, then immediately attempts to read from another server, and doesn't find what it's looking for. One example: TestRegistration. If the test pauses for long enough before proceeding with the call to EurekaConnection.HeartBeatInstance, the test succeeds.