Make attache-control a long running sidecar (#24)

Previous to this change, we've avoided hashicorp/nomad#10058 by adding a long
sleep before attache-control would otherwise exit successfully. Unfortunately
this still means that subsequent scaling deployments will fail if the sleep
incurred from the initial deployment has expired.

This change makes attache-control a proper sidecar (long running ephemeral
post-start task). Instead of exiting after a successful run, attache-control
will stop the ticker channel and continue running until it receives a kill
signal.

- Configure attache-control as a side-car in the example nomad job
- Remove attempt-limit flag and cliOpts.attemptLimit
- Split the main func of attache-control into helpers
- Add helper methods to scalingOpts 
- Replace calls to time.Tick with time.NewTicker
- Add renewChan (chan struct{}) and acquired (bool) to the lock struct
- lock.Acquire() sets lock.acquired to true, initializes lock.renewChan, and calls
  lock.periodicallyRenew() in a long-running go-routine
- lock.Cleanup() closes lock.renewChan and sets lock.acquired to false

Fixes #6
Part Of #11

Author: beautifulentropy

README.md

@@ -57,8 +57,6 @@ $ ./attache-control -help
 Usage of ./attache-control:
   -attempt-interval duration
     	Duration to wait between attempts to join or create a cluster (e.g. '1s') (default 3s)
-  -attempt-limit int
-    	Number of times to join or create a cluster before exiting (default 20)
   -await-service-name string
     	Consul Service for newly created Redis Cluster Nodes, (required)
   -consul-acl-token string

cmd/attache-control/config.go

@@ -20,10 +20,6 @@ type cliOpts struct {
 	// cluster.
 	attemptInterval time.Duration
 
-	// attemptLimit is the number of times to attempt joining or creating a cluster before Attache
-	// should exit as failed.
-	attemptLimit int
-
 	// awaitServiceName is the name of the Consul Service that newly created
 	// Redis Cluster nodes will join when they're first started but have yet to
 	// form or join a cluster. This field is required.
@@ -102,7 +98,6 @@ func ParseFlags() cliOpts {
 	// CLI
 	flag.StringVar(&conf.lockPath, "lock-kv-path", "service/attache/leader", "Consul KV path to use as a leader lock for Redis Cluster operations")
 	flag.DurationVar(&conf.attemptInterval, "attempt-interval", 3*time.Second, "Duration to wait between attempts to join or create a cluster (e.g. '1s')")
-	flag.IntVar(&conf.attemptLimit, "attempt-limit", 20, "Number of times to attempt join or create a cluster before exiting")
 	flag.StringVar(&conf.awaitServiceName, "await-service-name", "", "Consul Service for newly created Redis Cluster Nodes, (required)")
 	flag.StringVar(&conf.destServiceName, "dest-service-name", "", "Consul Service for healthy Redis Cluster Nodes, (required)")
 	flag.StringVar(&conf.logLevel, "log-level", "info", "Set the log level")