Failed DB Lock causes process exit

[beckermarc]

In case a DB Lock can't be obtained, e.g. because the database is in maintenance, the dblock.go utility can cause the process to exit. The process should not exit, but rather obtaining the lock should be retried.

See here for further details: https://github.com/cloudfoundry/app-autoscaler/blob/master/src/autoscaler/sync/dblock.go#L68

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/166325089

The labels on this github issue will be updated when the story is started.

[cdlliuy]

hi, @beckermarc , after double check the db lock logic, I think the implementation is reasonable.

db_lock is a guard for all the other inner operation processes , i.e. metric db pruner, app sync and etc.

if operator A get the lock, then its inner processes start to work. Then, if A lost the lock due to any possible issue, B can take over to get the lock and start its operation processes.
To avoid the inner process in operation A to operation B to run in parallel , operator A must exit to terminate all its inner processes...