Implement optional locking mechanism #3
Description
The tool would benefit from using a distributed mutex "lock" at so multiple instances of this tool doesn't run at the same time.
It might be possible to use a lockfile on the s3 destination itself, but using there is a high risk that is not mutually exclusive because of the 'read lockfile first, then write lockfile'. Example of race:
- A: tries to find lockfile via s3 getObject call
- B: tries to find lockfile via s3 getObject call
- A: sees no lockfile, creates lockfile via s3 putObject call
- B: sees no lockfile, creates lockfile via s3 putObject call
If this functionality is really needed and it's important that they do not run at the same time, a better option might be to use dynamodb or any other service that allows "upsert with conditions"