Replace Terrapin with BuildExecution to fix process deadlock when output to stderr is fills buffer

[gyfelton]

Why this change

In our CI cluster, we run into fastclone got stuck because of this bug: thoughtbot/terrapin#5
We verified that after this change it run smoothly after consistently reproducing it.
Our codebase is so big that even git remote update --prune returns a huge output causing this problem to manifest
When we interrupt the stuck process, we also see it stuck at the read stream function as indicated by above PR
Also verbose mode does not redirect output from git operations inside Terrapin to stdout for some reason. Therefore we think this is a good fix.

Also add log when cleaning cache because when lots of CI machiens got cache cleaned, we will throttle the network by trying to populate the cache at the same time. So adding log to warn about this.

• Fix spec, should have spec failing or need to add some
• Update version file and Gemfile.lock

[justinseanmartin]

I wonder if we should be more thoughtful here and only clear the cache directory in the case of certain failures that we know aren't recoverable, given that removing the cache can lead to thundering herd issues for us.

For example, for the recent issue where we had two case insensitive matching branch names, I don't think clearing the cache helped resolve the issue, at least not until the upstream ref has been deleted. We'd have to experiment to see if clearing the cache was actually necessary to resolve the broken state once the bad ref was removed server side.

error: cannot lock ref 'refs/heads/<branch>': is at <sha1> but expected <sha2>

[gyfelton]

So right now we clear cache only if we consider it as a retriable error, so if we converge the logic here to the one at the end (rescue block for. with_git_mirror method) at least for above error we face, we won't clear cache and will just fail hard? I think it's totally reasonable as we have it better than before (clear cache blindly)

[gyfelton]

@justinseanmartin Another look? In my manual test i have every call to fail_on_error/fail_pipe_on_error called with submodules and with/without -v set against git fastclone.

The output generated when no -v is still more than the original very silent operation. But
I also tested adding GIT_ALLOW_PROTOCOL=file at the beginning and indeed it failed with https protocol not supported error
The last thing I need to test is probably the two rescue blocks changed/added: plan is to manually raise error with/without the matching output and verify it indeed clear cache and retry

• Do above manual raising of error

[justinseanmartin]

Looks great! Few small things, but otherwise LGTM!