Retry when a different command has been issued

Situtuation:
1. @Hoff merge
2. CI fails on flaky test
3. @Hoff merge and deploy to production
4. Hoff refuses to merge because it already "failed" and won't restart CI.

In step 3 I can't issue a `retry` as due to changes in the merge queue I no longer want a merge but I also want a deploy. The only way to do this right now is to close the PR and open it again.