Check if PR is mergeable before queuing #199
Description
If a PR has no chance of being mergeable, it should not even be queued. Some examples:
- If a PR already has merge conflicts with the base branch, it can currently be queued and it'll fail when the batch starts. Note: this is only talking about the case when a PR already has merge conflicts at queue time — not merge conflicts with other queued PRs or with master after the running batch of PRs is merged.
- If the PR has required statuses that have failed. e.g. in the monorepo, we also require CI jobs to pass on the actual commit (not just the try commit). If these statuses are failed, we should not allow queueing
- If the PR is in Draft mode
The difficulty here is what happens if these two checks are in progress. GitHub checks merge conflicts asynchronously, and CI jobs take a while to finish. So if someone pushes up a new commit and immediately tries to queue the PR, what should happen? Should it fail "please wait until all checks have passed before queueing" or should it optimistically allow the queue? Should it check again when running the merge?
Currently, if a PR has merge conflicts, it would be caught when trying to make the merge commit, so that's not too bad of a problem. But let's say a commit fails CI checks on push, but the CI check passes in the merge job; the merge job would run completely, then fail at the very end (cannot update base branch). This is not ideal.
Current work done at chinn/check-pr-mergeable.
Related to #146