ci: avoid unnecessary builds and test runs

[dscho]

Sadly, VFSforGit's CI and PR builds are not exactly cheap. Not only in terms of combined runtime (where VFSforGit's 3 hours 45 minutes is not even close runner-up to git/git's 8.5 hours (!!!). A Functional Test job typically runs for 35 minutes to test on x86_64, and an even worse 1 hour and 15 minutes on arm64. The wall-clock time for a CI or PR build of VFSforGit therefore weighs around 1 hour and 20 minutes.

This hurts a lot when trying to fast-track a change from a PR targeting master to a full VFSforGit release because not only does the entire build run again on the (tree-same) master after merging, the next step (thanks to the original VFSforGit maintainer's decision to go for a Git Flow-like setup) requires the build to run a third-time on the (also tree-same) PR merge commit.

Let's try to reuse build and test results from previous, successful runs on what is essentially the same code.

[mjcheetham]

No doubt this works, but I'm very concerned about the fragility of it all, and how verbose and complicated some of the repeated expressions and inline GitHub scripts are.

Can we not do this in any other, simpler, way? Even if not optimal, but just some modest improvement. Would rather have the have the workflow/build change than workaround Actions limitations.

[mjcheetham]

- - name: look for prior successful runs
+ - name: Look for prior successful runs

[dscho]

Range-diff

• -: -------- > 1: 281c6b9 GVFS.Installers: update Inno Setup compiler

• 1: 906fc4f = 2: 2568836 ci(build): use Unix line endings

• 2: e34c0fb = 3: c5f1ca3 ci: explicitly restrict GITHUB_TOKEN permissions

• 3: 164e0b5 = 4: 5ca1b99 ci: avoid failing fast in the matrix jobs

• 4: 7617fd9 ! 5: a08ef4c ci: skip running if the same commit has already seen a successful run

  @@ .github/workflows/build.yaml: jobs:
   +      skip: ${{ steps.check.outputs.result }}
   +
        steps:
  -+    - name: look for prior successful runs
  ++    - name: Look for prior successful runs
   +      id: check
   +      uses: actions/github-script@v7
   +      with:
  @@ .github/workflows/build.yaml: jobs:
   +              repo: context.repo.repo,
   +              per_page: 500,
   +              workflow_id,
  -+              status: 'success',,
  ++              status: 'success',
   +            });
   +            for (const run of runs.workflow_runs) {
   +              if (head_sha === run.head_sha) return true

• 5: bf4b95f = 6: 2edc6c1 ci: skip also for successful runs for tree-same commits

• 6: eb9177b = 7: 4ee8c1c ci: when skipping runs, do not skip the required jobs

• 7: 4d4639b ! 8: 5099500 ci: when skipping runs, report the previous successful one

  @@ Commit message
   
    ## .github/workflows/build.yaml ##
   @@ .github/workflows/build.yaml: jobs:
  -               status: 'success',,
  +       uses: actions/github-script@v7
  +       with:
  +         github-token: ${{secrets.GITHUB_TOKEN}}
  ++        result-encoding: string
  +         script: |
  +           /*
  +             * It would be nice if GitHub Actions offered a convenient way to avoid running
  +@@ .github/workflows/build.yaml: jobs:
  +               status: 'success',
                });
                for (const run of runs.workflow_runs) {
   -              if (head_sha === run.head_sha) return true

• 8: effc5cb < -: -------- ci: do re-use successful jobs even from failing runs

• 9: 0f0f754 ! 9: a9fc9a5 ci: when looking for successful workflow jobs, wait for in-progress ones

  @@ Metadata
   Author: Johannes Schindelin <johannes.schindelin@gmx.de>
   
    ## Commit message ##
  -    ci: when looking for successful workflow jobs, wait for in-progress ones
  +    ci: when looking for successful workflow runs, wait for in-progress ones
   
  -    The runtime of the Functional Test jobs is long enough (35 minutes for
  -    x86_64, 1 hour 15 minutes for ARM64) that it is worth waiting for them
  -    to finish before kicking off a matrix build that tests the exact same
  -    code.
  +    The Functional Test jobs takes long enough (35 minutes for x86_64, 1
  +    hour 15 minutes for ARM64) that it is worth waiting for them to finish
  +    before kicking off a matrix build that tests the exact same code.
   
       So let's do that, wait for any in-progress run that might result in
       some saved time. Granted, this spends cycles of GitHub runners waiting
  @@ Commit message
   
    ## .github/workflows/build.yaml ##
   @@ .github/workflows/build.yaml: jobs:
  +               repo: context.repo.repo,
  +               per_page: 500,
                  workflow_id,
  +-              status: 'success',
                });
                for (const run of runs.workflow_runs) {
  +-              if (head_sha === run.head_sha) return run.html_url
  +-              else if (tree_id === run.head_commit?.tree_id) return run.html_url
  ++              if (head_sha !== run.head_sha && tree_id !== run.head_commit?.tree_id) continue
   +              if (context.runId === run.id) continue // do not wait for the current run to finish ;-)
   +
  -               let reason
  -               if (head_sha === run.head_sha) reason = 'the same commit'
  -               else if (tree_id === run.head_commit?.tree_id) reason = 'a tree-same commit'
  -               else continue
  - 
   +              if (run.status === 'in_progress') {
   +                // poll until the run is done
   +                const pollIntervalInSeconds = 30
  @@ .github/workflows/build.yaml: jobs:
   +                }
   +              }
   +
  -               const { data: jobs } = await github.rest.actions.listJobsForWorkflowRun({
  -                 owner: context.repo.owner,
  -                 repo: context.repo.repo,
  ++              if (run.status === 'success') return run.html_url
  +             }
  +             return ''
  +           } catch (e) {

• 10: 9b6e6a7 ! 10: 6575572 ci: when looking for successful workflow jobs, use a smart order

  @@ .github/workflows/build.yaml: jobs:
   +            const rank = (a) => (a.status === 'in_progress' ? 0 : (head_sha === a.head_sha ? 2 : 1))
   +            const demoteInProgressToEnd = (a, b) => (rank(b) - rank(a))
   +            for (const run of runs.workflow_runs.sort(demoteInProgressToEnd)) {
  +               if (head_sha !== run.head_sha && tree_id !== run.head_commit?.tree_id) continue
                  if (context.runId === run.id) continue // do not wait for the current run to finish ;-)
    
  -               let reason

[mjcheetham]

Great! This should really help improve run times of the build+test workflow.