Skip to content

fix: clean registry buildcache tags on PR close #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 12 commits into
base: master
Choose a base branch
from
Open

fix: clean registry buildcache tags on PR close #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
dc894f7
Initial plan
Copilot Dec 14, 2025
5e0ac23
Add cleanup-buildcache workflow for PR close/merge
Copilot Dec 14, 2025
cb82752
Add eicweb registry cleanup and update documentation
Copilot Dec 14, 2025
77ecd1c
Address code review feedback: use jq --arg and clarify variable names
Copilot Dec 14, 2025
3ab7b6e
Use github-slug-action to ensure GITHUB_REF_POINT_SLUG is set
Copilot Dec 14, 2025
965704e
Clarify buildcache tag pattern documentation
Copilot Dec 14, 2025
26ae7ad
Fix: Use branch slug (GITHUB_REF_POINT_SLUG) and add workflow_dispatch
Copilot Dec 14, 2025
a299e11
Only run slug action for pull_request events
Copilot Dec 14, 2025
5992906
fix: allow for eicweb curl failure
wdconinc Dec 14, 2025
685722d
fix: docs
wdconinc Dec 14, 2025
52a832e
fix: handle pagination for eicweb
wdconinc Dec 14, 2025
d7f1468
fix: add manual dispatch to docs
wdconinc Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions .github/workflows/cleanup-buildcache.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
name: Cleanup Buildcache

on:
pull_request:
types: [closed]
branches:
- master
workflow_dispatch:
inputs:
ref_slug:
description: 'Branch slug (GITHUB_REF_POINT_SLUG) to clean up'
required: true
type: string

permissions:
packages: write

jobs:
cleanup-buildcache:
name: Cleanup buildcache for ${{ github.event.pull_request.number || inputs.ref_slug }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Inject enhanced GitHub environment variables
uses: rlespinasse/github-slug-action@v5
if: github.event_name == 'pull_request'
- name: Set ref slug
id: ref_slug
run: |
# For workflow_dispatch, use the input value
# For pull_request, use the GITHUB_REF_POINT_SLUG set by slug action
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
REF_SLUG="${{ inputs.ref_slug }}"
else
REF_SLUG="${GITHUB_REF_POINT_SLUG}"
fi
echo "ref_slug=${REF_SLUG}" >> $GITHUB_OUTPUT
echo "Using ref slug: ${REF_SLUG}"
- name: Cleanup buildcache tags from ghcr.io
env:
GH_TOKEN: ${{ secrets.GHCR_REGISTRY_TOKEN }}
REF_SLUG: ${{ steps.ref_slug.outputs.ref_slug }}
run: |
# Delete buildcache tags for this branch from ghcr.io
# The buildcache image is at ghcr.io/eic/buildcache
# Tags follow patterns:
# - Base images: {BUILD_IMAGE}-{GITHUB_REF_POINT_SLUG}-{arch}
# - EIC images: {BUILD_IMAGE}{ENV}-{BUILD_TYPE}-{GITHUB_REF_POINT_SLUG}-{arch}
# GITHUB_REF_POINT_SLUG is derived from the branch name (e.g., "my-feature" for branch "my-feature")
# The pattern contains("-{REF_SLUG}-") matches both formats

echo "Cleaning up buildcache tags for branch slug '${REF_SLUG}' from ghcr.io"

# List all versions of the buildcache package
echo "Fetching buildcache package versions..."
VERSIONS=$(gh api \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"/orgs/eic/packages/container/buildcache/versions?per_page=100" \
--paginate \
--jq --arg ref_slug "${REF_SLUG}" '.[] | select(.metadata.container.tags[]? | contains("-\($ref_slug)-")) | .id')

if [ -z "$VERSIONS" ]; then
echo "No buildcache versions found for branch slug '${REF_SLUG}' in ghcr.io"
else
# Delete each matching version
echo "Found buildcache versions to delete from ghcr.io:"
for version_id in $VERSIONS; do
# Get the tags for this version for logging
TAGS=$(gh api \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"/orgs/eic/packages/container/buildcache/versions/${version_id}" \
--jq '.metadata.container.tags[]? | select(. != null)' 2>/dev/null | tr '\n' ',' | sed 's/,$//')

echo " Deleting version ${version_id} with tags: ${TAGS}"

gh api \
--method DELETE \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"/orgs/eic/packages/container/buildcache/versions/${version_id}" \
|| echo " Warning: Failed to delete version ${version_id}"
done
fi

echo "Buildcache cleanup from ghcr.io completed for branch slug '${REF_SLUG}'"

- name: Cleanup buildcache tags from eicweb.phy.anl.gov
env:
GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}
REF_SLUG: ${{ steps.ref_slug.outputs.ref_slug }}
run: |
# Delete buildcache tags for this branch from eicweb.phy.anl.gov GitLab Container Registry
# The buildcache image is at eicweb.phy.anl.gov/containers/eic_container/buildcache
# Tags follow patterns:
# - Base images: {BUILD_IMAGE}-{GITHUB_REF_POINT_SLUG}-{arch}
# - EIC images: {BUILD_IMAGE}{ENV}-{BUILD_TYPE}-{GITHUB_REF_POINT_SLUG}-{arch}
# GITHUB_REF_POINT_SLUG is derived from the branch name (e.g., "my-feature" for branch "my-feature")
# The pattern contains("-{REF_SLUG}-") matches both formats

echo "Cleaning up buildcache tags for branch slug '${REF_SLUG}' from eicweb.phy.anl.gov"

GITLAB_API="https://eicweb.phy.anl.gov/api/v4"
GITLAB_PROJECT_ID="290" # containers/eic_container project ID

# List all repository tags for the buildcache image
echo "Fetching buildcache repository tags..."
REPOSITORY_ID=$(curl -s --fail --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
"${GITLAB_API}/projects/${GITLAB_PROJECT_ID}/registry/repositories" \
| jq -r '.[] | select(.name == "buildcache") | .id')
CURL_STATUS=${PIPESTATUS[0]}
if [ $CURL_STATUS -ne 0 ]; then
echo "Error: Failed to fetch registry repositories from GitLab API"
exit 1
fi
if [ -z "$REPOSITORY_ID" ]; then
echo "Warning: Could not find buildcache repository in GitLab registry"
exit 0
fi

echo "Found buildcache repository ID: ${REPOSITORY_ID}"

# Fetch all tags and filter for this branch, handling pagination
TAGS_TO_DELETE=""
PAGE=1
while :; do
RESPONSE=$(curl -s --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
"${GITLAB_API}/projects/${GITLAB_PROJECT_ID}/registry/repositories/${REPOSITORY_ID}/tags?per_page=100&page=${PAGE}")
TAGS=$(echo "$RESPONSE" | jq -r --arg ref_slug "${REF_SLUG}" '.[] | select(.name | contains("-\($ref_slug)-")) | .name')
if [ -z "$TAGS" ]; then
break
fi
TAGS_TO_DELETE="${TAGS_TO_DELETE}
${TAGS}"
# If fewer than 100 tags returned, this is the last page
COUNT=$(echo "$RESPONSE" | jq 'length')
if [ "$COUNT" -lt 100 ]; then
break
fi
PAGE=$((PAGE+1))
done
if [ -z "$TAGS_TO_DELETE" ]; then
echo "No buildcache tags found for branch slug '${REF_SLUG}' in eicweb.phy.anl.gov"
else
# Delete each matching tag
echo "Found buildcache tags to delete from eicweb.phy.anl.gov:"
for tag in $TAGS_TO_DELETE; do
echo " Deleting tag: ${tag}"

curl -s -X DELETE --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
"${GITLAB_API}/projects/${GITLAB_PROJECT_ID}/registry/repositories/${REPOSITORY_ID}/tags/${tag}" \
|| echo " Warning: Failed to delete tag ${tag}"
done
fi

echo "Buildcache cleanup from eicweb.phy.anl.gov completed for branch slug '${REF_SLUG}'"
11 changes: 11 additions & 0 deletions docs/build-pipeline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,8 @@ cache-from: |
cache-to: type=registry,ref=ghcr.io/eic/buildcache:{image}-{branch}-{arch},mode=max
```

**Buildcache Cleanup**: When a pull request is closed or merged, the `cleanup-buildcache` workflow automatically removes all buildcache tags associated with that PR from both ghcr.io and eicweb.phy.anl.gov registries. This prevents buildcache accumulation and keeps the registries clean.

### Build Mount Cache

Uses [buildkit-cache-dance](https://github.com/reproducible-containers/buildkit-cache-dance) to persist mount caches:
Expand All @@ -150,13 +152,22 @@ Pre-built binaries are stored in OCI registries:

## Workflow Triggers

### build-push workflow

| Trigger | Behavior |
|---------|----------|
| Schedule (cron) | Every 6 hours - nightly builds |
| Push to master | Build and push with `pipeline-*` tag |
| Pull Request | Build with `unstable-pr-*` tag |
| Manual Dispatch | Allows overriding EDM4EIC, EICRECON, JUGGLER versions |

### cleanup-buildcache workflow

| Trigger | Behavior |
|---------|----------|
| Pull Request closed | Automatically removes all buildcache tags associated with the branch slug (typically unique to the PR) from ghcr.io and eicweb.phy.anl.gov |
| Manual Dispatch | Allows manual cleanup of buildcache tags for a specific branch or PR by specifying a custom `ref_slug` parameter |

## Environment Matrix

The EIC job builds the following matrix:
Expand Down
Loading