-
Notifications
You must be signed in to change notification settings - Fork 110
Description
We are currently facing a mysterious build issue since last Friday, where the existing code suddenly stopped building.
The error:
- occurred in other unrelated parts,
- the symptom has been changing when attempted fixes were committed,
- and can't be reproduced locally.
Which hinders the investigation. But now we think we are onto something.
We are building full operating system image, and have previously configured the action as follows:
- name: Maximize build space
uses: easimon/maximize-build-space@v10
with:
root-reserve-mb: 24576
swap-size-mb: 2048
remove-dotnet: true
remove-android: true
remove-haskell: true
remove-codeql: true
remove-docker-images: trueThe large root-reeserve-mb is used by devcontainer, which calls docker and build the final image. As of now, the image itself is more than 8GB, plus the rootfs has to be built first before deploying into the image file, thus the high root reserve size, since docker consumes root space.
On earlier working job, we can see the rootfs being resized correctly:
Available storage:
Filesystem Size Used Avail Use% Mounted on
/dev/root 73G 49G 24G 67% /
tmpfs 7.9G 172K 7.9G 1% /dev/shm
tmpfs 3.2G 1.0M 3.2G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
efivarfs 128M 9.7K 128M 1% /sys/firmware/efi/efivars
/dev/sda15 105M 6.1M 99M 6% /boot/efi
/dev/sdb1 74G 70G 100M 100% /mnt
tmpfs 1.6G 12K 1.6G 1% /run/user/1001
/dev/mapper/buildvg-buildlv 79G 24K 79G 1% /home/runner/work/rsdk/rsdk
However, with failing job (it has been reran a few times), we can see the root size is very small now:
Available storage:
Filesystem Size Used Avail Use% Mounted on
/dev/root 146G 146G 100M 100% /
tmpfs 7.9G 172K 7.9G 1% /dev/shm
tmpfs 3.2G 1020K 3.2G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
efivarfs 128M 9.6K 128M 1% /sys/firmware/efi/efivars
/dev/sda15 105M 6.1M 99M 6% /boot/efi
tmpfs 1.6G 12K 1.6G 1% /run/user/1001
/dev/mapper/buildvg-buildlv 103G 24K 103G 1% /home/runner/work/rsdk/rsdk
Looking closer, we can see /dev/root size has been changed, and /dev/sdb1 is now gone. This is likely why the existing script stopped working.
Additionally, the new runner image now has a default free root space of 92G:
Available storage:
Filesystem Size Used Avail Use% Mounted on
/dev/root 146G 55G 92G 38% /
tmpfs 7.9G 172K 7.9G 1% /dev/shm
tmpfs 3.2G 1012K 3.2G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
efivarfs 128M 9.6K 128M 1% /sys/firmware/efi/efivars
/dev/sda15 105M 6.1M 99M 6% /boot/efi
tmpfs 1.6G 12K 1.6G 1% /run/user/1001
This should be enough for a lot of projects now. So workaround for now is to simply delete this action call.
The new root size has been observed on both ubuntu-22.04 and ubuntu-latest.