composefs backend: bootc-finalize-staged fails when incoming upgrade deletes a folder from /etc

[chaserhkj]

I have a custom image upgrade deleting a folder from /etc and bootc-finalize-staged.service fails when upgrading my composefs deployment:

-- Boot 708867703f1c4606a53bfee2ba08fc57 --
Jan 16 01:54:35 hatsune systemd[1]: Finished Composefs Finalize Staged Deployment.
Jan 16 01:55:24 hatsune systemd[1]: Stopping Composefs Finalize Staged Deployment...
Jan 16 01:55:24 hatsune bootc[6800]: error: Merging: Merging added files: Creating symlink "systemd/system/multi-user.target.wants/machines.target": No such file or directory (os error 2)
Jan 16 01:55:24 hatsune systemd[1]: bootc-finalize-staged.service: Control process exited, code=exited, status=1/FAILURE
Jan 16 01:55:24 hatsune systemd[1]: bootc-finalize-staged.service: Failed with result 'exit-code'.
Jan 16 01:55:24 hatsune systemd[1]: Stopped Composefs Finalize Staged Deployment.

/etc/systemd/system/ of my current deployed state:

$ ls -l /etc/systemd/system
total 32
lrwxrwxrwx 1 root root  37 Jan 14 20:40 ctrl-alt-del.target -> /usr/lib/systemd/system/reboot.target
lrwxrwxrwx 1 root root  45 Jan 14 20:40 dbus-org.freedesktop.home1.service -> /usr/lib/systemd/system/systemd-homed.service
lrwxrwxrwx 1 root root  48 Jan 14 20:40 dbus-org.freedesktop.network1.service -> /usr/lib/systemd/system/systemd-networkd.service
lrwxrwxrwx 1 root root  48 Jan 14 20:40 dbus-org.freedesktop.resolve1.service -> /usr/lib/systemd/system/systemd-resolved.service
lrwxrwxrwx 1 root root  49 Jan 14 20:40 dbus-org.freedesktop.timesync1.service -> /usr/lib/systemd/system/systemd-timesyncd.service
drwxr-xr-x 1 root root  52 Jan 14 20:40 factory-reset.target.wants
drwxr-xr-x 1 root root  36 Jan 14 20:40 getty.target.wants
drwxr-xr-x 1 root root 306 Jan 16 01:06 multi-user.target.wants
drwxr-xr-x 1 root root  72 Jan 14 20:40 network-online.target.wants
-rw-r--r-- 1 root root 325 Jan  7 22:39 rfkill-on-boot.service
drwxr-xr-x 1 root root 504 Jan 16 01:06 sockets.target.wants
drwxr-xr-x 1 root root 420 Jan 14 20:40 sysinit.target.wants
drwxr-xr-x 1 root root  60 Jan 14 20:40 systemd-homed.service.wants
drwxr-xr-x 1 root root  58 Jan 14 20:40 systemd-journald.service.wants
lrwxrwxrwx 1 root root   9 Jan  7 20:52 systemd-rfkill.service -> /dev/null
lrwxrwxrwx 1 root root   9 Jan  7 20:52 systemd-rfkill.socket -> /dev/null

/etc/systemd/system/ of my previous image:

$ ls /mnt/etc/systemd/system -l
total 16
drwxr-xr-x 2 root root  61 Jan  7 20:51 multi-user.target.wants
-rw-r--r-- 1 root root 325 Jan  7 22:39 rfkill-on-boot.service
lrwxrwxrwx 1 root root   9 Jan  7 20:52 systemd-rfkill.service -> /dev/null
lrwxrwxrwx 1 root root   9 Jan  7 20:52 systemd-rfkill.socket -> /dev/null

/etc/systemd/system/ of my upcoming image:

$ ls /tmp/mnt/etc/systemd/system -l
total 12
-rw-r--r-- 1 root root 362 Jan 16 00:10 rfkill-on-boot.service
lrwxrwxrwx 1 root root   9 Jan  7 20:52 systemd-rfkill.service -> /dev/null
lrwxrwxrwx 1 root root   9 Jan  7 20:52 systemd-rfkill.socket -> /dev/null

So folder /etc/systemd/system/multi-user.target.wants exists in current deployment and previous image, is deleted from upgraded image, and this situation triggers the bug. When I added back the folder as empty and do another upgrade, the bug is gone.

[Johan-Liebert1]

I can reproduce this bug. Seems like an untested edge case. Happens if directory contents are modified but that directory is not present in the new etc.