Allow to setup the environment blob for grub

[schaefi]

Added new section to the existing section which allows to specify environment variables for setting up an environment blob for the selected loader. With this commit we add support for grub by using grub2-editenv. Other loaders do not yet have an implementation or does not support environment blobs. Settings will be ignored for unsupported loaders.

This Fixes #2922

[Conan-Kudo]

Do we have a staging image build to observe this with?

[schaefi]

Do we have a staging image build to observe this with?

I will do this today, stay tuned

[schaefi]

I have to refactor several parts of the code in order to implement this better.

[schaefi]

@Conan-Kudo I updated the Staging build with the current code of this PR and also pointed the integration test to it. See the integration test build here:

• https://build.opensuse.org/projects/Virtualization:Appliances:Images:Testing_x86:tumbleweed/packages/test-image-disk/files/_service:obs_scm:appliance.kiwi?expand=1

As you can see I used menu_auto_hide=1 and I can also see the build to apply the respective grub2-editenv call. In the later image I checked via

grub2-editenv list

and it prints the setting as expected.

However I'm unsure about the real behavior of the bootloader with this setting applied. The integration test also sets up the console to serial. Therefore if I run the integration test in qemu I can see the grub loader and the menu shown on the serial console.

Should that menu_auto_hide=1 thing have any effect here ?

I would like to build something that actually shows an effect.

Thanks

[Conan-Kudo]

setting 0 will make the grub menu show up, setting 1 will make it skip and go straight to boot.

[schaefi]

setting 0 will make the grub menu show up, setting 1 will make it skip and go straight to boot.

Hmm, doesn't behave like that for the built TW integration test. Can you do me a favor and do the following

wget https://download.opensuse.org/repositories/Virtualization:/Appliances:/Images:/Testing_x86:/tumbleweed/images/kiwi-test-image-disk.x86_64-Standard.raw.xz
xz -d kiwi-test-image-disk.x86_64-Standard.raw.xz

qemu-kvm -cpu Broadwell-v2 -m 4096 -netdev user,id=user0 -device virtio-net-pci,netdev=user0 -serial stdio -bios /usr/share/qemu/ovmf-x86_64.bin -hda kiwi-test-image-disk.x86_64-Standard.raw

You will see the menu shown on the serial console. Boot into the system and call

Have a lot of fun...
localhost:~ # grub2-editenv list
menu_auto_hide=1
localhost:~ # 

From my perspective I have done it correctly but it has no impact as you described.

Thanks

[Conan-Kudo]

This feature doesn't exist in Tumbleweed yet.

[Conan-Kudo]

The test needs to be on Rawhide.

[schaefi]

The test needs to be on Rawhide.

ok that is an important information :) ... more fun tomorrow

[schaefi]

• runtime check bootloader_env_compatible_with_loader

[schaefi]

@Conan-Kudo this is now working. I have enabled integration tests for TW and rawhide. As you know rawhide doesn't build due to the libboost thing (not related to this PR). I want this to be merged not before the rawhide test can build again. As such setting to blocked

[schaefi]

On TW the test setup with menu_auto_hide=1 also has an impact now. When booting the grub menu is not shown. I tried with some key-combinations to show it but that did not work (or I didn't know the right key combo). After the timeout it then boots. As a user you don't see anything from the grub menu and you can't interact. Maybe that's the intended behavior. How it behaves on rawhide we will find out as soon as the build tests starts to work again

Thanks

[schaefi]

@rdoxenham Thanks :)

[schaefi]

@Conan-Kudo rawhide tests are building now again. Thanks for the quick turnaround. See the integration test here

• https://build.opensuse.org/package/show/Virtualization:Appliances:Images:Testing_x86:rawhide/test-image-live-disk

I enabled menu_auto_hide=1 in the virtual profile

When I boot the integration test I still see the grub menu on the serial console. After boot I checked the env settings

[root@fedora ~]# grub2-editenv list
saved_entry=261085b0944a43d080397380f90993d9-6.18.0-65.fc44.x86_64
menu_auto_hide=1
boot_success=0

[root@fedora ~]# cat /etc/os-release 
NAME="Fedora Linux"
VERSION="44 (Workstation Edition Prerelease)"
RELEASE_TYPE=development
ID=fedora
VERSION_ID=44
VERSION_CODENAME=""
PRETTY_NAME="Fedora Linux 44 (Workstation Edition Prerelease)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:44"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/rawhide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=rawhide
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=rawhide
SUPPORT_END=2027-05-19
VARIANT="Workstation Edition"
VARIANT_ID=workstation

Please check the result as you see fit.

[Conan-Kudo]

I don't see any logging of the step for setting it and execution of the command or any output from it.

[schaefi]

I don't see any logging of the step for setting it and execution of the command or any output from it.

osc rbl Virtualization:Appliances:Images:Testing_x86:rawhide test-image-live-disk images x86_64 -M Virtual

[  121s] [ INFO    ]: 12:30:41 | Set grub environment variables: ['menu_auto_hide=1']
[  121s] [ DEBUG   ]: 12:30:41 | EXEC: [chroot /var/tmp/kiwi_mount_manager.apio_za8 grub2-editenv - set menu_auto_hide=1]

This integration test builds multiple profiles. I have set the grub env for the Virtual profile only. As you can also see in PR code

[Conan-Kudo]

Oh we need a test case with /boot on Btrfs like we had for the one about the partition numbers.

[schaefi]

Oh we need a test case with /boot on Btrfs like we had for the one about the partition numbers.

Well we can certainly add that but it should not make any difference. Point is this integration sets the value, it's there in the grubenv and that should be enough to test this feature. I also showed the log entry which allows to double check that things are happening. As such I'm seeking for some feedback and review if it does what it is expected to do. I'm a bit tired of getting more and more requests when I'm actually wanted to get some feedback on the current state.

If it does not what it is expected to do on a simple layout image it will not help to add it to a complex layout image. As such can someone first share feedback if the current integration test:
a) does the right thing
b) behaves as expected

If there is a yes to both of these questions I'm willing to extend it to more integration tests

Thanks

[Conan-Kudo]

The reason is that we need to check that the command actually works and returns the right result in that case. Because if it doesn't, then we're doing some other setup wrong.

In principle, the code looks fine, but we need to deliberately check for different behavior when Btrfs is used.

[schaefi]

The reason is that we need to check that the command actually works and returns the right result in that case. Because if it doesn't, then we're doing some other setup wrong.

The layout of the disk is really immaterial. It has to work in any case.

when I'm actually wanted to get some feedback on the current state.

I answered it myself. There are more settings that must play well together to make menu_auto_hide to work. These are the correct timeout_style and we have to indicate boot_success

I updated the integration test and now the image behaves as expected. No menu shown

Now we can apply the same settings to a btrfs integration test

[schaefi]

In principle, the code looks fine, but we need to deliberately check for different behavior when Btrfs is used.

@Conan-Kudo done and tested, works for me. Please check

• https://build.opensuse.org/package/show/Virtualization:Appliances:Images:Testing_x86:rawhide/test-image-live-disk

The Virtual_btrfs profile build

[schaefi]

@Conan-Kudo this one is imho also ready for merge

[Conan-Kudo]

I'm doing a local verification and will merge after that.

[Conan-Kudo]

For some reason, I see it written to /boot/grub2/grubenv but not in the grub btrfs environment block:

You can test yourself with: dd if=/dev/vda1 bs=512 count=1 skip=512 | hexdump -C

[schaefi]

As I wrote in the matrix chat I believe this is not the way how you should check for env functionality. Your hexdump shows the environment blob of the grub boot code in the first 512 byte (MBR).

That is not what grub loads via load_env -f ${config_directory}/grubenv as written by the fedora grub-mkconfig tool.

The grubenv blob from the correct boot volume is loaded by this action and also effective in all my tests for the image. Meaning the behavior with regards to the menu_auto_hide=1 is working as I would expect it. So unless your image boot process shows a menu I don't see a mistake.

The env blob in the MBR is imho meaningless in this layout. In general the env blob in the MBR (if there is any) is the result of grub2-install this call takes the distribution provided grub image into account which contains this version of the grub env blob. As kiwi does not create its own grub image (grub2-mkimage) unless the distribution does not provide one, the grub env blob in its original form is defined by the distro.

[Conan-Kudo]

I'm not sure I understand what's going on here. The menu_auto_hide=1 thing makes sense and that's fine, but boot_success=0 in the envblock should make the menu show up again, but it doesn't.

@WenhuaChang, @lsandov1: what is going wrong here?