systemd[1]: bpf-lsm: Failed to load BPF object: No such process

[jm355]

When I use linux-hardened on arch linux, the following error gets logged: Mar 01 08:12:46 linux systemd[1]: bpf-lsm: Failed to load BPF object: No such process
Searching the logs for "bpf", I see:

Mar 01 08:12:27 archlinux kernel: Linux version 6.7.6-hardened1-1-hardened (linux-hardened@archlinux) (gcc (GCC) 13.2.1 20230801, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Sat, 24 Feb 2024 23:47:28 +0000
...
Mar 01 08:12:27 archlinux kernel: LSM: initializing lsm=capability,landlock,lockdown,yama,bpf,integrity
...
Mar 01 08:12:27 archlinux kernel: hid_bpf: error while preloading HID BPF dispatcher: -22
...
Mar 01 08:12:46 linux systemd[1]: bpf-lsm: Failed to load BPF object: No such process

Booting with the standard linux kernel, when I search the logs for "bpf" I find:

Feb 29 20:00:06 archlinux kernel: Linux version 6.7.6-arch1-2 (linux@archlinux) (gcc (GCC) 13.2.1 20230801, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Thu, 29 Feb 2024 19:08:51 +0000
...
Feb 29 20:00:06 archlinux kernel: LSM: initializing lsm=capability,landlock,lockdown,yama,bpf,integrity
...
Feb 29 20:00:45 linux systemd[1]: bpf-lsm: LSM BPF program attached

I found some discussion mentioning that linux-hardened had issues with bpf here systemd/systemd#27705. @RA-Kooi, you mentioned you'd look into why that is. Did you find anything?

[RA-Kooi]

I haven't taken the time to look into that yet. I'm also running into this on my custom kernel config, which is a zen-kernel with hardened patches applied. I'm having a feeling it's the case of an unset kernel config option.
Looking at a regular kernel setup on Arch I notice that CONFIG_BPF_LIRC_MODE2=y and CONFIG_BPF_KPROBE_OVERRIDE=y are set whereas they are unset on a hardened kernel.
Last time I grepped the hardened patch I didn't notice any structural changes that would break this. Unless it so happens that bpf_jit_harden = 2 triggers a broken code path that nobody knew is broken. But I think that that is rather unlikely, so my first step would be to set those config parameters and see if it changes anything.

Next time I'll build my kernel I'll make sure to set those BPF options and report back if needed.

[jm355]

Another potential clue: under https://wiki.archlinux.org/title/Security#Kernel_lockdown_mode it says:

integrity: kernel features that allow userland to modify the running kernel are disabled (kexec, bpf).

i don't think integrity mode is set by default though, when I saw that I was running linux-hardened and integrity mode was off

[RA-Kooi]

It is not, but it would explain why it's not working on my machine, since I use lockdown=confidentiality.

[ImagineRats]

While I cannot specifically recall what if any other changes elsewhere may result in similar breakage the main issue is most likely your config having CONFIG_BPF_LSM disabled; the intention behind why this would be in your config is probably not due to any opinion on that option specifically but because it depends on these additional settings:

CONFIG_FTRACE=y
(CONFIG_KPROBES=y && CONFIG_KPROBE_EVENTS=y) || (CONFIG_UPROBES=y && CONFIG_UPROBE_EVENTS=y)

As indicated by your logs this is something SystemD wants but does not explicitly need, that's because it's used to implement the RestrictFileSystems feature in service units. Last I understood SystemD's dependence on BPF_LSM this was limited to that relatively minor feature. So long as other necessary settings are used all other security relevant service features depending on BPF will function properly without BPF_LSM or the aforementioned settings on which it depends.

That said unlike some other situations these will be silent failures should they occur so I would suggest confirming their functionality with your current configuration via simple service units if this is of particular concern to you.

For reference this document can give you some idea of what kernel functionality SystemD depends on, though as is often the case with such things it's likely not all encompassing:

https://github.com/systemd/systemd/blob/main/README

[jm355]

Interesting, that's good to know. Even if RestrictFileSystems is a relatively minor feature, it seems like linux-hardened would want to enable it if it can improve hardening? Especially considering that the default kernel enables it, unless there's some other tradeoff I'm not aware of?

[ImagineRats]

Unfortunately it's complicated and I'm woefully unfit to offer any meaningful insight on the hardening significance of those changes which will vary depending on your particular threat model. Regardless these are things you'll see recommended against commonly in various places, particularly based on the references used by this project:

https://github.com/a13xp0p0v/kernel-hardening-checker

While this is unrelated to SystemD here are some discussions that center around some of the aforementioned dependencies of BPF_LSM:

lkrg-org/lkrg#27 (comment)
GrapheneOS/os-issue-tracker#408 (comment)

Aside from that though this isn't a bug/issue in the linux-hardened project itself as it's not responsible for the configuration you're using, if you believe the decision to not support BPF_LSM by the maintainer of your system's linux-hardened package should be reconsidered that's something you'd have to handle via your distribution's communication channels for such things.

[jm355]

Ah I see, I'll close this ticket and talk to the maintainer. Thanks for the insight!

[jm355]

Looks like the packager for arch linux is @anthraxx of this repo, so I'll ping them here. @anthraxx, what do you think about this?

[RA-Kooi]

So I recently updated my kernel and config and did a bit of testing.
I did not enable CONFIG_BPF_LIRC_MODE2 because it's for lirc devices, which I do not have.

I did enable the following:

CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_KPROBES=y
CONFIG_OPTPROBES=y
CONFIG_KPROBES_ON_FTRACE=y
CONFIG_UPROBES=y
CONFIG_KRETPROBES=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_NO_PATCHABLE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_DYNAMIC_FTRACE_WITH_ARGS=y
CONFIG_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE_MCOUNT_USE_CC=y
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT=y
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT_MULTI=y

And I also disabled kernel lockdown. And in this case systemd still gives me the error that there's no such process.
I did however use kernel hardening checker to check my config, where the majority of options are enabled, so that may have something to do with it. I think we'd do well to open a thread on the systemd repo to ask what the incompatibility could be.

[jm355]

What about CONFIG_BPF_LSM?

[RA-Kooi]

CONFIG_BPF_LSM has been enabled in my config from the start.

[ImagineRats]

So I recently updated my kernel and config and did a bit of testing. I did not enable CONFIG_BPF_LIRC_MODE2 because it's for lirc devices, which I do not have.

I did enable the following:

CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_KPROBES=y
CONFIG_OPTPROBES=y
CONFIG_KPROBES_ON_FTRACE=y
CONFIG_UPROBES=y
CONFIG_KRETPROBES=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS=y
CONFIG_HAVE_DYNAMIC_FTRACE_NO_PATCHABLE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
CONFIG_DYNAMIC_FTRACE_WITH_ARGS=y
CONFIG_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE_MCOUNT_USE_CC=y
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT=y
CONFIG_HAVE_SAMPLE_FTRACE_DIRECT_MULTI=y

And I also disabled kernel lockdown. And in this case systemd still gives me the error that there's no such process. I did however use kernel hardening checker to check my config, where the majority of options are enabled, so that may have something to do with it. I think we'd do well to open a thread on the systemd repo to ask what the incompatibility could be.

Have you first confirmed that this is not also an issue when using the same kernel parameters (sysctl+cmdline) on your distro kernel, i.e. not linux-hardened? Your problem is most likely due to enforcing a lockdown mode considering relevant documentation suggestions and that the same message appears on machines of my own which are not running linux-hardened, but do include additional parameter hardening.

Whether this results in bpf-lsm being unsupported is not something I've tested though doing so would require only a simple service unit to confirm. It's unlikely this issue is related to bpf-lsm being broken due to linux-hardened's patchset.