In-kernel, per-process sandboxing mechanism

Requirements

• clang-21 and the accompanying LLVM toolchain (v21)
• asm/ linux header files (if they are installed but not detected by clang, create a symbolic link from /usr/include/x86_64-linux-gnu/asm to /usr/include/asm)
• sudo apt install pkg-config libbpf-dev
• bpftool (compiled from kernel source since the correct version is not available via Ubuntu repositories)

Compiling the project

mkdir build && cd build
cmake -DLLVM_DIR=$(llvm-config-21 --cmakedir) ..
make

Produced Artifacts

• libLLVMPass.so The LLVM 21 Pass Plugin (build/llvm-pass/libLLMVPass.so)
• KernelSpaceMonitor.bpf.o Compiled Kernel Space BPF Object (build/ebpf-monitor/KernelSpaceMonitor.bpf.o)
• KernelSpaceMonitor.skel.h BPF Object Skeleton extracted by bpftool (build/ebpf-monitor/KernelSpaceMonitor.skel.h)
• eBPFLoader User Space BPF Object Skeleton loader (build/ebpf-monitor/eBPFLoader)

Running the instrumentation pass at LTO with lld [1,2]

Convenience environment variable to set the required compiler flags

LCS_FLAGS="-flto=full -fuse-ld=lld -Wl,--load-pass-plugin=path/to/libLLVMPass.so -Wl,--lto-newpm-passes=null-call-instrumentation-pass"

clang-21 $LCS_FLAG [targets]

To examine the instrumentation in the LLVM IR

# Save temporary files generated at different stages of LTO
clang-21 $LCS_FLAGS -Wl,--plugin-opt=save-temps [targets]
llvm-dis a.out.0.5.precodegen.bc

Utilizing the eBPF monitor

Generating a policy

1. Before compiling a target with instrumentation, create a directory ./output where the compiler is being invoked
2. Run the compiler with flags as given above, the instrumented executable binary as well as DOT files for indivudal functions, the entire program and a serialized policy file will be created in the ./output folder.

The eBPF monitor matches a program being executed to a policy file by the name of the executable. Hence, copy the generated policy file to a standard policy directory (i.e., /home/vboxuser/.config/LibCallSandboxing/policies) and give it the same name as of the executable file.

Running the eBPF monitor

sudo ./eBPFmonitor /home/vboxuser/.config/LibCallSandboxing/policies

• To view logs produced by the monitor, run

sudo bpftool prog trace

Now run any instrumented executable with name matching that of a policy

Caveats

• The instrumentation and monitor cannot handle indirect function calls. Such a function call will most likely result in the process being killed.
• Due to restrictions on BPF code, the maximum number of frontier states and the maximum number of edges from a state is limited to a small number (16 for now in include/ebpf-monitor/Common.h)
• The syscall number has been arbitrarily chosen (467 for now) based on availability in the v6.14 kernel source
• It is hardcoded into the LLVM pass for inserting instrumentation

Compiling mbedtls Benchmark

The benchmark file is available in the mbedtls-3.6.5 branch

1. Run the default compilation process to generate all library object files using make
2. Go to the programs/test directory where the benchmark.c file is located
3. Create the directory for policy output ./output
4. Provide the benchmark.c source file for instrumentation and compiled library objects directly

LCS_FLAGS="-flto=full -fuse-ld=lld -Wl,--load-pass-plugin=path/to/libLLVMPass.so -Wl,--lto-newpm-passes=null-call-instrumentation-pass"
clang-21 $(LCS_FLAGS) benchmark.c ../../library/*.o -o benchmark_instrumented

5. Copy the generated policy file to the policy folder

cp ./output/policy /home/vboxuser/.config/LibCallSandboxing/policies/benchmark_instrumented

6. Run the instrumented executable

./benchmark_instrumented

NOTE: The benchmark fails right now because it utilizes indirect function calls

Compiling the patched Linux v6.14 Kernel on Ubuntu 24.04.03 LTS VirtualBox

Adding the dummy system call [3]

• Syscall defined in LibCallSandboxing/nullcall.c using the SYSCALL_DEFINE1 macro

#include <linux/kernel.h>
#include <linux/syscalls.h>

SYSCALL_DEFINE0(hello) {
pr_info("Hello World\n");
return 0;
}

• Create /LibCallSandboxing/Makefile with obj-y := nullcall.o
• Add asmlinkage long sys_nullcall(long label); in /include/linux/syscalls.h
• Add entry to syscall table arch/x86/entry/syscalls/syscall_64.tbl

467   common      nullcall    sys_nullcall

Requirements

sudo apt install build-essential vim git cscope libncurses-dev libssl-dev bison flex \
      libelf-dev elfutils libdw-dev pahole

• Compile with CONFIG_DEBUG_INFO_BTF=y so that kernel provides /sys/kernel/btf/vmlinux which can be used for BTF CO-RE

References

1. https://github.com/Jakob-Koschel/llvm-passes
2. https://gist.github.com/tommymcm/1c65f9ed5f169bdc8ee1f4d1268558bb
3. https://medium.com/@aryan20/create-custom-system-call-on-linux-6-8-126edef6caaf