'struct pt_regs' has no member named 'ip'

[pfourguet]

Hi,
I'm having some issues building ktf.
I'm cross-compiling ktf for arm64 architecture.
Everything was fine until i updated my local ktf repository to test the function override feature.
Currently, building fails in the "Making all in kernel" phase:

make[1] : on entre dans le répertoire « /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel » make -C /home/pfourguet/sigfox/dev/linux-kernel M=/home/pfourguet/sigfox/dev/ktf/build/ktf/kernel make[2] : on entre dans le répertoire « /home/pfourguet/sigfox/dev/linux-kernel » arch/arm64/Makefile:48: Detected assembler with broken .inst; disassembly will be unreliable AR /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/built-in.o CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/kbase.o CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/nl.o CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_map.o CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/kcheck.o CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_debugfs.o CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_cov.o CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_override.o /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_override.c: In function 'ktf_override_function_with_return': /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_override.c:46:6: error: 'struct pt_regs' has no member named 'ip' regs->ip = (unsigned long)&ktf_just_return_func; ^
The pt_regs structure is highly architecture dependent and the ip field is not a member of pt_regs structure on arm64.
Thanks a lot.

[alan-maguire]

thanks for reporting this! as you can probably tell thus far we've been a bit x86_64-centric, but ARM support is something we'd definitely like to add. i'll try and resolve this asap; would you be able to verify a fix at your end? thanks! Alan

[pfourguet]

No problem to test a fix when you're ready. Pascal

[alan-maguire]

thanks Pascal! Knut pushed an update I made today which will (I hope) solve this specific issue; I tried adding arm[64] support in general but some things may well still be broken. If you could retry compilation, and if you're feeling particularly brave (and have a setup which can tolerate panics!) it would be great to try the selftests. "insmod kernel/ktf.ko ; insmod selftest/selftest.ko ; ktfrun". As I say don't do this if you're running in an environment in which panics are a problem! In the meantime I'm going to try and setup arm64 emulation so I can test things locally here.

[pfourguet]

Hi alan,
thanks for your update.
I've tried to compile ktf. I've unfortunately got the followin errors message:

Making all in kernel
make[1] : on entre dans le répertoire « /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel »
make -C /home/pfourguet/sigfox/dev/linux-kernel M=/home/pfourguet/sigfox/dev/ktf/build/ktf/kernel
make[2] : on entre dans le répertoire « /home/pfourguet/sigfox/dev/linux-kernel »
arch/arm64/Makefile:48: Detected assembler with broken .inst; disassembly will be unreliable
AR /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/built-in.o
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/kbase.o
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/nl.o
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_map.o
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/kcheck.o
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_debugfs.o
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_cov.o
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf_override.o
LD [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf.o
Building modules, stage 2.
MODPOST 1 modules
CC /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf.mod.o
LD [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/ktf.ko
make[2] : on quitte le répertoire « /home/pfourguet/sigfox/dev/linux-kernel »
make[1] : on quitte le répertoire « /home/pfourguet/sigfox/dev/ktf/build/ktf/kernel »
Making all in selftest
make[1] : on entre dans le répertoire « /home/pfourguet/sigfox/dev/ktf/build/ktf/selftest »
make -C /home/pfourguet/sigfox/dev/linux-kernel SUBDIRS=/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest KBUILD_EXTRA_SYMBOLS="/home/pfourguet/sigfox/dev/ktf/build/ktf/kernel/Module.symvers" modules
make[2] : on entre dans le répertoire « /home/pfourguet/sigfox/dev/linux-kernel »
arch/arm64/Makefile:48: Detected assembler with broken .inst; disassembly will be unreliable
CC [M] /home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.o
In file included from /home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:10:0:
./include/linux/slab_def.h: In function 'nearest_obj':
./include/linux/slab_def.h:94:30: error: dereferencing pointer to incomplete type 'struct page'
void *object = x - (x - page->s_mem) % cache->size;
^
In file included from /home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:12:0:
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c: In function 'myfunc_override':
/home/pfourguet/sigfox/dev/ktf/src/ktf/kernel/ktf.h:172:41: error: 'struct pt_regs' has no member named 'ARM_r0'
#define KTF_SET_RETURN_VALUE(value) regs->ARM_r0 = (value)
^
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:249:2: note: in expansion of macro 'KTF_SET_RETURN_VALUE'
KTF_SET_RETURN_VALUE(0);
^
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c: In function 'probesumhandler':
/home/pfourguet/sigfox/dev/ktf/src/ktf/kernel/ktf.h:172:41: error: 'struct pt_regs' has no member named 'ARM_r0'
#define KTF_SET_RETURN_VALUE(value) regs->ARM_r0 = (value)
^
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:281:2: note: in expansion of macro 'KTF_SET_RETURN_VALUE'
KTF_SET_RETURN_VALUE(-1);
^
scripts/Makefile.build:328: recipe for target '/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.o' failed
make[3]: *** [/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.o] Error 1
Makefile:1523: recipe for target 'module/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest' failed
make[2]: *** [module/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest] Error 2
make[2] : on quitte le répertoire « /home/pfourguet/sigfox/dev/linux-kernel »
Makefile:28: recipe for target 'module' failed
make[1]: *** [module] Error 2
make[1] : on quitte le répertoire « /home/pfourguet/sigfox/dev/ktf/build/ktf/selftest »
Makefile:414: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

For the first dereferenced pointer issue, i think adding the good header file to self.c may solve the problem.
For the ARM_r0 issue, it is not defined for arm64 architecture. Using the KTF_ENTRY_PROBE_ARG0 macro defined in ktf.h may solve the problem.
Pascal

[alan-maguire]

hi Pascal, latest ktf should resolve the compilation issues with registers - I've tested it in a KVM-emulated ARM64 environment and build works okay. WRT the struct page issues; do you have an equivalent to the "kernel-devel" package in Red Hat distros installed? We rely on some kernel headers when building KTF, but I don't think a full kernel headers package should be needed.

There are still some issues with behaviour of coverage, probe entry/return and override support on ARM64; I'm going to investigate them next. Thanks!

Alan

[pfourguet]

Hi,
with the modifications i previously suggested, i managed to compile and run selftests.
Unfortunately i got the following errors:

[==========] Running 15 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 15 tests from selftest
[ RUN ] selftest./cov
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:352: Failure
Assertion '0==ktf_syms.Z_ktf_cov_enable(((&__this_module))->name, 0x1)' failed: 0==0x0, ktf_syms.Z_ktf_cov_enable(((&__this_module))->name, 0x1)==0xffffffda
[ FAILED ] selftest./cov, where GetParam() = "cov" (39 ms)
[ RUN ] selftest./dummy
[ OK ] selftest./dummy (1 ms)
[ RUN ] selftest./map_keyoverflow
[ OK ] selftest./map_keyoverflow (1 ms)
[ RUN ] selftest./mapcmpfunc_map1
[ OK ] selftest./mapcmpfunc_map1 (8 ms)
[ RUN ] selftest./mapcmpfunc_map2
[ OK ] selftest./mapcmpfunc_map2 (1 ms)
[ RUN ] selftest./mapref_map1
[ OK ] selftest./mapref_map1 (1 ms)
[ RUN ] selftest./mapref_map2
[ OK ] selftest./mapref_map2 (1 ms)
[ RUN ] selftest./msg
[ OK ] selftest./msg (0 ms)
[ RUN ] selftest./override
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:260: Failure
Assertion 'register_kprobe(&__ktf_override_myfunc_override)==0' failed: register_kprobe(&__ktf_override_myfunc_override)==0xffffffda, 0==0x0
[ FAILED ] selftest./override, where GetParam() = "override" (0 ms)
[ RUN ] selftest./probeentry
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:234: Failure
Assertion 'register_kretprobe(&__ktf_entry_handler_printkhandler)==0' failed: register_kretprobe(&__ktf_entry_handler_printkhandler)==0xffffffda, 0==0x0
[ FAILED ] selftest./probeentry, where GetParam() = "probeentry" (1 ms)
[ RUN ] selftest./probereturn
/home/pfourguet/sigfox/dev/ktf/build/ktf/selftest/self.c:303: Failure
Assertion 'register_kretprobe(&__ktf_handler_printkrethandler)==0' failed: register_kretprobe(&__ktf_handler_printkrethandler)==0xffffffda, 0==0x0
[ FAILED ] selftest./probereturn, where GetParam() = "probereturn" (0 ms)
[ RUN ] selftest./simplemap_map1
[ OK ] selftest./simplemap_map1 (3 ms)
[ RUN ] selftest./simplemap_map2
[ OK ] selftest./simplemap_map2 (1 ms)
[ RUN ] selftest./symbol
[ OK ] selftest./symbol (58 ms)
[ RUN ] selftest./thread
[ OK ] selftest./thread (8 ms)
[----------] 15 tests from selftest (123 ms total)

[----------] Global test environment tear-down
[==========] 15 tests from 1 test case ran. (123 ms total)
[ PASSED ] 11 tests.
[ FAILED ] 4 tests, listed below:
[ FAILED ] selftest./cov, where GetParam() = "cov"
[ FAILED ] selftest./override, where GetParam() = "override"
[ FAILED ] selftest./probeentry, where GetParam() = "probeentry"
[ FAILED ] selftest./probereturn, where GetParam() = "probereturn"

4 FAILED TESTS

[alan-maguire]

hi Pascal, I just updated my ARM64 kernel to 4.18 and ran into the issue with "struct page". The latest KTF has a fix for that. With the updated KTF I see 2 selftest failures; the symbol and override tests fail. The reason for both failures is that the kernel I'm testing on was compiled with CONFIG_KALLSYMS_ALL not set, so we don't have kernel-internal symbols in kallsyms. Both the override and symbol tests rely on that. As it stands, I'm building a custom kernel with these facilities enabled and will update on progress. But I'd suggest trying 4.18 or later (ideally 4.19) at your end if possible and let me know how you get on. I ran into failures in the probeentry/return tests on 4.9 but AFAICT kprobe support on ARM64 was pretty new then so there could have been issues on the kernel side.