How to run cm.jalt instruction of Zcmt Extension in Embench-iot programs by Spike?

[gakubochama]

I tried to run cm.jalt instruction of Zcmt Extension in Embench-iot programs by Spike, but I can't.

I asked same content in riscv-isa-sim repository, but I didn't make sense...

command↓
spike --isa=rv32ima_zicsr_zca_zcb_zcmp_zcmt -d cubic
(cubic is elf file)

the console output↓

core   0: 0x80000032 (0x00004c81) c.li    s9, 0
core   0: 0x80000034 (0x00004d01) c.li    s10, 0
core   0: 0x80000036 (0x00004d81) c.li    s11, 0
core   0: 0x80000038 (0x00004e01) c.li    t3, 0
core   0: 0x8000003a (0x00004e81) c.li    t4, 0
core   0: 0x8000003c (0x00004f01) c.li    t5, 0
core   0: 0x8000003e (0x00004f81) c.li    t6, 0
core   0: 0x80000040 (0x80014137) lui     sp, 0x80014
core   0: 0x80000044 (0x01701073) csrw    jvt, zero
core   0: 0x80000048 (0x0000a1a6) cm.jalt 105
core   0: exception trap_instruction_access_fault, epc 0x80000048
core   0:           tval 0x000001a4
(spike) 
core   0: exception trap_instruction_access_fault, epc 0x00000000
core   0:           tval 0x00000000

Tell me how to solve this problem.

I could run Zca,Zcb,Zcmp instructions by Spike without any problems.

[jnk0le]

core 0: exception trap_instruction_access_fault, epc 0x80000048
core 0: tval 0x000001a4

jvt CSR needs to be configured with valid address of pointer table.
Here it seems to try to read from area that doesn't have memory mapped into it.

[gakubochama]

I changed my code a liitle

(spike) r 40
core   0: 0x00001000 (0x00000297) auipc   t0, 0x0
core   0: 0x00001004 (0x02028593) addi    a1, t0, 32
core   0: 0x00001008 (0xf1402573) csrr    a0, mhartid
core   0: 0x0000100c (0x0182a283) lw      t0, 24(t0)
core   0: 0x00001010 (0x00028067) jr      t0
core   0: 0x80000000 (0x00000001) c.nop
core   0: 0x80000002 (0x00004081) c.li    ra, 0
core   0: 0x80000004 (0x00004101) c.li    sp, 0
core   0: 0x80000006 (0x00004181) c.li    gp, 0
core   0: 0x80000008 (0x00004201) c.li    tp, 0
core   0: 0x8000000a (0x00004281) c.li    t0, 0
core   0: 0x8000000c (0x00004301) c.li    t1, 0
core   0: 0x8000000e (0x00004381) c.li    t2, 0
core   0: 0x80000010 (0x00004401) c.li    s0, 0
core   0: 0x80000012 (0x00004481) c.li    s1, 0
core   0: 0x80000014 (0x00004501) c.li    a0, 0
core   0: 0x80000016 (0x00004581) c.li    a1, 0
core   0: 0x80000018 (0x00004601) c.li    a2, 0
core   0: 0x8000001a (0x00004681) c.li    a3, 0
core   0: 0x8000001c (0x00004701) c.li    a4, 0
core   0: 0x8000001e (0x00004781) c.li    a5, 0
core   0: 0x80000020 (0x00004801) c.li    a6, 0
core   0: 0x80000022 (0x00004881) c.li    a7, 0
core   0: 0x80000024 (0x00004901) c.li    s2, 0
core   0: 0x80000026 (0x00004981) c.li    s3, 0
core   0: 0x80000028 (0x00004a01) c.li    s4, 0
core   0: 0x8000002a (0x00004a81) c.li    s5, 0
core   0: 0x8000002c (0x00004b01) c.li    s6, 0
core   0: 0x8000002e (0x00004b81) c.li    s7, 0
core   0: 0x80000030 (0x00004c01) c.li    s8, 0
core   0: 0x80000032 (0x00004c81) c.li    s9, 0
core   0: 0x80000034 (0x00004d01) c.li    s10, 0
core   0: 0x80000036 (0x00004d81) c.li    s11, 0
core   0: 0x80000038 (0x00004e01) c.li    t3, 0
core   0: 0x8000003a (0x00004e81) c.li    t4, 0
core   0: 0x8000003c (0x00004f01) c.li    t5, 0
core   0: 0x8000003e (0x00004f81) c.li    t6, 0
core   0: 0x80000040 (0x80100137) lui     sp, 0x80100
core   0: 0x80000044 (0x800102b7) lui     t0, 0x80010
core   0: 0x80000048 (0x04028293) addi    t0, t0, 64
(spike) 
core   0: 0x8000004c (0x01729073) csrw    jvt, t0
(spike) 
core   0: 0x80000050 (0x00004281) c.li    t0, 0
(spike) 
core   0: 0x80000052 (0x0000a1a6) cm.jalt 105
(spike) 
core   0: exception trap_instruction_access_fault, epc 0x00000000
core   0:           tval 0x00000000
(spike) 
core   0: exception trap_instruction_access_fault, epc 0x00000000
core   0:           tval 0x00000000

Now I just need to manually add the function pointer to JVT?

[gakubochama]

If you could give me a simple example using zcmt, I would be very happy.

[jnk0le]

according to psabi you need to add .riscv.jvt (input) section to the linker script, with __jvt_base$ symbol at the beggining
then:

    # Recommended way to initialize the jvt CSR.
1:  auipc a0, %pcrel_hi(__jvt_base$)
    addi  a0, a0, %pcrel_lo(1b)
    csrw  jvt, a0

https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc

[gakubochama]

@jnk0le

Thank you so much for telling me.

↓crt0.s

    .section .text.init
    .globl _start
_start:
    nop
    li x1, 0
    li x2, 0
    li x3, 0
    li x4, 0
    li x5, 0
    li x6, 0
    li x7, 0
    li x8, 0
    li x9, 0
    li x10, 0
    li x11, 0
    li x12, 0
    li x13, 0
    li x14, 0
    li x15, 0
    li x16, 0
    li x17, 0
    li x18, 0
    li x19, 0
    li x20, 0
    li x21, 0
    li x22, 0
    li x23, 0
    li x24, 0
    li x25, 0
    li x26, 0
    li x27, 0
    li x28, 0
    li x29, 0
    li x30, 0
    li x31, 0

    # set the stack pointer 64 KiB
    li sp, 0x80100000

    # set the jvt csr
1:  auipc a0, %pcrel_hi(__jvt_base$)
    addi  a0, a0, %pcrel_lo(1b)
    csrw  jvt, a0
    li a0, 0

    jal main

    # shutdown
    li t0, 1
    sw t0, tohost, t1
1:  j 1b


put_string:
    li t0, 0
1:  lbu t1, (a0)
    beqz t1, 1f
    or t1, t1, t0
    sw t0, tohost, t2
    addi a0, a0, 1
    j 1b
1:  ret

    .globl start_trigger
start_trigger:
    la a0, str_start
    j put_string
    .globl stop_trigger
stop_trigger:
    la a0, str_stop
    j put_string

    .section .tohost
    .align 6
    .globl tohost
tohost: .dword 0
    .align 6
    .globl fromhost
fromhost: .dword 0

    .section .riscv.jvt
__jvt_base$: .dword 0

    .section .rodata
    .align 4
str_start:
    .string "start!\n"
    .align 4
str_stop:
    .string "stop!\n"
    .align 4

↓link.ld

OUTPUT_ARCH(riscv)
ENTRY(_start)

SECTIONS
{
  . = 0x80001000;
  .tohost : { *(.tohost) }
  . = 0x80000000;
  .text.init : { *(.text.init) }
/*
  . = ALIGN(0x1000);
  .tohost : { *(.tohost) }
*/
  . = ALIGN(0x2000);
  .text : { *(.text) }
  . = ALIGN(0x1000);
  .rodata : { *(.rodata) }
  .data : { *(.data) }
  .bss : { *(.bss) }
  . = ALIGN(0x10000);
  .riscv.jvt : { *(.riscv.jvt) }
  . = ALIGN(0x80);
}

I changed crt0.s & link.ld in this way.
And I show part of the disassembly below.

.
.
.
Disassembly of section .text.init:

80000000 <_start>:
80000000:	0001                	nop
80000002:	4081                	li	ra,0
80000004:	4101                	li	sp,0
80000006:	4181                	li	gp,0
80000008:	4201                	li	tp,0
8000000a:	4281                	li	t0,0
8000000c:	4301                	li	t1,0
8000000e:	4381                	li	t2,0
80000010:	4401                	li	s0,0
80000012:	4481                	li	s1,0
80000014:	4501                	li	a0,0
80000016:	4581                	li	a1,0
80000018:	4601                	li	a2,0
8000001a:	4681                	li	a3,0
8000001c:	4701                	li	a4,0
8000001e:	4781                	li	a5,0
80000020:	4801                	li	a6,0
80000022:	4881                	li	a7,0
80000024:	4901                	li	s2,0
80000026:	4981                	li	s3,0
80000028:	4a01                	li	s4,0
8000002a:	4a81                	li	s5,0
8000002c:	4b01                	li	s6,0
8000002e:	4b81                	li	s7,0
80000030:	4c01                	li	s8,0
80000032:	4c81                	li	s9,0
80000034:	4d01                	li	s10,0
80000036:	4d81                	li	s11,0
80000038:	4e01                	li	t3,0
8000003a:	4e81                	li	t4,0
8000003c:	4f01                	li	t5,0
8000003e:	4f81                	li	t6,0
80000040:	80100137          	lui	sp,0x80100
80000044:	00010517          	auipc	a0,0x10
80000048:	ff850513          	addi	a0,a0,-8 # 8001003c <__jvt_base$>
8000004c:	01751073          	csrw	jvt,a0
80000050:	4501                	li	a0,0
80000052:	a1a6                	cm.jalt	105
80000054:	4285                	li	t0,1
80000056:	00001317          	auipc	t1,0x1
8000005a:	fa532523          	sw	t0,-86(t1) # 80001000 <tohost>
8000005e:	a001                	j	8000005e <_start+0x5e>
.
.
.
Disassembly of section .riscv.jvt:

8001003c <__jvt_base$>:
	...

It still does not seem to generate a valid jump table.
I read the documentation and your advice and made my own interpretation and changes, but it did not work.
I apologize for my lack of study.

Could you be more specific...?

[jnk0le]

Get rid of this:

.section .riscv.jvt
__jvt_base$: .dword 0

and define it as a linker symbol:

. = ALIGN(0x10000);
.riscv.jvt : { 
    __jvt_base$ = .;
    *(.riscv.jvt) 
}

[abukharmeh]

Embench is too small to make any use of Zcmt. The linker should auto relax jumps to the correct entries in the table and auto fill the table once you have had its location defined as recomended (If and only if it thinks that it will be able to save code size doing so). In your dissassembly I see a cm.jalt 105. I doubt this was a result of a linker relaxation.

If the intention is to simply test Zcmt, then you need to fill the table manually with correct addresses pointing to functions for a directed test. Otherwise I would get a larger benchmark to test with !

[gakubochama]

@jnk0le

I appreciate it.

I changed like this.

↓crt0.s

    .section .text.init
    .globl _start
_start:
    nop
    li x1, 0
    li x2, 0
    li x3, 0
    li x4, 0
    li x5, 0
    li x6, 0
    li x7, 0
    li x8, 0
    li x9, 0
    li x10, 0
    li x11, 0
    li x12, 0
    li x13, 0
    li x14, 0
    li x15, 0
    li x16, 0
    li x17, 0
    li x18, 0
    li x19, 0
    li x20, 0
    li x21, 0
    li x22, 0
    li x23, 0
    li x24, 0
    li x25, 0
    li x26, 0
    li x27, 0
    li x28, 0
    li x29, 0
    li x30, 0
    li x31, 0

    # set the stack pointer 64 KiB
    li sp, 0x80100000

    # set the jvt csr
1:  auipc a0, %pcrel_hi(__jvt_base$)
    addi  a0, a0, %pcrel_lo(1b)
    csrw  jvt, a0
    li a0, 0

    jal main

    # shutdown
    li t0, 1
    sw t0, tohost, t1
1:  j 1b


put_string:
    li t0, 0
1:  lbu t1, (a0)
    beqz t1, 1f
    or t1, t1, t0
    sw t0, tohost, t2
    addi a0, a0, 1
    j 1b
1:  ret

    .globl start_trigger
start_trigger:
    la a0, str_start
    j put_string
    .globl stop_trigger
stop_trigger:
    la a0, str_stop
    j put_string

    .section .tohost
    .align 6
    .globl tohost
tohost: .dword 0
    .align 6
    .globl fromhost
fromhost: .dword 0

    .section .rodata
    .align 4
str_start:
    .string "start!\n"
    .align 4
str_stop:
    .string "stop!\n"
    .align 4

↓link.ld

OUTPUT_ARCH(riscv)
ENTRY(_start)

SECTIONS
{
  . = 0x80001000;
  .tohost : { *(.tohost) }
  . = 0x80000000;
  .text.init : { *(.text.init) }
  /*. = ALIGN(0x10000);*/
  . = ALIGN(0x2000);
  .text : { *(.text) }
  . = ALIGN(0x1000);
  .rodata : { *(.rodata) }
  .data : { *(.data) }
  .bss : { *(.bss) }
  . = ALIGN(0x10000);
  .riscv.jvt : { 
    __jvt_base$ = .;
    *(.riscv.jvt) 
  }
}

But I couldn't execute on Spike.

As the contributor says, is the linker not generating the jump table because the embench is too small to use zcmt?

[gakubochama]

@abukharmeh
Thanks, appreciate it!

I have a few questions.

What is relaxation for a linker?

Why is the jump instruction compiled as cm.jalt 105 when linker has determined that Zcmt is ineffective in reducing code size?

You mentioned that we should manually create a jump table to test the zcmt.
How exactly do we manually create a jump table?
Is it like writing it directly in assembly language?
If you have an example of manually creating a jump table, I would like to know.

You also say that another alternative is that we should use a larger benchmark, what is the specific benchmark?

[jnk0le]

Why is the jump instruction compiled as cm.jalt 105 when linker has determined that Zcmt is ineffective in reducing code size?

The concern is that a tbljal to a single entry has a net negative size reduction, and there is at least 105 of those on a benchmark that tends to be massively inlined.
Are you compiling it with -O0 or maye an old compiler build?

You can show also the most recent output + spike messages.

[jnk0le]

One more thing: on eclipse CDT build, after changing anything in linker files, you have to do a project clean and modify&save at least one of the project source files. Otherwise the old linker file is somehow still in use. Not sure how it works in raw compilations.

[jnk0le]

also initialization of the global pointer is missing

[gakubochama]

@jnk0le

Thanks a lot.

These are flags for compilers and linkers

cc = 'path/to/riscv-gnu-toolchain/rv32ima_zicsr_zca_zcb_zcmp_zcmt/bin/riscv32-unknown-elf-gcc'
#cflags = (['-c', '-O2', '-march=rv32i', '-mabi=ilp32', '-ffunction-sections', '-fdata-sections'])
cflags = (['-c', '-O2', '-march=rv32ima_zicsr_zca_zcb_zcmp_zcmt', '-mabi=ilp32', '-nostartfiles'])
#ldflags = (['-march=rv32i', '-mabi=ilp32', '-Tlink.ld', '-Wl,-gc-sections'])
ldflags = (['-march=rv32ima_zicsr_zca_zcb_zcmp_zcmt', '-mabi=ilp32', '-nostartfiles', '-T../../../config/riscv32/boards/rv32ima_zicsr_zca_zcb_zcmp_zcmt/link.ld'])
user_libs = (['-lm'])
cpu_mhz = 1

Since the jump table did not exist, the spike message was not useful information.

One more thing: on eclipse CDT build, after changing anything in linker files, you have to do a project clean and modify&save at least one of the project source files. Otherwise the old linker file is somehow still in use. Not sure how it works in raw compilations.

Sorry, I did not know how to improve on this↑ part...

also initialization of the global pointer is missing

So, how do we initialize the global pointer?

I am not quite sure about the role of the global pointer...

[jnk0le]

    # Recommended way to initialize the gp register.
    .option push
    .option norelax
1:  auipc gp, %pcrel_hi(__global_pointer$)
    addi  gp, gp, %pcrel_lo(1b)
    .option pop

.data : { 
    __global_pointer$ = .
    *(.data) 
}

It should be enough, normally it's done like like this: https://github.com/openwch/ch32v003/blob/main/EVT/EXAM/SRC/Ld/Link.ld#L117

[gakubochama]

@jnk0le

I set up gp as you said, but it failed to compile...
crt0.S

    .section .text.init
    .globl _start
_start:
    nop
    li x1, 0
    li x2, 0
    li x3, 0
    li x4, 0
    li x5, 0
    li x6, 0
    li x7, 0
    li x8, 0
    li x9, 0
    li x10, 0
    li x11, 0
    li x12, 0
    li x13, 0
    li x14, 0
    li x15, 0
    li x16, 0
    li x17, 0
    li x18, 0
    li x19, 0
    li x20, 0
    li x21, 0
    li x22, 0
    li x23, 0
    li x24, 0
    li x25, 0
    li x26, 0
    li x27, 0
    li x28, 0
    li x29, 0
    li x30, 0
    li x31, 0

    # Recommended way to initialize the gp register.
    .option push
    .option norelax
1:  auipc gp, %pcrel_hi(__global_pointer$)
    addi  gp, gp, %pcrel_lo(1b)
    .option pop

    # set the stack pointer 64 KiB
    li sp, 0x80014000

    # set the jvt csr
1:  auipc a0, %pcrel_hi(__jvt_base$)
    addi  a0, a0, %pcrel_lo(1b)
    csrw  jvt, a0
    li a0, 0

    jal main

    # shutdown
    li t0, 1
    sw t0, tohost, t1
1:  j 1b

link.ld

OUTPUT_ARCH(riscv)
ENTRY(_start)

SECTIONS
{
  . = 0x80001000;
  .tohost : { *(.tohost) }
  . = 0x80000000;
  .text.init : { *(.text.init) }
  . = ALIGN(0x2000);
  .text : { *(.text) }
  . = ALIGN(0x1000);
  .rodata : { *(.rodata) }
  .data : { 
    __global_pointer$ = .
    *(.data) 
  }
  .bss : { *(.bss) }
  . = ALIGN(0x10000);
  .riscv.jvt : { 
    __jvt_base$ = .;
    *(.riscv.jvt) 
  }
}

Could you please tell me what the difference is?

Sorry for repeating myself. I appreciate it.