sodimm_ddr5_tester target has timing errors

[sqazi]

Commandline:

make build TARGET_ARGS="--l2-size 256 --build --iodelay-clk-freq 400e6 --bios-lto --rw-bios --no-sdram-hw-test"

Git Commit : 3664720

See attached logs.

vivado.log

antmicro_sodimm_ddr5_tester_timing.rpt.log

[mtdudek]

Hi @sqazi

Recently we've also noticed some instabilities when building bitstreams.
It was due to some race condition inside Vivado, which causes it to be no longer deterministic.
We solved it by adding extra line to the Vivado initialization script.

echo "set_param general.maxThreads 1" > $XILINX_HOME/scripts/Vivado_init.tcl

It disables multi threading, but makes Vivado builds deterministic.

Can you test newest main branch?
It has some payload executor improvements that should improve design timings with larger payload sizes.

[sqazi]

Thanks for the suggestion. I tried that and it did not help. Additionally, do we need changes to the script to utilize the new payload executor implementation correctly? Some of my private scripts are not running the payload correctly anymore on top-of-tree and reporting 3 cycles of execution time for the payload.

I am using the default payload size but note that the target is sodimm_ddr5_tester.

[mtdudek]

Adding pipelining unfortunety added some execution time to the whole payload. It takes 2 cycles from start to instruction being sent on the dfi bus.
There is also another change that affects jumps, they now require extra 2 cycles to allow for pipeline to refill with correct instructions.

But this would only change payload execution by 2 multiples, is it possible for you to share this payload so we can test it locally?

[sqazi]

I can get similar behavior out of rowhammer.py:

$ python ./rowhammer.py --row-pairs const --const-rows-pair 1 3 --read_count 100 --payload-executor --no-refresh
Using generated target files in: ../../build/sodimm_ddr5_tester
Board info: Row Hammer Tester SoC on xc7k160tffg676-3, git: f00aea42e0e4eb586af1da1fbbf38fafa5d595a5 2025-01-27 15:08:13

Preparing ...

Filling memory with data ...

Disabling refresh ...

Running Rowhammer attacks ...
Repeatable unit: 118
Repetitions: 59

Transferring the payload ...

Executing ...
Payload execution started at: 596800539735
Payload execution finished at: 596800539738
Total execution time (system clock cycles): 3

Total elapsed time: 0.253 ms

Reenabling refresh ...

Verifying attacked memory ...
OK

Note that exactly 3 cycles are measured as execution time :(.

[sqazi]

Note: the command works fine the first time, and the second time it doesn't actually execute the payload and returns after 3 cycles.

[sqazi]

I tried the latest change, and now the payloads are executing properly. Thanks for the fix!

But, I still need to nail down the timing issues. What version of Vivado are you using, and what are the command lines for ddr5_tester and sodimm_ddr5_tester that you are using? I've been using 2020.2, mostly out of inertia. But I can update if needed.

[mtdudek]

We are using Vivado 2023.2 version running within Debian 12 container.
We are building sodimm_ddr5_tester using this command:
TARGET=sodimm_ddr5_tester make build ARGS="--l2-size 256 --build --iodelay-clk-freq 400e6 --bios-lto --rw-bios --no-sdram-hw-test"
and this command for ddr5_tester:
TARGET=ddr5_tester make build ARGS: "--l2-size 256 --build --iodelay-clk-freq 400e6 --bios-lto --rw-bios --module MTC10F1084S1RC --no-sdram-hw-test"

[sqazi]

For the ddr5_tester, with the default arguments and Vivado 2023.2 and the Vivado_init.tcl contents, it builds without timing errors.

However, if I add SPD data, it shows timing errors:

make build ARGS="--l2-size 256 --build --iodelay-clk-freq 400e6 --bios-lto --rw-bios --no-sdram-hw-test --from-spd /home/sqazi/dram_spds/mtc20f2085s1rc48ba1.bin"

specifically :

main_crg_s7mmcm_clkout2 -0.177 -32.800 489 59528 0.044 0.000 0 59528 1.826 0.000 0 20798

[sqazi]

Please ignore my previous comment. I had placed Vivado_init.tcl in the wrong place :(. Trying again...

[sqazi]

Nope... same issue:

main_crg_s7mmcm_clkout2 -0.177 -32.800 489 59528 0.044 0.000 0 59528 1.826 0.000 0 20798

[sqazi]

For the sodimm_ddr5_tester, the timing is not met even with the default command line (i.e. without specifying an SPD file). The issue is again this one:

main_crg_s7mmcm_clkout2 -0.068 -2.909 86 93748 0.052 0.000 0 93748 1.826 0.000 0 32840

[mtdudek]

I'm looking into it and I'll update you as soon as I'll find something that improves timing.

[mtdudek]

Hi @sqazi,

Can you check timings with newest main?
We updated LiteDRAM submodule with some fixes that should improve timing closure.
Timing report for our internal builds show no timing violations for default configuration.

[sqazi]

So, I tried for ddr5_tester and it seems to work both for the default command line and with one particular module's (mtc20f2085s1rc48ba1) SPD data.

The sodimm_ddr5_tester works for the default commandline, but if I try to use the attached SPD data then there are timing violations. I had to hexdump the SPD data to get github to accept it as an attachment.

mtc8c1084s1sc48ba1.bin.hex.txt

I haven't tried increased payload sizes yet.