The meaning of stripe size

[xzy789]

Dear code author:
What is the meaning of stripe size in compilation options？I do not understand it.
Hope to get your answer.
Ziyue, Xu
Nankai university

[TheNetAdmin]

I don't find a stripe size option, do you mean stride_size?

Also, the Makefile does not contain this compilation option, what options are you referring to?

[xzy789]

I'm sorry I didn't make it clear.
For examplke, When I input ‘echo "task=3,access_size=1024,parallel=24,stride_size=1024" > /proc/lattester'， I do not know the meaning of stride_size .
Hope to get your answer.
Ziyue, Xu
Nankai university

[TheNetAdmin]

The task=3 refers to Strided Bandwidth Test:

OptaneStudy/src/kernel/lat.c

Line 24 in a8bf9d1

"Strided Bandwidth Test",

The corresponding implementation is here:

OptaneStudy/src/kernel/tasks.c

Lines 331 to 367 in a8bf9d1

	int strided_bwjob(void *arg)
	{
	struct latencyfs_worker_ctx *ctx = (struct latencyfs_worker_ctx *)arg;
	struct latency_sbi *sbi = ctx->sbi;
	static unsigned long flags;
	int op = ctx->sbi->op;
	long access_size = ctx->sbi->access_size;
	long stride_size = ctx->sbi->strided_size;
	long delay = ctx->sbi->delay;
	struct latencyfs_timing *timing = &sbi->timing[ctx->core_id];
	uint8_t *buf = ctx->addr;
	uint8_t *region_end = buf + PERTHREAD_WORKSET;
	long count = PERTHREAD_CHECKSTOP / access_size;
	if (stride_size * count > PERTHREAD_WORKSET)
	count = PERTHREAD_WORKSET / stride_size;
	kr_info("Working set begin: %p end: %p, size=%ld, stride=%ld, delay=%ld, batch=%ld\n", buf, region_end, access_size, stride_size, delay, count);
	BENCHMARK_BEGIN(flags);
	while (1) {
	lfs_stride_bw[op](buf, access_size, stride_size, delay, count);
	timing->v += access_size * count;
	buf += stride_size * count;
	if (buf + access_size * count >= region_end)
	buf = ctx->addr;
	if (kthread_should_stop())
	break;
	schedule();
	}
	BENCHMARK_END(flags);
	wait_for_stop();
	return 0;
	}

And low-level memory access implementation depends on op operation, which selects one of the strided memory access jobs:

OptaneStudy/src/kernel/memaccess.h

Lines 292 to 299 in a8bf9d1

	static lfs_test_stridefunc_t lfs_stride_bw[] = {
	&stride_load,
	&stride_nt,
	&stride_store,
	#ifdef AEP_SUPPORTED
	&stride_storeclwb,
	#endif
	};

With op defaults to 0, you are using stride_load:

OptaneStudy/src/kernel/memaccess.c

Lines 280 to 312 in a8bf9d1

	void stride_load(char *start_addr, long size, long skip, long delay, long count)
	{
	KERNEL_BEGIN
	asm volatile (
	"xor %%r8, %%r8 \n" /* r8: access offset */
	"xor %%r11, %%r11 \n" /* r11: counter */
	// 1
	"LOOP_STRIDELOAD_OUTER: \n" /* outer (counter) loop */
	"lea (%[start_addr], %%r8), %%r9 \n" /* r9: access loc */
	"xor %%r10, %%r10 \n" /* r10: accessed size */
	"LOOP_STRIDELOAD_INNER: \n" /* inner (access) loop, unroll 8 times */
	SIZEBTLD_64_AVX512 /* Access: uses r10[size_accessed], r9 */
	"cmp %[accesssize], %%r10 \n"
	"jl LOOP_STRIDELOAD_INNER \n"
	SIZEBTLD_FENCE
	"xor %%r10, %%r10 \n"
	"LOOP_STRIDELOAD_DELAY: \n" /* delay <delay> cycles */
	"inc %%r10 \n"
	"cmp %[delay], %%r10 \n"
	"jl LOOP_STRIDELOAD_DELAY \n"
	"add %[skip], %%r8 \n"
	"inc %%r11 \n"
	"cmp %[count], %%r11 \n"
	"jl LOOP_STRIDELOAD_OUTER \n"
	:: [start_addr]"r"(start_addr), [accesssize]"r"(size), [count]"r"(count), [skip]"r"(skip), [delay]"r"(delay):
	"%r11", "%r10", "%r9", "%r8");
	KERNEL_END
	}

In this case, stride_size=1024 is passed to stride_load's parameter skip, which defines the offset between the current memory access start address and the next one's:

OptaneStudy/src/kernel/memaccess.c

Line 303 in a8bf9d1

"add %[skip], %%r8 \n"

[xzy789]

Thank you for your detailed answer. I have one last question: if I want to do a random write(ntstore) of 64B
test with 24 threads, should I input ‘echo "task=4,op=1,access_size=64,parallel=24" > /proc/lattester'?
Hope to get your answer.