Problem with SimpleAutoSchedule for GPU

[GunnarDahlgren]

Hi,

Being a (gradient) Halide newbie, I've only made some simple stuff yet, but I've come across an issue with the SimpleAutoScheduler I don't understand. I later discovered that I get exactly the same kind of errors if I try to schedule the tests in src/SimpleAutoSchedule.cpp for the GPU... I did the following

 > make test_internal
works fine by default.

I then did this edit:
--- a/src/SimpleAutoSchedule.cpp
+++ b/src/SimpleAutoSchedule.cpp
@@ -837,6 +837,7 @@ namespace Internal {
 void simple_autoschedule_test() {
     // For now we just test whether it compiles or not.
     SimpleAutoscheduleOptions cpu_options;
 +    cpu_options.gpu = true;
     Var x("x"), y("y"), z("z");
     { // Simple pointwise operations. Should inline.
         Func in("in");

, and ran the tests again with:

 > export HL_JIT_TARGET=x86-64-linux-avx-opencl-sse41
(which I believe is the correct target for my machine - is the same as is returned by find_gpu_target() in tutorials/lesson12)
 > make test_internal
now fails with this error message:
Error at gradient-halide/test/internal.cpp:45:
Input buffer b17 is accessed at 16387, which is beyond the max (16383) in dimension 0
bash: line 1: 24583 Aborted                 gradient-halide/bin/test_internal
Makefile:1523: recipe for target 'test_internal' failed
make: *** [test_internal] Error 134

(gdb tells me the error happens in the test at src/SimpleAutoSchedule.cpp:873: Buffer output = conv.realize(16384 - 5); )

I'm not sure if I'm doing something fundamentally in the wrong way here though... Any feedback would be very much appreciated.
Thanks, brgds
Gunnar

[mgharbi]

Tzu-Mao Li might know better, but we have not tested the scheduler on OpenCL GPU hardware, only CUDA at the moment. There might be a mismatch between the size of the thread block, shared memory and our hard coded constants. You may want to look at the tail_strategy wherever there is a gpu_tile command in the scheduler as well, and perhaps set it to GuardWithIf: https://halide-lang.org/docs/class_halide_1_1_stage.html#a6560a421fac168dadd4fb0a056550a68

[BachiLi]

This is indeed a bug on our side related to the tail strategy. I have fixed this in other branches but haven't ported it back yet.
Michael is also correct in that OpenCL backend might produces incorrect results: we use atomicAdd in the CUDA backend for general scattering, and we didn't do the same to OpenCL. I can look into this.

[GunnarDahlgren]

Many thanks for the feedback! I will test using CUDA on monday; for the moment OpenCL/CUDA does not matter to us, even though OpenCL is preferred here in the long run.
@BachiLi , any chance you can port back the bugfix too in near time?

[BachiLi]

Should be fixed now. Let me know if there's any issues. I'll keep this issue open as I need to write tests.

[GunnarDahlgren]

Super, my code now works on GPU as well 👍 (tested both OpenCL and CUDA with identical results) Thanks a lot for your help.
FYI, to be able to run make on my machine I had to re-uncomment line 56 in Makefile (LLVM_CXX_FLAGS += -Qunused-arguments), which had been commented since my last pull, otherwise I got an error (something with "-fuse-ld=gold" not being used...)