GPU Compute

[max-mapper]

Outside of browsers there are things like CUDA and OpenCL for writing code to run on graphics cards. The general term for this is GPGPU

In the browser we currently only have WebGL. With WebGL in order to send data in and out of the graphics card you have to use image buffers:

(following image taken from this presentation)

There is a specification called WebCL which is web bindings to OpenCL. However, no major browser has implemented it and Firefox even said they have no plans to implement it: https://bugzilla.mozilla.org/show_bug.cgi?id=664147#c30, instead saying "The future for GPU compute in Firefox is ARB_compute_shader".

Open questions:

• What is that actual status of GPU compute, when will we be able to use it in browsers?
• What are the positives/negatives of OpenCL vs ARB_compute shader for scientific computing applications?
• What scientific computing applications actually need GPGPU that can't be done today using the image buffer tricks?

Commants/feedback? Leave a comment below

[znmeb]

GPU computing in the open source world is a bit of a morass - patents, competition, etc. You'll probably find you need lawyers and signed contracts to "get at the good stuff". I've given up on it - even on Linux workstations with proprietary drivers provided by the GPU vendor a lot of things don't work or require more effort than the resulting performance improvement justifies. Your mileage may vary.

[hughsk]

@maxogden somewhat tangentially: Vulkan is supposed to be coming to desktops "soon", which opens up a standardised cross-platform API with similar goals to CUDA, Metal, etc. That might mean that things start to clean up for OSS GPGPU stuff over time. We'll probably not get anything similar in browsers for years though, and even then there's security considerations that come with the added flexibility which might mean browser vendors avoid it entirely. Compute shaders open up some cool opportunities but I can't speak for how it compares to OpenCL, especially in the context of scientific computing...

[kgryte]

@hughsk For my own knowledge, what are those security considerations?

[mikolalysenko]

WebCL is pretty much dead. I think compute shaders and longer term some Vulkan-like thing would be better.

Today there is a lot we can do with WebGL, but we are currently hamstrung by synchronous data transfers and the overhead of extremely rigorous data validation. It is possible to get good performance if you can keep everything on the GPU, but bouncing back and forth between JS <-> WebGL doesn't really give much advantage yet.

[kgryte]

@mikolalysenko what led to the demise of WebCL? And why would compute shaders / Vulkan be better?

[binarymax]

I couldn't find the reference, but I remember reading that WebCL was too dangerous, and there was no effective way to prevent a hostile script from accessing unauthorized system memory or crashing the machine.

[mikolalysenko]

Also on the desktop compute shaders make it sort of irrelevant, and it never really caught up to the functionality and performance of CUDA.

Long term I think compute shaders and ultimately Vulkan/SPIR-V are the future. OpenCL and especially WebCL seem like evolutionary dead ends.

[kgryte]

@mikolalysenko Can you elaborate as to why you consider them evolutionary dead ends?

@binarymax My understanding is that, yes, some initial concerns around WebCL were unbounded memory access and something like denial-of-service. However, part of the WebCL spec was amended to include out-of-bounds checks and safe guards against DOS.

I can see where compute shaders may make sense: where computation is intimately tied to rendering. But for cases where all that is desired is computation, I see writing/compiling GPGPU code in GLSL as unnecessary and, on the surface, hacky, especially as OpenGL is first and foremost a graphics language, not a compute language.

And even if WebCL never caught up to functionality and performance of CUDA (which I am not convinced is a realistic expectation given the tight coupling of CUDA with NVIDIA hardware), if using it would provide speed-ups over, say, using web workers for parallel compute, that would still be a win and generally useful.

[kgryte]

A particular use case I have in mind is as follows...

During my research, we invested considerable time and effort parallelizing maximum likelihood estimation of parameters of multivariate distributions (e.g., in order to extract mean photon counts and point-spread-function (PSF) widths). Initially, we used multicore computing, splitting video frames across cores. This resulted in a speed-up, but was obviously limited by the number of cores.

So, we then evaluated using GPUs (NVIDIA graphics cards + CUDA) to further parallelize our algorithms. By doing so, we were able to reduce analysis of 2-3 GB videos from 30 minutes to less than 30 seconds. This was huge, as it meant we could do near-real-time high precision analysis, allowing us to adjust our experiments on-the-fly and as-needed. And when analyzing a day's worth of videos (~100), going from 50 hours to 50 minutes was great as this meant a reduced turnaround time for insight to action.

One problem, however, was that we had to buy dedicated equipment (~$10K mega PCs) which came equipped with NVIDIA GPUs. These were in limited supply, so analysis of multiple people's data was often queued, thus increasing turnaround time. In contrast, everyone was able to run analysis locally using a multicore approach.

Where I see WebCL (or some alternative for GPGPU compute in browser) being advantageous in this situation is that people could run analysis locally using their own GPU which came with their Mac. Admittedly, this will, in all likelihood, not be as performant as the NVIDIA graphics cards we used, but would be an improvement over the multicore approach and would fit nicely with the in-browser analysis software we built for analyzing experimental data. By analyzing everything in-browser/locally, we would avoid having to traffic data around and contend for limited resources.

[waylonflinn]

I've got a start on pulling the ideas around WebGL based methods into a single place that's easy for people to try out. I've only been working on it for a week or so, but I've got performance on par with OpenBLAS for GEMM.

You can find it here: https://github.com/waylonflinn/weblas

This builds on the work of several others, including @watmough and @petewarden.

My goal with the library is to enable Machine Learning and Deep Learning applications in the browser. There's been so much great work on this in other languages and platforms, but it's so difficult to set up. It's still inaccessible to lots of interested people. Bringing the same performance to javascript, could really accelerate applications in the field.