About Sparse-conv module

[Overtown]

hi, thanks for your project, I would like to know how to construct sparse-conv that mentioned in your paper?

[IwakuraRein]

Seem like this PyTorch project only simulates the skip-conv with the nn.Conv2d. 🤔

Does this mean I won't have speed improvement from deploying these codes in my project as sparse-conv isn't truly implemented?

[dabeschte]

Looks like it. This implementation uses a dense convolution + extra work, so it is even slower than the dense version. This is only a proof of concept

[IwakuraRein]

Looks like it. This implementation uses a dense convolution + extra work, so it is even slower than the dense version. This is only a proof of concept

This seems the implementation of sparse-conv (paper). I guess the skip-conv can be achieved by deploying the NormGate/GumbelGate into it.

[IwakuraRein]

I tried SBNet's lib but weirdly seeing my network is heavily slower. I started an issue but since this repo is 2018's, I'm unlikely to get responses there. 😥

Their paper said they implemented skip-conv with img2col based approach from More is less: A more complicated network with less inference complexity. I wonder where these codes are released.

[jiaerwang0328]

＠IwakuraRein　I've been working on this problem recently, have you found a solution?

[dabeschte]

CBInfer is the only working sparse video framework (that I was able to find) that can actually improve frame rates of some CNNs. However, it does not support batched inference - you will probably only see speedups over cuDNN on low end GPUs because of that.

[jiaerwang0328]

@dabeschte
I followed the semantic segmentation example provided by cbinfer and found that the time cost is similar:

time_cb: 0.13297295570373535
time_ref: 0.13713407516479492
accuracy (reference): 94.11%
accuracy (CBinfer): 93.88%

Am I doing something wrong? GPU: 2060s, cuda10.0, pytorch: 1.4.0

[dabeschte]

yes, the problem here is that CBInfer does not support batched inference out of the box and your GPU is not fully utilized with a batch size of 1 - utilizing less of it does not make inference faster, because you always have to wait on the one core that is actually processing a pixel.
It can make a difference when you use huge feature maps (I guess mainly with large resolutions).

[jiaerwang0328]

@dabeschte I get it, thank you

[dabeschte]

@wuranran @IwakuraRein @Overtown
I developed my own sparse CNN implementation that provides CUDA kernels and a PyTorch integration. The repository is still work in progress (will do more refactoring and add more examples), but I wanted to give you early access since you seem to be looking for a sparse convolution for a while now.
Check out my repository DeltaCNN