Did you reimplement for object detection?

[twmht]

Hi,

Did you reimplement for object detection? I have tried ReviewKD for my own dataset and my own model, but found out it's not good.

[zjykzj]

Hi,

Did you reimplement for object detection? I have tried ReviewKD for my own dataset and my own model, but found out it's not good.

No related plan. In my practice, the effectiveness of distillation depends very much on the data set used. by the way, I also recommend you to try another distillation algorithm - https://github.com/ZJCV/overhaul

[twmht]

@zjykzj

Yup. Recently I have tried many distilltion methods which work on COCO well, but for widerface I can't reproduce a better student. it's very sensitive with the learning rate and other hyper parameters. It would be great if you can share your experience on this.

overhaul is on going.

[zjykzj]

What scene are you working in? Beside distillation, i also recommend you to try network pruning, which works for me. You can check another two repositories that i did for network pruning.

1. ZJCV/NetworkSlimming
2. ZJCV/SSL

[twmht]

Well, I have also tried network slimming or other pruning methods(https://github.com/microsoft/nni) before, but the result is the same as https://arxiv.org/abs/1810.05270. So I am using anytime network to do the pruning now, it's much easier to implement. but I also want to migrate distilling after pruning a network structure.

I have experimented KD like overhaul and Knowledge Review on object detection, but I found out they are not stable when training, we may have loss divergence in the middle of the training.

other methods like https://github.com/yzd-v/FGD, which is specific for object detection but very hard to tune on my own dataset.