CIFAR 100 Performance discrepancy

[chojw]

Hi,
I really appreciate that you shared your code online.
I tried testing out your code on CIFAR10 and got similar performance to the paper, but after testing on CIFAR100, the reported scores are quite far off. I am getting around 20% accuracy with 10% data, and only around 39% accuracy at 40% data.
I thought it could be the instability of the VAAL, so I tested random sampling and I seem to be getting similar performance here.

Is there parameters I need to change or that I am missing?

Thanks in advance

[jer112]

Hi, @chojw, I wonder whether this is caused by the normalization. Maybe you could try the normalization values in https://github.com/Mephisto405/Learning-Loss-for-Active-Learning.

Also, I actually find it difficult to achieve the paper's performance on CIFAR10. I found the discriminator loss to be fixed or stuck after running a few thousand iterations. Could you please share what parameters you've been using or did you change anything? Did you use default values for all parameters? Thanks in advance!

[chojw]

Hi, @jer112,
I have tried several different normalizations but I am getting similar results. Have you gotten anything close to those reported in the paper for CIFAR-100??

Also for CIFAR-10, I just used the default settings and when I meant similar, I meant around 1~3% off from the actual paper, but still the curves seem quite similar. For example, I got
61.14 | 66.82 | 70.9 | 74.28 | 75.63 | 77.05 | 79.18
while the reported is roughly if you eyeball it off of the paper
61 | 68.5 | 72.1 | 77.5 | 79 | 80 | 81

Let me know if you have anymore questions!
Also if you could let me know what results you get on CIFAR-100 it would be nice.
Thanks in advance!