Questions on balanced data

[zhiqihuang]

Thank you for the nice project. I have a few questions about label distributions.

• Does the AUCMLoss or CompositionalAUCLoss work for balanced training data? Suppose both train and test are (almost) balanced binary classification tasks.
• What if the training data is balanced while the test data is imbalanced?

Additionally, I have noticed that AUCMLoss converges very slowly on balanced training data. Do you have any suggestions on the training parameters?

Thanks again!

[optmai]

Thanks for your interest!

• Yes, I think they work for balanced training data as long as you optimize AUC. Nevertheless, it is unclear if it can deliver better performance on standard cross-entropy loss.

• It is a good question. If the test data is imbalanced, you should definite consider AUC maximization or even AUPRC maximization (SOAP) or partial AUC maximization (SOPA-s).

There are many factors could affect the convergence, including the nature of your data, the model structure and also the optimizer.