Training with EfficientNet0

[kwade4]

Initial training using:

• EfficientNet0
• Adam optimizer
• Learning rate of 1e-3
• 500 epochs
• Batch size of 48

[kwade4]

Validation accuracy was ~86%, and increasing slowly after 500 epochs, but the loss function was almost at 0.
It seems like the model was overfitting, so retrying using EfficientNet0 with pre-trained weights.

[kwade4]

The batch loss is oscillating, so maybe the learning rate is too high. Also looking into learning rate schedulers.
Validation accuracy was ~96%.

[kwade4]

With AdamW, StepLR learning rate scheduler (gamma=0.95, step=2), and pre-trained weights, validation accuracy is ~98%.

[kwade4]

Best performing model had initial learning rate of 5e-4 and gamma of 0.85.

validation accuracy = 98.5%

[kwade4]

Excluding the images with watermarks gives a slightly lower validation accuracy (~98.1%), likely because the size of the categories is reduced. In some cases (Sweet Pea) the size is reduced by 55%.

• Optimizer: AdamW
• Learning rate: 5e-4
• Learning rate scheduler: StepLR with step=2 and gamma=0.85
• Trained for 50 epochs