Validation data and test data are overlapped for tahoe-100m holdout experiment #226
Description
Hi dear authors,
Thank you so much for this wonderful work! In the toml file you released https://huggingface.co/datasets/arcinstitute/Tahoe-Preprint/blob/main/generalization.toml, it seems that there are a categories that appeared both in validation and test data: "[('Bestatin (hydrochloride)', 0.05, 'uM')]", "[('Bestatin (hydrochloride)', 0.5, 'uM')]". Could you please help to clarify which one should be really used as the test data? And whether this overlap would impact model performance, as the model is seeing some of the test data in its validation set? Thanks for the help!
Best regards,
Xinyu