Missing Caption Files For YouCook2 Dataset

[fzohra]

Hi,

I am unable to evaluate SwinBert on the YouCook2 dataset due to missing coco captions files. The evaluation expects the following files:

validation.caption_coco_format.json
testing.caption_coco_format.json
training.caption_coco_format.json

However, the annotations downloaded for YouCook2 using the provided scripts does not contain these files.

I'm training/evaluating with the default configurations provided in the source code and README so I expect I should be able to run the code without modifying the evaluation logic for this dataset. Please advise on how to train/evaluate this dataset. Thanks!

[AdrienneDeganutti]

Hi @fzohra,

I was able to generate the _.caption_coco_format.json files using the tsv_preproc_youcook2.py pre-processing script.

I have found that this will not generate a file for the testing split as these are processed from the dataset's ground truth annotations: youcookii_annotations_trainval.json which only include training and validation splits.

Hope this helps!