Script to generate all features

[cjiang2]

Hello,

Again thank you for providing the ResNet50 feature files last time. However I'd actually be more helpful if you can provide the missing script that you used to generate all those feature files. That is, the script that you used to generate all:

• resnet
• vgg19
• resnet50_keras_feature_no_sub_mean
• vgg16_keras_feature_no_sub_mean
  and so on...

One of the keys for training you mentioned during your paper is the way you augment the frames with synthetic imagenet frames. I think this part of the code is also included in your missing script. (This is also the part that interests me the most)

Thank you again in advance!

[cjiang2]

Had the email reply from author and will leave the issue close for now.

[cjiang2]

Update

I worked on my own feature extraction script with tensorflow and ran 5-run experiments specifically with the ResNet50 features. The extracted features provided by the author is generally better:

Result from my TF feature extraction (ResNet50):
------------------------------
BLEU-1: average score: 0.389 +- 0.004, best: 0.405
BLEU-2: average score: 0.273 +- 0.004, best: 0.288
BLEU-3: average score: 0.212 +- 0.004, best: 0.226
BLEU-4: average score: 0.178 +- 0.004, best: 0.188
METEOR: average score: 0.189 +- 0.002, best: 0.196
ROUGE_L: average score: 0.391 +- 0.003, best: 0.405
CIDEr: average score: 1.529 +- 0.024, best: 1.625

Result from author's feature extraction (ResNet50):
------------------------------
BLEU-1:average score: 0.403 +- 0.001, best: 0.408
BLEU-2:average score: 0.292 +- 0.001, best: 0.295
BLEU-3:average score: 0.231 +- 0.001, best: 0.235
BLEU-4: average score: 0.196 +- 0.001, best: 0.200
METEOR: average score: 0.197 +- 0.001, best: 0.200
ROUGE_L: average score: 0.401 +- 0.001, best: 0.405
CIDEr: average score: 1.635 +- 0.012, best: 1.677

In addition, I'm using the synthetic ImageNet frame from the Caffe repo. Features are extracted the same way described in the paper. All hyperparameters stay the same. There's no additional changes in the model script.

Using my own feature extraction script results mostly in average 2% lower than author's ResNet50 features. Using features extracted from PyTorch has even worse results compared to features extracted from Tensorflow, given my initial looks of them. Using InceptionV3 has even worse results compared to ResNet50 features.

Again, would you minding uploading your complete script to extract the ResNet50 features including the synthetic ImageNet frame you use so that it will really help me understand what results in the difference?

Additionally, I'm providing my best pre-trained model using the author's ResNet50 feature, in case anyone needs it. I'm also more than happy to share my feature extraction script to see what's going on.

Edit: Updated the shared link for pretrained model.