Difficulty replicating results of the paper

[anthonycou]

I am training on the DailyDialog dataset with the same hyperparameters as described in the paper. I cannot seem to get the model to perform to the standards described in the paper, specifically the BLEU score for the testing data is half the reported value. In addition, looking at the generated text for the testing dataset shows that the model is generating responses that have little to do with the actual context. Are there any solutions to this?

[guxd]

Can you provide the training curves for us to identify the problem?

[anthonycou]

Here are the BLEU scores and valid losses over the training iterations. I also retrained the model on multiWoz, and the produced dialogue improved but the BLEU score is far below what is expected. Perhaps there is an error in the generation of BLEU score?

[guxd]

Did you use the default BLEU script that we have provided in the repo?
Below are our training curves and the test BLUEs.

It seems that something gets wrong with the generation.