Are you planning to release a Faireq Version of the same.

[spprabhu]

Can you give special inputs on how to adapt it to Fairseq version.

[zomux]

Do you mean that you want to know how to train models pre-processed by fairseq?

[spprabhu]

Yes and also If I want to adapt this model to Fairseq coz its is currently using NMT and I want to move away from NMT rather use Faiseq Transformer as Base.

[spprabhu]

How will I get those KL Budgeting and other modules.Will I need to refactor the whole code.
Coz Pytorch only has LR Schedulizer.

[zomux]

The nmtlab package is closer to openmt, so it will be pretty easy changing the nmtlab part to opennmt-py

[zomux]

KL budget is super easy to implement. It can be computed based the iteration number

[zomux]

I will investigate how fairseq preprocess texts

[spprabhu]

Sure coz Fairseq is not using the sentencepiece BPE.

[spprabhu]

Also Do you have .sp datasets for other language pairs.Have you tried running your model on the same.What were their results apart from ASPEC Dataset.For eg : En-It, En-Fr etc.

[spprabhu]

It would be helpful if you already have those datasets .

[zomux]

No I don't have them, I just took the pre-processed dataset from opennmt.

[spprabhu]

Hi Raphael do you have EN-IT dataset in the .sp format

[zomux]

My WMT14 dataset is actually obtained from openNMT, the link is here https://s3.amazonaws.com/opennmt-trainingdata/wmt14-ende-sp.tar.gz . I don't think they have a EN-IT dataset. But I guess it won't be difficult to process it using sentencepiece ---> https://github.com/google/sentencepiece

[spprabhu]

Thanks for all the help.Just one question Did you look at the previous version of this model which is already present on pytorch page.Its released in Feb 2018.
Will that help me in adapting this Updated model to Fairseq.

[zomux]

Which one, Jason's model? Can you give me a link.

[zomux]

Actually if fairseq people want, I can try to build my model into their package lol. Although I'm not familiar with fairseq API.