Skip to content

Running pretrained models on Other datasets #15

@Cosmopal

Description

@Cosmopal

Hi,
You some great work here! Is there a way to run your pre trained models on another dataset? I tried just replacing the train.document and train.summary files with other data, but the final-test-output-convs2s-checkpoint-best.pt results were totally unrelated, and repeated. It seems it is still trying to map my custom values to previously seen titles??

Here's what I did:
I was not sure which file is the data read from for the test, so I replaced train.document, test.document, valid.document, validation.document all with the texts (same in each) and train.summary, test.summary, valid.summary, validation.summary with the titles. (same in each). I copied he dict.document.txt and dict.summary.txt from your original tar.

Then I ran

cd XSum-ConvS2S
python generate.py ./convs2s-emnlp18/data-convs2s --path ./convs2s-emnlp18/checkpoints-convs2s/checkpoint-best.pt --batch-size 1 --beam 10 --replace-unk --source-lang document --target-lang summary > test-output-convs2s-checkpoint-best.pt
cd ..
python scripts/extract-hypothesis-fairseq.py -o XSum-ConvS2S/test-output-convs2s-checkpoint-best.pt -f final-test-output-convs2s-checkpoint-best.pt

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions