Skip to content

Obtaining results provided in the original paper #16

@enekoatxa

Description

@enekoatxa

Hi, I have been using the code provided in this repository for some time. The code is brilliant, well structured and easy to understand. However, I am quite far from achieving the results that the original authors of the paper provide.

In the original paper, the results on LaFAN1 dataset are as follows:
L2Q: 0.17, 0.42, 0.69, 0.94 for transition lengths 5, 15, 30 and 45.
L2P: 0.23, 0.65, 0..28, 2.24 for transition lengths 5, 15, 30 and 45.

However, in my attempts, using the hyperparameters provided in the config file (I also tried changing some of them, but i got worse results), I get L2Q values 4.55, 4.61, 4.61 and L2P values 1.79, 2.43, 3.14 for transition lengths 5, 15 and 30.

These values are extremely different from the original values provided. Is my evaluation code wrong? I mean, is it possible to get the values near to the ones in the original article?

Thanks in advance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions