Failed to reproduce the results on Mutag dataset

[ciortanmadalina]

Hello,

I retrained the downstream model following the documentation (e.g. python mutag_gnn.py) and also the Refine model using the recommended: python refine_train.py --dataset mutag --hid 100 --epoch 100 --ratio 0.4 --lr 1e-3 --batch_size 64

However, at the end of the training I get an AUC of 0.82 which is quite far from the reported 0.955. Am I missing something?

2022-07-01 18:01:23,919 - refine_train.py[line:163] - INFO: ACC:[[0.86 0.81 0.78 0.78 0.77 0.79 0.77 0.8 0.86 1. ]] ACC-AUC: 0.822 Mean P: 0.099 2022-07-01 18:01:44,651 - refine_train.py[line:158] - INFO: Epoch: 98, LR: 0.00001, Ratio: 0.40, Train Loss: -163.027, ValLoss: -1.704

[Wuyxin]

Hi,

Could you try

1. run evaluate.py after the training
2. train a GNN model using a different seed and then run the cmd again