How to reproduce zero-shot performance?

[XULU42]

NDT2 is a great job for neural decoding! With the detail exp configs, it's convenient to reproduce most result showed in paper. But I got stucked in reproducing the zero-shot performance(subplot 1 in Figure 13)

Here was my attempt to do this:
Using dataset.eval_ratio=1 to guarantee zero-shot，and use all Indy sessions to pretrain and supervised finetuning. My max eval R^2 is 0.168，which has a great gap to 0.7.

Could you please help me to figure this out? Specific exp config, wandb run id or reproduce steps would be a great help! Thanks.

[joel99]

Sorry for the delay. I'm not immediately sure what might be happening, but let me try to trace down the configs for you.
https://github.com/joel99/context_general_bci/blob/main/scripts/figures/intra_decode.py#L250 shows the line that generated the zero-shot datapoint. This corresponds specifically to the following run:

context_general_bci/scripts/figures/intra_decode.py

Line 29 in 9c17b37

ROBUST_RUN = 'session_cross_noctx-wc24ulkl'

, so you can find the wandb run here: https://wandb.ai/joelye9/context_general_bci/runs/wc24ulkl/overview

I hope this helps with reproduction. Please ping my email if I'm not responding here.