Using offical evaluation scripts and offical weights, success rate always is zero and videos show the robot execute actions which are very strange.

[786739982]

Hello, thank you for open-sourcing this work !
I reproduced your work with great interest, including downloading the weights from HuggingFace and running the evaluation scripts. However, I consistently observe a success rate of 0.

Below is my reproduction procedure:

1. download the offical weights from HuggingFace “Timsty/mixture_of_horizons”
   /root/ckpt/mixture_of_horizons/
   ├── README.md
   ├── norm_stats.json
   ├── pi05_moh_libero_3to30stride3
   │ └── model.pt
   ├── pi0_moh_libero_3to30stride3
   │ └── model.pt
   └── pi0reg_moh_libero_3to30stride3
   └── model.pt

          4 directories, 5 files 
   

2. modify the evaluation scripts (as followed image) and "sys.path": (In the first time, I run the test command manually in your terminal without piping to a log file.)

3. bash ./scripts/eval_on_libero.sh
   Here is the output log from my terminal after running the script:

I'd like to ask for your help in identifying where the problem lies. I can also provide recorded videos if needed. The videos show that the robot performs similarly strange operations regardless of the scenario.

Thank you very much for paying attention to and replying to this issue.

[Timsty1]

Hi, thanks for your interest in our work. Based on the information you provided, I suggest checking the following potential solutions:

i) Verify that the checkpoint parameters are loading correctly.

ii) Make sure you are using the norm_stats.json we provided on HuggingFace.

iii) I noticed you are loading four environments simultaneously on CUDA:0. I suggest you run just one environment first, as I am not sure about the stability of running multiple environments on a single GPU.

iv) Please ensure you are using the Libero environment provided in packages/libero, which allows for reproducing the experiment results.

If the issue persists, please feel free to provide more details.