The world model seems useless! #32
Description
Hello @liyingyanUCAS
Thank you for sharing your code and paper. I’m currently trying to reproduce the ablation study comparing LAW with and without the World Model (WM), and I ran into results that I don’t fully understand. I’d really appreciate your guidance.
Experiment 1 (WM removed):
I used the same training setup as in the original LAW configuration, but disabled the World Model by removing self.wm. With this change, the L2 error is exactly the same as the full model with WM. However, the collision rate is unstable and consistently worse than what is reported in the paper.
Experiment 2 (learning rate adjusted):
Starting from Experiment 1, I made one additional change: I increased the default learning rate from 5e-5 to 7e-5. My reasoning was that removing the WM loss reduces the overall gradient magnitude, which might effectively shrink the update step during backpropagation. After this adjustment, the results are even slightly better than those reported in the original paper.
This seems inconsistent with the paper’s conclusions. Could you help me understand why removing WM (with a small LR adjustment) might yield comparable or even better performance?
In my understanding, the World Model is a major contribution of this work, so I expected a clearer performance drop without it. Any explanation or tips on what I might be missing would be very helpful.
Thank you again for your time.
Best regards