About using Lora

[msra-jqxu]

Hi, thanks for open-sourcing this excellent code repository!

I wonder does the code include a part that directly enables LoRa to fine-tune the policy model? Or do I need to manually use PEFT to warp the policy model?

Thank you!

[hejujie]

We haven't used its LoRA training feature ourselves. You might need to seek help from the official verl repository https://github.com/volcengine/verl/ ; modifications there should be relatively quick to adapt here.