Skip to content

Conversation

@yuekaizhang
Copy link

This PR introduces a recipe using veRL to conduct RL training experiments on cosyvoice2GitHub Repo stars llm models.

Specifically, we conducted GRPO experiments and got below results:

Model Seed-TTS test_zh CER(%) ⬇️ Cosyvoice3 zero_shot_zh CER (%)⬇️ Comment
SFT (initialized from Qwen2-0.5B-Instruct) 1.81 % 4.83% See PR #1887
GRPO (this project, trained on AIShell-3) 1.06 % 4.03% See here

Features:

  1. Using Pytriton based Sensevoice ASR sever to achieve fast reward calculation.
  2. Using phoneme error rate (PER) as the reward metrics.
  3. Support both pretrained cosyvoice2 llm and custom sft version of cosyvoice2

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a new project, CosyVoice-TTS-GRPO, to the list of projects using verl in the main README.md. The change is straightforward, but I've pointed out a minor formatting inconsistency that should be addressed to maintain consistency with the rest of the document.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@eric-haibin-lin
Copy link
Collaborator

nice! do you plan to contribute the full recipe, or just the readme? BTW please resolve conflict with main branch. thanks!

@yuekaizhang
Copy link
Author

nice! do you plan to contribute the full recipe, or just the readme? BTW please resolve conflict with main branch. thanks!

I'd love to contribute the recipe here. Let me update the recipe into this PR.

@CLAassistant
Copy link

CLAassistant commented Jul 25, 2025

CLA assistant check
All committers have signed the CLA.

@yuekaizhang yuekaizhang changed the title [doc] Add Cosyvoice TTS GRPO training project based on veRL. [new recipe] Add Cosyvoice TTS GRPO training recipe based on veRL. Jul 25, 2025
@yuekaizhang
Copy link
Author

@eric-haibin-lin Updated. Would you mind checking it again? Many thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants