Inquiry About Caption Data Collection for Reproducing the Model

[tskim9439]

First of all, thank you for your outstanding work on DeSTA2! Your contributions to this field are truly impressive, and I appreciate the effort you have put into this project.

I am currently trying to reproduce the model by training it from scratch and am attempting to collect the caption data you have used. Upon checking PromptTTS, it appears that the synthesized version of PromptTTS was used in your work. However, I could not find this dataset publicly available as an open-source resource.

Could you kindly clarify whether the PromptTTS database you used was newly generated following the PromptTTS paper, or if there is any publicly available source from which I could collect the dataset? Any guidance on this would be greatly appreciated.

[kehanlu]

Hello @tskim9439,
Thank you for your kind words! 😄

Yes, we use the synthesized version of PromptTTS in our work. This dataset is not publicly available, so we use Azure TTS with tags (such as speaker, speaking speed, pitch, etc.) to reproduce the audio ourselves.

From my experience, training without PromptTTS does not affect the performance on Dynamic-SUPERB much.

Let me know if you need any more details 👍