Any suggestions of how to use with Speech LLMs

[mush42]

Hello

Thanks for making the code available.

Any recommendations regarding using this codec with audio LLMs?

Best
Musharraf

[yzGuu830]

Hi @mush42, similar to mainstream codecs like SoundStream and EnCodec, ESC can also function as an acoustic tokenizer, generating codes from multiple streams. I suggest drawing inspiration from recent audio generative models (AudioLM, VALL-E, MusicGen) to effectively model the multi-level discrete codes.