Is it possible to access Audio buffers to retrieve info for a VUMeter?

[ruirodrigues-eyenov]

Basically I would like to show a VUmeter with the audio volume/gain/amplitude being sent when streaming.
I could not found any way, I wonder if that is possible?
Thanks

[ThibaultBee]

Hello :)

I was kind of expecting this request.
There is a branch where you can get peak and/or rms value of each audio channels: https://github.com/ThibaultBee/StreamPack/tree/feature/vu-meter. I just rebased it. Is it what you are looking for? Are you going to use rms and/or peak value?

The only things missing for now is that when preview is running, you won't get any value cause audio capture is not running and I still don't really know how to implement this part.

Regards,
Thibault

[ruirodrigues-eyenov]

Hello :)

this is great! exactly what I wanted! thanks :)

I don't need the values while previewing but from what I can understand you can just startRecording on the AudioRecord and discard the Frames after getting the measurements.

[ThibaultBee]

Hi,
This is the issue: there aren't any audio frames when encoder is not running. It means that you can only access audio frame when StreamPack is live streaming.

[ruirodrigues-eyenov]

Yes, I know. To get audio frames you need to start recording when preview starts and discard audio frames instead of streaming them. But in my case this isn't an issue at all.

Another question on a different matter: Is it possible to get any metrics from streaming? just wondering :)

[ThibaultBee]

What metrics do you have in mind?

[ruirodrigues-eyenov]

latency, bandwidth, frame rate, bitrate. I am not sure if those are possible at all.

[ThibaultBee]

Do you use SRT or RTMP?
I guess it is quite easy to return latency, bandwidth and network bitrate with SRT, not sure about RTMP.

[ruirodrigues-eyenov]

SRT 😄

[ThibaultBee]

Starting from v3.1, an AudioProcessor will appear and it will give access to audio buffer. It is a work in progress.

[dimadesu]

Btw I have an audio meter UI working in my app for some time now, so it's possible. I remember I just asked AI to find a good spot to make it work with all my video/audio sources.
I did need to modify StreamPack code a bit to make it work, it was a small change. If anyone's interested here is the PR that adds it dimadesu/LifeStreamer#211 for the reference.

[ThibaultBee]

Ok thank for the example.
I can't put it like this in StreamPack for few reasons:

• support only 16 bits audio format whereas StreamPack supports 32 Int, Float
• processing is performed on the capture thread. I would avoid that.
• effect is not scalable (can't add other effects)

But you get the main idea and the purpose of the audio processor.

[ThibaultBee]

I pushed a PR: #278
The work is in progress but developer could add audio effect without forking the repo.
The order of the audio effects is important here.

The API might change in the future but you get the idea. Like at some point, some effects would need to be configurable.

[dimadesu]

Smart! Loving the idea!

[ThibaultBee]

Could you valide the PR? I am not working on it for the moment.
It's not yet configurable but anyone can implement a VU meter on the app side.