How to play audio data decoded from mp4 with NAudio or waveOut API?

[iocmet]

Currently i copying unmanaged data[0] byte* array to managed byte[] with following code:

byte[] managed = new byte[resampledFrame.linesize[0]];
for (int i = 0; i < resampledFrame.linesize[0]; i++) {
managed[i] = resampledFrame.data[0][(uint) i];
}

It almost works, i can hear source sound but with lot of noise, what im doing wrong?

I using this example https://github.com/IOL0ol1/EmguFFmpeg/blob/dev/example/DemuxingDecoding.cs

And i can't record result that im getting because screen recorders ignores it

[IOL0ol1]

https://github.com/IOL0ol1/EmguFFmpeg/blob/dev/example/Other/Mp4ToPcm.cs#L23

[IOL0ol1]

https://github.com/IOL0ol1/EmguFFmpeg/blob/dev/example/Other/Mp4ToPcm.cs#L23

Two outputs are demonstrated, one that converts MP4 to PCM audio data and one that encodes as WAV.

[IOL0ol1]

Currently i copying unmanaged data[0] byte* array to managed byte[] with following code:

byte[] managed = new byte[resampledFrame.linesize[0]]; for (int i = 0; i < resampledFrame.linesize[0]; i++) { managed[i] = resampledFrame.data[0][(uint) i]; }

It almost works, i can hear source sound but with lot of noise, what im doing wrong?

I using this example https://github.com/IOL0ol1/EmguFFmpeg/blob/dev/example/DemuxingDecoding.cs

And i can't record result that im getting because screen recorders ignores it

For planar audio, each channel has a separate data pointer, and linesize[0] contains the size of each channel buffer.
like LLLLLLLL.... RRRRRR.....
For packed audio, there is just one data pointer, and linesize[0] contains the total size of the buffer for all channels.
like LRLRLRLRLRLRLRLRLRLR.....
You need packed audio AVFrame, So I used SampleConverter to convert to S16 format instead of S16P (Planar)

[iocmet]

Thanks, i will try it next day and close issue

[iocmet]

Currently i copying unmanaged data[0] byte* array to managed byte[] with following code:
byte[] managed = new byte[resampledFrame.linesize[0]]; for (int i = 0; i < resampledFrame.linesize[0]; i++) { managed[i] = resampledFrame.data[0][(uint) i]; }
It almost works, i can hear source sound but with lot of noise, what im doing wrong?
I using this example https://github.com/IOL0ol1/EmguFFmpeg/blob/dev/example/DemuxingDecoding.cs
And i can't record result that im getting because screen recorders ignores it

For planar audio, each channel has a separate data pointer, and linesize[0] contains the size of each channel buffer. like LLLLLLLL.... RRRRRR..... For packed audio, there is just one data pointer, and linesize[0] contains the total size of the buffer for all channels. like LRLRLRLRLRLRLRLRLRLR..... You need packed audio AVFrame, So I used SampleConverter to convert to S16 format instead of S16P (Planar)

But how i was supposed to know that? There is nothing in ffmpeg documentation or anywhere else about it

[iocmet]

And you know why i can't record result with screen recorders? Currently im using NAudio, firstly used waveOut

[IOL0ol1]

Currently i copying unmanaged data[0] byte* array to managed byte[] with following code:
byte[] managed = new byte[resampledFrame.linesize[0]]; for (int i = 0; i < resampledFrame.linesize[0]; i++) { managed[i] = resampledFrame.data[0][(uint) i]; }
It almost works, i can hear source sound but with lot of noise, what im doing wrong?
I using this example https://github.com/IOL0ol1/EmguFFmpeg/blob/dev/example/DemuxingDecoding.cs
And i can't record result that im getting because screen recorders ignores it

For planar audio, each channel has a separate data pointer, and linesize[0] contains the size of each channel buffer. like LLLLLLLL.... RRRRRR..... For packed audio, there is just one data pointer, and linesize[0] contains the total size of the buffer for all channels. like LRLRLRLRLRLRLRLRLRLR..... You need packed audio AVFrame, So I used SampleConverter to convert to S16 format instead of S16P (Planar)

But how i was supposed to know that? There is nothing in ffmpeg documentation or anywhere else about it

https://www.ffmpeg.org/doxygen/trunk/structAVFrame.html#afca04d808393822625e09b5ba91c6756

[iocmet]

extended_data? im using only data

[iocmet]

Im getting same result when using SWR and "FFmpegSharp.FFmpegException: "FFmpeg error [-22] Invalid argument"" on
FFmpegSharp.SampleConverter.d__14.MoveNext
MediaFrame.cs:line 54 AllocateBuffer
With SampleConverter

[IOL0ol1]

extended_data? im using only data

The data size is not enough to represent more than 8 channels of planar audio, and extended_data is required。
In other cases, they are consistent.
These are the experiences of ffmpeg. Please go to learn about ffmpeg.

[IOL0ol1]

Im getting same result when using SWR and "FFmpegSharp.FFmpegException: "FFmpeg error [-22] Invalid argument"" on FFmpegSharp.SampleConverter.d__14.MoveNext MediaFrame.cs:line 54 AllocateBuffer With SampleConverter

Can you provide your minimum executable code?

[iocmet]

            using (var fmtctx = MediaDemuxer.Open(url)) {
                var videoStream = fmtctx.FirstOrDefault(_ => _.CodecparRef.codec_type == AVMediaType.AVMEDIA_TYPE_VIDEO);
                var audioStream = fmtctx.FirstOrDefault(_ => _.CodecparRef.codec_type == AVMediaType.AVMEDIA_TYPE_AUDIO);

                using (var videoDecCtx = MediaDecoder.CreateDecoder(videoStream.CodecparRef))
                using (var audioDecCtx = MediaDecoder.CreateDecoder(audioStream.CodecparRef))
                using (var converter = SampleConverter.Create(audioDecCtx.ChLayout, audioDecCtx.SampleRate, AVSampleFormat.AV_SAMPLE_FMT_S16, audioDecCtx.FrameSize))
                // Use align = 1 AVFrame instead uint8_t *video_dst_data, less and safer code.

                using (var videoDstData = MediaFrame.CreateVideoFrame(videoDecCtx.Width, videoDecCtx.Height, videoDecCtx.PixFmt, 1))
                using (var pkt = new MediaPacket())
                using (var frame = new MediaFrame()) {
                    //currentVideoWidth = videoDecCtx.Width;
                    //currentVideoHeight = videoDecCtx.Height;

                    foreach (var packet in fmtctx.ReadPackets(pkt)) {
                        if (packet.StreamIndex == videoStream.Index) {
                            foreach (var f in videoDecCtx.DecodePacket(packet, frame)) {
                                //var cFrame = vfc.Convert(f.Ref);
                                //_currentFrameData = new Frame(cFrame.linesize, cFrame.data);

                                Thread.Sleep(15);
                            }
                        }
                        if (packet.StreamIndex == audioStream.Index) {
                            foreach (var f in audioDecCtx.DecodePacket(packet, frame)) {
                                AVFrame rframe = *ffmpeg.av_frame_alloc();

                                rframe.sample_rate = frame.Ref.sample_rate;
                                rframe.channel_layout = frame.Ref.channel_layout;
                                rframe.channels = frame.Ref.channels;
                                rframe.format = frame.Ref.format;

                                SwrContext* ctx;
                                ctx = ffmpeg.swr_alloc();
                                ffmpeg.swr_config_frame(ctx, &rframe, frame.pFrame);
                                ffmpeg.swr_init(ctx);

                                //ffmpeg.av_opt_set_channel_layout(ctx, "in_channel_layout", (long)ffmpeg.AV_CH_LAYOUT_STEREO, 0);
                                //ffmpeg.av_opt_set_channel_layout(ctx, "out_channel_layout", (long)ffmpeg.AV_CH_LAYOUT_MONO, 0);
                                ffmpeg.av_opt_set_int(ctx, "in_sample_rate", f.SampleRate, 0);
                                ffmpeg.av_opt_set_int(ctx, "out_sample_rate", f.SampleRate, 0);
                                //ffmpeg.av_opt_set_sample_fmt(ctx, "in_sample_fmt", AVSampleFormat.AV_SAMPLE_FMT_S16P, 0);
                                //ffmpeg.av_opt_set_sample_fmt(ctx, "out_sample_fmt", AVSampleFormat.AV_SAMPLE_FMT_S16, 0);

                                ffmpeg.swr_convert_frame(ctx, &rframe, frame.pFrame);

                                int unpadded_linesize = rframe.nb_samples * ffmpeg.av_get_bytes_per_sample((AVSampleFormat)rframe.format);

                                    byte[] managed = new byte[unpadded_linesize];

                                    for (int t = 0; t < unpadded_linesize; t++) {
                                        managed[t] = (byte)rframe.extended_data[0][(uint) t];
                                    }

                                    WaveOut waveout = new WaveOut();

                                    var bufferedWaveProvider = new BufferedWaveProvider(WaveFormat.CreateIeeeFloatWaveFormat(rframe.sample_rate, rframe.channels));
                                    bufferedWaveProvider.BufferLength = unpadded_linesize * 16; //16 is pretty good
                                    bufferedWaveProvider.DiscardOnBufferOverflow = true;
                                    
                                    bufferedWaveProvider.AddSamples(managed, 0, unpadded_linesize);

                                    waveout.Init(bufferedWaveProvider);
                                    waveout.Play();
                            }
                        }
                    }
                }
            }

It requires NAudio to run, put "string url = "path or url to video"" before this code, run and you will get audio with lot of noise
You can replace raw swr with SampleConverter and get ffmpeg -22 error, my ffmpeg version is 5.1.2

[iocmet]

How i can fix it?