How to change JPEG quality when using rapid image capture

[HappyStoat]

Hi there,
I am sending jpeg frames from the rapid image capture over the network and I am quite happy with the performance!
-> Thank you for that.

However, the images are very large. Is there a way to change the quality of the images afterwards with this lib?
The quality parameter probably only affects still images.

Another Question: Do I need the MMALNullSinkComponent? I removed it without any effect. I just want to make sure I'm not using anything unnecessary.

[techyian]

Hi,

This project hasn't been worked on for a few years now due to the Pi Foundation moving away from MMAL in favour of libcamera. However if you're still using this library, I'd recommend looking at the wiki for more information as I spent a long time making that area comprehensive.

What resolution are you taking images in? If you're not configuring this, it'll be defaulting to 1280x720 which will be rather large. Regarding quality, I would like to say that if you set the quality against a MMALPortConfig instance that it would have an impact on quality as it's essentially MJPEG - but I can't remember if this has an impact for rapid still capture, sorry.

Regarding the MMALNullSinkComponent, this is necessary for calculating exposure compensation, please see here for info.

Thanks,

Ian

[HappyStoat]

Wow, I didn't expect to get an answer that fast for a project that hasn't been worked on for a few years. Thanks again!
Since Libcamera is probably not for C#, I am not interested in it ;) So far I am quite happy with your Lib.
This is my Code:

public async Task TakePictureFromVideoPort()
{
    try
    {
        MMALCamera cam = MMALCamera.Instance;
        MMALCameraConfig.VideoResolution = new Resolution(640, 400);
        MMALCameraConfig.VideoFramerate = new MMAL_RATIONAL_T(40, 1);

        using (var myCaptureHandler = new MyInMemoryCaptureHandler())
        using (var splitter = new MMALSplitterComponent())
        using (var imgEncoder = new MMALImageEncoder(continuousCapture: true))
        {
            cam.ConfigureCameraSettings();
            var portConfig = new MMALPortConfig(MMALEncoding.JPEG, MMALEncoding.I420, 10);

            imgEncoder.ConfigureOutputPort(portConfig, myCaptureHandler);
            cam.Camera.VideoPort.ConnectTo(splitter);
            splitter.Outputs[0].ConnectTo(imgEncoder);

            await Task.Delay(2000);

            await cam.ProcessAsync(cam.Camera.VideoPort);
        }
        cam.Cleanup();
    }
    catch (Exception e)
    {
        Console.WriteLine("Error in Camera.TakePictureFromVideoPort():  " + e.Message);
    }
}

public class MyInMemoryCaptureHandler : InMemoryCaptureHandler
{
    public override void Process(ImageContext context)
    {
        try
        {
            base.Process(context);

            if (context.Eos)
            {
                byte[] data = base.WorkingData.ToArray();
                this.WorkingData.Clear();
                Net.SendJPEG(data);

                // This works for lowering the quality, but it is far too slow (2fps) and the size is not less^^
                //using (MemoryStream stream = new MemoryStream(data))
                //{
                //    Image image = Image.Load(data);
                //    image.SaveAsJpeg(stream, new JpegEncoder() { Quality = 50 });
                //    Net.SendJPEG(stream.ToArray());
                //    image.Dispose();
                //}

                //Console.WriteLine("Framesize: " + data.Length);
            }
        }
        catch (Exception e)
        {
            Console.WriteLine("Error in Camera.Process() : " + e.Message);
        }
    }
}

[techyian]

No problem, I've enjoyed my time supporting this library over the years but as to its future, I'm unsure due to my previous statement.

If you're looking to further reduce the overall size of your images, you could potentially look to using a different pixel format which has a lower bit depth per pixel. At the moment you're using I420 (YUV420) as defined against your MMALPortConfig which is a 16 bit format; what you could do is look at using something like YV12 or NV12 which are 12 bit, or YUV10COL which is a 10 bit format - I cannot guarantee that these will work, but apart from lowering your resolution more I'm unsure what more you can do to reduce the overall size of your images when in rapid capture mode unfortunately.

A list of the pixel formats available in MMALSharp can be found here and search for EncodingType.PixelFormat - have a play with some and see what they do.

Out of interest, what filesize are the images when you store them?

[HappyStoat]

To use only 10bit for colors is actually a really nice idea. This reminds me of the old days when I set my computer's screen colors to 8bit so that the games ran more smoothly ;)
But unfortunately that doesn't seem to work. Because YUV10COL is strangely not available under MMALEncoding. YV12 and NV12 is there but it makes no difference. It probably doesn't matter what value I enter as PixelDecoding, the size doesn't change.

Right now I have 70kbyte per image. With 40fps I have about 22mbit/s at the moment. This is too much. I could also just create an h.264 stream using v4l2rtspserver, but if the connection is bad, the rtsp player always stops.

If I just send the JPEGs over with your lib, that's no problem, and the latency is much better (100ms)

[techyian]

Ah I see, I did have my suspicions the pixel format change may not work, the native library can seemingly accept a change for certain operations but not actually honour this. Can you not lower the requested FPS to a lower value to achieve your goal?