Interference between calibration and hflip vflip

[dgalland]

Hello,
I used your calibration script without hflip and vflip for the V2 successfully (except for a small blue band at the bottom).
Now I try with the V1, I spent a lot of time understanding the script (a good numpy exercise!) And I found an interference with hflip and vflip, the result is correct only if hflip = True and vflip = True . To put it in evidence here is a comparison of the 5 iterations (complete image, channels B G R). Do you have an idea ?
Note the other two cases are also bad
hflip vflip false false There seems to be a shift to the right

hflip vflip true true

[dgalland]

I think of course that has some things to do with the bayer pattern.
If you look at another project
link
we see that it padd differently according to hflip and vfip and also for filling the table.
Now I am a little lost to change the code !

[dgalland]

Yes, that was it. For V1 the script only works if hflip and vflip = True. For other values it is necessary to reorient the table as in the cpixip project. I will study now how it works for the V2
if bayer == 1 : #False False
lens_shading[0,:,:] = lens_shading[0,::-1,::-1]
lens_shading[1,:,:] = lens_shading[1,::-1,::-1]
lens_shading[2,:,:] = lens_shading[2,::-1,::-1]
lens_shading[3,:,:] = lens_shading[3,::-1,::-1]
elif bayer == 2 : #True False
lens_shading[0,:,:] = lens_shading[0,::-1,:]
lens_shading[1,:,:] = lens_shading[1,::-1,:]
lens_shading[2,:,:] = lens_shading[2,::-1,:]
lens_shading[3,:,:] = lens_shading[3,::-1,:]
elif bayer == 0 : #False True
lens_shading[0,:,:] = lens_shading[0,:,::-1]
lens_shading[1,:,:] = lens_shading[1,:,::-1]
lens_shading[2,:,:] = lens_shading[2,:,::-1]
lens_shading[3,:,:] = lens_shading[3,:,::-1]

[rwb27]

Hi @dgalland thanks for posting this - you are indeed correct, I'm afraid as I don't tend to use the v1 camera, and haven't touched the flip settings on my v2, that I never added support for flipped images. I think there are fields in the underlying MMAL structure that loads in the lens shading table that tell the firmware whether the lens shading table is intended for flipped or non-flipped images, but I must confess I've not yet figured out which way round they are supposed to go, and so they are probably just set to whatever the v2 camera does by default.

Lastly, there are probably two issues - firstly the obvious one (the correction image being flipped) and secondly a slight shift of up to 64 pixels that results from the way the lens shading table is upscaled on the camera. Flipping the image with numpy as you're doing will fix the first one, it may or may not fix the second one - but if it's working for you, I wouldn't worry about it; it would be a subtle issue at worst!

If you fancied figuring out the flipping parameters buried within my fork of picamera, I'd be very happy - but no worries if your workaround above is sufficent that you are happy leaving things as-is.

All the best,

Richard

[dgalland]

Thank you for your answer. I'm looking now for the V2 camera. Your script only works if vflip and hflip are False, the opposite of v1 ! It seems that for V1 and V2 the correction to be reversed!
After reorienting the table, I get a good result in all four cases.
However, I already noted in the two cases where vflip = False the presence of a horizontal bar at the bottom. Does this have something to do with the padding (cpixip pad differently depending on the orientation)
I also have another question about calibration interference with white balance. You do the calibration on captures after freezing the white balance (others do the calibration on a bayer capture ). Do you think that means that the calibration is only valid for this fixed balance and that we have to remember it for subsequents captures.
Regards, Dominique
Images for False False (original script) and True False (modified script)

[rwb27]

Hello,
I strongly suspect the bar at the bottom is due to something padding related, but I’ve not yet figured out exactly what... from memory I pad the image using the edge values, which will distort the correction slightly at the bottom and right, but it should only be slightly...

I initially did my correction using raw values, but switched to the closed loop version when I couldn’t get it to work properly. It turned out the issue was black point, because the black point on the v2 camera module is 64 (I had assumed that zero meant zero, because the sensor has built in dark count correction). Taking that into account means the open loop calibration based on raw images works nicely, and that is now what I’m using in th new version of the microscope software, where there is a recalibrate plugin.

That code should work without the rest of the software, though I notice it seems to be missing a call to freeze_camera_settings before the correction is applied.

R

[dgalland]

Complicated things about bayer pattern !
So I try your two scripts old and new with V1 and V2 and also hflip and vflip trying to find a coherent explanation

For the V2:

Old script from RGB capture
In theory you have to reorient the table by canceling VFlip and Hflip
Conclusion: The table is R G G B and applies to the sensor without flip
Question: The four tables are different or the same ???
Response the hflip = vflip = False calibration works with other flips !
Conclusion Always calibrate with hflip = vflip = False
My mistake was to do the calibration in each case

New script from BayerArray
Notes on the script:
with your pattern you extract B G G R !
but you correct after calibration !!
return lens_shading_table[::-1,:,:]
It would be better to do as in array.py:

BAYER_OFFSETS = {
0: [(0, 0), (1, 0), (0, 1), (1, 1)], #True True
1: [(1, 0), (0, 0), (1, 1), (0, 1)], #True False
2: [(1, 1), (0, 1), (1, 0), (0, 0)], #False False
3: [(0, 1), (1, 1), (0, 0), (1, 0)], #False True
}

def channels_from_bayer_array(bayer_array, bayer_order):
"""Given the 'array' from a PiBayerArray, return the 4 channels."""
bayer_pattern = BAYER_OFFSETS[bayer_order]

where bayer_order comes from the array
bayer_order = a.header.bayer_order

After this the same remark, the same table must work in all four flips cases.

For the V1 I will try again.
There is an unexplained difference with v2 because your script works if Hflip = vflip = True

After that I have questions about the interference between the calibration and the white balance (red and blue gain).
First question for the new script it does not seem necessary to fix the settings of gains since the capture bayer does not do any treatment?

Regards,
Dominique