Implement digital binning and ROI selection

[jwlodek]

In some cases it is desirable to do binning/ROI at the driver level rather than plugins. UVC cameras cannot do this in hardware, but it would be nice to do in software at least.

[ericonr]

As discussed in the collaboration meeting, this was intended to allow image properties/metadata to be queried without requiring a frame to be forwarded through plugins or inspecting the plugin chain.

---

Now, for my opinion about this, I believe the issue should be moved to ADCore, because it's relevant for all drivers, and I don't think it makes sense to implement individually per-driver. Furthermore, areaDetector already implements its abstraction for image processing as plugins, and I think it would be a general loss to move everything to the inside of a driver, including in resource scalability.

I talked to @henriquesimoes about this, and he suggested using "meta-frames", which are used to propagate setting changes, without requiring actual data. Implementing this would require a thorough design, but I believe it would be a good path forward.

Some initial design considerations, if we used this approach:

• it seems easier to start with only plugins forwarding their information, since plugin development is more centralized than drivers, and not all drivers allow changing their output format. Detector native ROI/binning and data type selection are important features, but would probably require per-driver implementations (though if implemented as part of Move more functionality into the ADDriver base class ADCore#251 it might be possible to simply migrate drivers to the new functions and get the functionality for free)
• one way of taking advantage of the current plugin implementations would be to include some sort of frame data in the meta-frames (either zeroed-out or based on the last processed frame) so that plugins don't need a separate path to calculate metadata for their output frames; the meta-frames simply include an additional flag. The flag is needed because plugins like StdArrays/PVAPlugin/File need to ignore these frames for their side effects (publishing PVs/writing to files), while others, such as ROI/Stats/Convert, can probably simply process them normally (though it would be necessary to make sure the metaflag is copied)
• plugins should probably have to indicate that they support metadata frames (so these frames are either not queued or dropped from the queue when unsupported), to make sure they are treated correctly
• after booting an IOC, a single acquisition/meta-frame from the detector driver is probably needed as a starting point for the plugins to do their thing. This proposal would simply limit the need for warm-up acquisitions for each setting change
• meta-frames probably shouldn't be dropped when a queue is full

[jwlodek]

What about this: currently, for plugins we use endProcessCallbacks instead of doCallbacksGenericPointer directly, which allows us to add functionality to the superclass. What if we introduce something similar at the driver level, and then we implement Digital Binning/ROI Pvs that manipulate the array in that function before doCallbacksGenericPointer. This would be simpler I think than walking dimensions all the way through the plugin chain.

Walking the plugin chain would work too, but it wouldn't be enough to do just at IOC bootup, you'd need to tie a callback to any parameters that modify image dimensions to do this. So ROI/binning/bit depth/color mode, both in the driver and the plugins, and I expect that will get pretty complex.

[henriquesimoes]

What about this: currently, for plugins we use endProcessCallbacks instead of doCallbacksGenericPointer directly, which allows us to add functionality to the superclass. What if we introduce something similar at the driver level, and then we implement Digital Binning/ROI Pvs that manipulate the array in that function before doCallbacksGenericPointer.

I don't like this because it assumes we don't want to apply any other operation before Digital Binning and ROI. If we consider a detector which contains a bad pixels, we would like to mask them before cutting the image. The same goes for flatfield or any image correction. In other words, this approach isn't general enough to solve the root problem here in my opinion. It defeats the purpose of the plugin infrastructure by replicating code and logic, which doesn't smell right for me.

Walking the plugin chain would work too, but it wouldn't be enough to do just at IOC bootup, you'd need to tie a callback to any parameters that modify image dimensions to do this. So ROI/binning/bit depth/color mode, both in the driver and the plugins

Yes, it'd need to be tied to parameter changes indeed.

and I expect that will get pretty complex.

Another approach would be to generate a fake frame under demand, so that plugins update themselves before the actual acquisition. You can think of that as a "quick warm-up frame". What gets annoying with this approach is that each driver would probably require a custom code to generate such frames, as the way parameters affect an image for specific detectors is not standardized, AFAIK.

We would probably want to evolve it so that fake frames do not update statistics values, don't get into StdArray's and PVA's waveforms, etc. That begins to get closer to the idea of a metadata frame, which contains a bit to distinguish it from real frames.

[ericonr]

Further discussion suggested using more lazy methods, such as HDF5 lazy open, in order to avoid the need for warm-up frames.