checkerboard error after model INT8 PTQ #275
Description
Hi, thank you for your project.
I am trying to deploy LightStereo-S. I modified the model to output both prob and spx_pred, and I run disparity_regression and context_upsample on the CPU as part of the model post-processing. However, after INT8 PTQ, the results appear to have checkerboard error. Do you have any thoughts on this issue? Thank you.
left: float model result
right: quant model result