About the definition of "resolution"

[ZekaiGalaxy]

Hi! I run the resize.sh to prepare the dataset, but I find that the prepared dataset does not follow the normal definition of resolution.

For example, in "text_data/infograph_256", many of the pictures are way bigger than 256x256, and the texts are hard to recognize, there is no way for VAEs to reconstruct such data and get scores.

Can you provide the resized subset (e.g. on huggingface)? I wonder if I miss some steps.

[wjf5203]

Hi @ZekaiGalaxy, thank you for your attention, and apologies for the delayed response.

1. Regarding resizing, please refer to the definition in our paper. We treat the resize operation as the upper bound for different resolution tokenizers. The image processing pipeline is as follows: (1) pad the image to a square, (2) resize the padded image to the target size (such as 256x256), (3) resize the image back to the padded size, and (4) crop out the padded region. After this process, the resulting image has exactly the same resolution as the input, which is beneficial for evaluating faces and text. The upper bound of image quality is determined by the resize and its inverse process. Inevitably, some information is lost during resizing, just as with any tokenizer that processes images at a fixed size. However, the information loss from resizing is minimal, so this resize setting serves as the upper bound for that resolution.

2. For different resolutions, we set the minimum instance scale that can be evaluated, as described in Appendix A.2 "Difficulty Rating" of our arXiv paper. This section defines the scale range of instances that need to be evaluated at each resolution. For example, if a text instance is too small at 256 resolution (e.g., less than 5x5 pixels), it is not reasonable to expect it to be reconstructed and correctly recognized, so such instances are excluded from evaluation.