What is idx2?

idx2 is a compressed file format for scientific data represented as 2D or 3D regular grids of data samples. From a single idx2 dataset, you can retrieve data at different resolution and tolerance levels that best suit your needs at hand.

Compilation

idx2 can be built using CMake. The dependencies are:

• CMake (>= 3.8)
• A C++ compiler supporting C++17
• (Optional) nanobind (do git submodule update --recursive --init to pull it from GitHub)
• (Optional) Python 3

The optional dependencies are only needed if BUILD_IDX2PY is set to ON in CMake.

Using the idx2App command line tool to encode raw to idx2

idx2App --encode Miranda-Viscosity-[384-384-256]-Float64.raw

For convenience, the dimensions of the input are automatically parsed if the input file is named in the Name-Field-[DimX-DimY-DimZ]-Type.raw format.

Note that Type can only be either Float32 or Float64 (currently idx2 only supports floating-point scalar fields). If the input raw file name is not in this form, please additionally provide --name, --field, --dims, and --type.

Using the idx2App command line tool to decode idx2 to raw

idx2App --decode Miranda/Viscosity.idx2 --downsampling 1 1 1 --tolerance 0.001

--downsampling specifies the desired downsampling passes along each axis (each pass halves the number of samples along an axis), and --tolerance to specify the desired absolute error tolerance.

Optionally, use --first x_begin y_begin z_begin and --last x_end y_end z_end (the end points are inclusive) to specify the region of interest instead of decoding the whole field.

Using the C++ API to read from an idx2 dataset to memory

See the Source/Applications/idx2Samples.cpp file for an example of how to use idx2's C++ API.

(Most convenient option) Using the header-only library idx2.hpp

Just include a single header file for convenience. The idx2.hpp header file can be included anywhere, but you need to #define idx2_Implementation in exactly one of your cpp files before including it.

#define idx2_Implementation
#include <idx2.hpp>

Using the compiled idx2 library

Alternatively, with CMake, you can build an idx2 library and link it against your project. Then, just #include <idx2.h> to use it.

References

Efficient and flexible hierarchical data layouts for a unified encoding of scalar field precision and resolution D. Hoang, B. Summa, H. Bhatia, P. Lindstrom, P. Klacansky, W. Usher, P-T. Bremer, and V. Pascucci. 2021 - IEEE Transactions on Visualization and Computer Graphics

For the paper preprint and presentation slides, see http://www.sci.utah.edu/~duong