How to read an HDF5 file as a whole?

[yucale830]

After performing the hsds test, it was observed that in the actual file storage scenario, the h5 file was fragmented into multiple blocks, while the complete source file ceased to exist. The question at hand is whether there is currently a corresponding solution to fulfill the requirement of directly importing h5 files into image processing and disciplinary software such as ImageJ.

[jreadey]

By default, HSDS uses a sharded storage format to optimize use in the cloud.

If you just need to access HDF5 files as a whole, can't use just copy files stored in a file system or cloud storage?

[yucale830]

Thank you! Then, do you have any better way to achieve multiple writes and reads to the same file?

[jreadey]

It depends on the nature of your application. If each writer each writing to a different dataset, you could put each dataset in a separate file, and have a master file with external links to the individual datasets.

Another approach would be to use VDS with different writers writing to different source files.
(if you have questions on these topics, the HDF5 forum would be a good place to pursue them - https://forum.hdfgroup.org/c/hdf5/8)

Back to HSDS, if you just need an HDF5 file representation periodically, you can use the hsget utility (included with h5pyd) to convert from the HSDS storage schema to a regular HDF5 file.

Hope this helps!