File I/O Standardization and Improvements

[zhardy-lanl]

I think our file I/O system could use a little bit of a refresh. Currently, the LBSSolver binary files for flux moments and angular fluxes are a bit different. The flux moments binary first writes grid information, then the indexed data, while the angular flux binary only writes the indexed data. Additionally, there is the ExportMultipleToVTK function in FieldFunctionGridBased.

There are two primary items I think we should think about:

1. Standardizing full solution binaries and moving them to field functions

   • Field functions have all of the requisite information for writing solution data (grid, unknown structure, data).
   • More specific header information would be lost (i.e. moment/angle -> unknown, group -> component)
   • Tighter integration of field functions in the code may be necessary.

2. Developing specialized file I/O

   • Adding an in house mesh file i/o (as a companion to above solution binaries if mesh info is not included)
   • Adding groupset-wise file i/o
   • Adding outgoing angular flux output files

I'll likely be doing a bit of file i/o development in the coming months for an application I am working on, so I would like to see what everyone's thoughts are on the final state.

[zhardy-lanl]

Recap of 12/06/2023 Discussion

• Look into using HDF5 file formats with the h5pp C++ wrapper library.
• More discussion needed on whether to move generic solution binaries to field functions (may require field function rework).
• Discussed supporting both lightweight binaries (in house format or HDF5) and larger consolidated outputs (ExodusII, VTK, etc.).
• More discussion needed on whether or not to include mesh information in lightweight binaries, and whether to have a lightweight mesh only binary or not.
• Discussed the need for being able to read on a fewer processors than used to write.

If I missed anything, please feel free to comment below.

[andrsd]

I personally support the use of some higher level API for file I/O. It is really a PITA to do anything low-level, since one has to write a lot of error handling code. So, for example, using something like h5pp will make the developer's life much simpler, since the only thing the application is concerned with is the structure of data being read/written. This would be in regard to the binary files used for "restart".

For output regarding visualization, this is a bit different story IMO. Here, a standardized format like ExodusII, VTK, whatnot should be chosen. If not, a conversion tool/code must be written and maintained. That's typically not a good idea.

The reasoning behind having 2 kinds of outputs, i.e. one for "restart" and one for viz, is that restart typically requires to store some internal state of the code (IDK how much this is true for openSn) and the standardized file formats typically do not have the capacity to do so.