While human readable, the output toml files can become quite large for 2D time series measurements.
Instead of dumping to toml, rex could instead write to parquet etc. Rex could provide functionality to convert to toml if needed. rex_utils can then work with both file types interchangeably as well.
This would allow for better handling of data during the measurement process. Though this is a maybe, as much of the benefit of the current rex architecture is being able to view the data in a low latency way.
While human readable, the output toml files can become quite large for 2D time series measurements.
Instead of dumping to
toml,rexcould instead write to parquet etc.Rexcould provide functionality to convert totomlif needed.rex_utilscan then work with both file types interchangeably as well.This would allow for better handling of data during the measurement process. Though this is a maybe, as much of the benefit of the current
rexarchitecture is being able to view the data in a low latency way.