Distributing simple model by itself - individual yaml file or OrsoDataset? Standardisation is required.

[andyfaff]

I'm getting requests for (de-)serialisation of simple models. I am trying to go down the lines of using the simple model language.

Most of the test examples that use the simple model language are individual YAML files (e.g. https://github.com/reflectivity/orsopy/blob/main/examples/sample_model_example_2.yml).

There is no functionality for testing a file to see if they are model language files, and whether they are ORSO compliant or not. In order to do this one has to load the yaml and try to resolve the model. If there's an exception it can't have been ORSO compliant.

• could we have a function that says if these files are model language files and check for ORSO compliance (schema?)?
• should we have # # ORSO reflectivity data file | 0.1 standard | YAML encoding | https://www.reflectometry.org/ at the top of those simple YAML files?

More to the point, we need some standardisation as to how we pass around a simple model by itself. Should we be placing the simple model in bare yaml by itself, or should it be placed into an OrsoDataset? My hope is for the latter.

Unfortunately it's not possible to create, or read, an OrsoDataset that only has the model and no data. The following raises an exception:

from orsopy.fileio import Orso, OrsoDataset
header = Orso.empty()
# inject a simple model into `info` at this point.
od = OrsoDataset(info=header, data=None)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[25], line 1
----> 1 od = OrsoDataset(info=header, data=None)

File <string>:5, in __init__(self, info, data)

File [~/miniforge3/envs/dev3/lib/python3.13/site-packages/orsopy/fileio/orso.py:157](http://localhost:8888/lab/tree/refnx-testdata/data/reflect/~/miniforge3/envs/dev3/lib/python3.13/site-packages/orsopy/fileio/orso.py#line=156), in OrsoDataset.__post_init__(self)
    156 def __post_init__(self):
--> 157     if self.data.shape[1] != len(self.info.columns):
    158         raise ValueError("Data has to have the same number of columns as header")

AttributeError: 'NoneType' object has no attribute 'shape'

[aglavic]

Hi @andyfaff ,

you are tuching on a few points that were on my todo list or which I started to think about but didn't reach a conclusive answer to.
Here are my current thoughts about the points you raised:

Validating model language
This is a bit of a tricky one. If we just want to validate that a model is conforming to the yaml definition, this is the same as with other parts of the orsopy header classes. The model language is a valid sub-class of our generic Header and can thus be validated by either trying to create the class or by validating against the schema.
I think this is something we did not yet expose well in the orsopy user API. Currently creating from dictionary bascially always works but may generate None values while creating from class instantication can't use user attributes and is thus the simplest check if it fails. I assume the nicest would be to have a classmethod that can be called with a dictionary to return a plane boolean telling if it is complient or not.

More tricky is the question, if it is a correct model, meaning that all names can be resolved. The issue there is, that names can either be resolved internally from within the model (e.g. substacks or layers) or can be looked up from a database. This ties to the fact that the class currently doesn't through very helpful errors for models where the name resolution fails. My approach here would be to build a better name resolution scheme in the code that could then be used to return to the user a list of names that are not defined in the model and thus would need database resolution.

Standalone model
The examples are just the plain yaml to keep it more clean, normally the whole structure would just be pasted in the orso header attribute for the model. I agree that it would be useful to have a simple way to pass around just the models, but it should not be confused with actual orso datafiles.
My suggestion is a new file type that maintains the model language schema but is actual yaml without the # in front of the lines. This should have a individual header line and possibly suffix. (Either 'yml/yaml' or 'orm' for ORSO Reflectivity Model.)
Header could be:
# # ORSO reflectivity model file | 0.1 standard | YAML encoding | https://www.reflectometry.org/