Regular 3D grid typed object

[BenLewis-Seequent]

Description

This introduces "typed objects", using regular 3D grid as an example.

The existing interfaces in the evo-objects SDK is around handling Geoscience Objects generically with respect to the object type. Though in a lot of use cases, the user of the SDK knows which type of object they want to handle, whether that is a point set, regular 3D grid, etc. Providing classes that are specific to a particular type of object, provides better user experience as the user does not need to know the details about the schemas, and can just use auto complete or refer to the class directly. It also gives the ability to do more validation.

Examples

Creating an regular 3D grid:

grid = Regular3DGridData(
    name="Test Grid21",
    origin=Point3(0, 0, 0),
    size=Size3i(10, 10, 5),
    cell_size=Size3d(2.5, 5, 5),
    cell_data=pd.DataFrame(
        {
            "value": np.random.rand(10 * 10 * 5),
            "cat": pd.Categorical.from_codes(np.random.choice(range(4), size=10 * 10 * 5), ["a", "b", "c", "d"]),
        }
    ),
    vertex_data=pd.DataFrame(
        {
            "elevation": np.random.rand(11 * 11 * 6),
        }
    ),
    rotation=Rotation(90, 0, 0),
)
created_grid = await Regular3DGrid.create(manager.get_context(), grid)

Download:

grid = await Regular3DGrid.from_reference(manager.get_context(), created_grid.metadata.url)
name = grid.name
cells_df = await grid.cells.as_dataframe()

Update:

grid.name = "Updated Test Grid"
await grid.cells.set_dataframe(
    pd.DataFrame(
        {
            "value": np.ones(10 * 10 * 5),
        }
    )
)
# Update the object to create a new version with the modified properties
await grid.update()

Schema versions

This is designed to gracefully handle multiple major versions of object schemas, through the use of adapters that map the details of the schemas to the class properties. This isn't fully fleshed out yet, but these details are not part of the public(stable) interface.
This also only inspects schema properties upon access, meaning it should be able to handle newer minor versions of the object schemas, as long as the parts of the object that is accessed isn't using values added in the newer versions(i.e. new enum values).

Checklist

• I have read the contributing guide and the code of conduct

[RyanMillerSeequent]

LGTM

[rileyhowley-seequent]

This is great work @BenLewis-Seequent and will be very beneficial for users. I just had a couple of questions but nothing blocking.

[wordsworthc]

I think this is a good start, but I wonder how this could evolve over time.

There are a few areas where I am overwhelmed by the sheer volume of code, and I am left wondering if there is a cleaner solution. The _BaseObject, for example, seems to duplicate most of the functionality of DownloadedObject (and yes, it adds a little new functionality for uploading too). I've also made a specific suggestion to clean up usage of the SchemaProperty descriptor (I like the PoC!).

Is this intended as a preview release? I have a few ideas it would be nice to try out and share in the new year sometime, but I would be very worried about breaking the library API

[BenLewis-Seequent]

There are a few areas where I am overwhelmed by the sheer volume of code, and I am left wondering if there is a cleaner solution. The _BaseObject, for example, seems to duplicate most of the functionality of DownloadedObject (and yes, it adds a little new functionality for uploading too). I've also made a specific suggestion to clean up usage of the SchemaProperty descriptor (I like the PoC!).

I have split out some of the logic from _BaseObject into a new SchemaModel, to hopefully reduce the size of that class to make it more digestible. The functionality of it is very similar to DownloadedObject but most of the logic is to do with down casting into the appropriate type, or creating/replacing object from a data class. The other important difference is DownloadedObject represents one specific version of an object, which exactly reflects what is on the service. Where as these new typed objects are meant to be mutable so they can be modified in place before being uploaded. So I think it being "unrelated" to DownloadedObject is not a problem.

Is this intended as a preview release? I have a few ideas it would be nice to try out and share in the new year sometime, but I would be very worried about breaking the library API

Agree, I just don't know how a preview release would work though, as other changes may well want to be released well we still evaluate this. A possibly is to somehow mark this package as a "preview" package, other libraries sometimes have a package named "preview". So the import for this could be from evo.objects.preview.typed import Regular3DGrid.

[BenLewis-Seequent]

To reduce the scope of this PR, I removed support for loading and saving non-attribute arrays/tables. These will be needed in the future, but can be added when needed.

[wordsworthc]

Happy New Year 🎉

I must apologize, I have been swamped with other initiatives and have not yet found the time to provide a qualitative re-review of your pull request. Unfortunately I do not see myself having the chance in the near future either.

Because I know you, and as we have discussed in person, I trust that you have addressed my earlier comments thoroughly and sensibly, and it is on that basis that I am adding my approval here.

I agree with you that it would be nice to be able to mark some of this as experimental, i.e., subject to change at a later date. I think we are somewhat protected by using dev versioning here (0.y.z), although some additional comments in docstrings would go a long way to making it explicit IMO. I will leave it for others to discuss the best way to communicate the specifics here.

Thank you for your significant effort, I believe your work is an invaluable contribution to the future of this SDK 🙂