Per-instance materials

[miroslawpawlowski]

I have faced a problem in OSPRay and just would like to make sure it is not an issue for ANARI. Let’s assume we want to create a scene with two objects that share geometry but have different position and material (e.g. wooden teapot and gold teapot). We want to use a single geometry and thus construct a single BVH. This means we will have a single anari::Group object. In the current 1.0 ANARI version there is no way we can specify per-instance material. Will it be solved by defining another instance sub-type e.g. KHR_INSTANCE_TRANSFORM_AND_MATERIAL which would additionally contain index into some material array? But then, where to store this material array, in anari::Surface?

[jeffamstutz]

The short answer is that alternate methods of associating materials with geometries isn't inherently blocked by ANARI's API design -- OSPRay certainly can implement any additional material assignment possibilities it wants. I am about to argue against the need for override-based approaches for material/geometry association, but that is purely from the perspective of it being a core extension that other vendors are expected to implement.

I am skeptical for it to be necessary to do instance based overrides in general, including materials, because the original reason that was invented for OSPRay was to deal with a problem that ANARI does not have: renderer-specific materials.

For years OSPRay required that materials must be compatible with the renderer that is currently in-use: the pathtracer and scivis renderers had completely different implementations they used when rendering a frame. This then required applications to create different versions of the same world or required them to go and modifiy a single world by swapping out all the OSPMaterial instances. At the time, there was urgency to show enormous scenes flipping back and forth between different renderers -- enter material-list overrides. What this did was let file loaders create N different versions of a single OSPMaterial, one for each renderer, then when the renderer was switched the global list of materials could be swapped to the one that matches the new renderer.

ANARI doesn't have this limitation in that all materials are supposed to be compatible with all renderers (though shading choices may differ, of course). If you want to change an object's material, you swap it out on the surface you want to modify.

Philosophically, we have very much held to doing things only one way when this has come up in various other places (instancing, volumes/fields, lights, etc.) -- so the way ANARI would solve having a single geometry use different materials is for the geometry to be shared between different surfaces each in their different groups/instances. This avoids creating "implicit surface objects" by ensuring that the lifetime of ANARISurface is indeed the lifetime of a particular geometry/material combination -- that association is not always cheap depending on implementation choices (e.g. GL state changes), so assuming it's a cheap association for everyone like it is for OSPRay is not as portable as you might think.

[miroslawpawlowski]

Thank you for your exhaustive answer. I fully agree with the majority of opinions expressed in it. The only one I have mixed feelings is this statement:

"so the way ANARI would solve having a single geometry use different materials is for the geometry to be shared between different surfaces each in their different groups/instances"

For raytracing engines like OSPRay the most intuitive place to keep object local BVH is Group object. It conveniently gathers all geometries and volumes that share the same Euclidean space. This means that for my initial example of a scene with two teapots, we have two Group objects and thus two identical BVHs, wasting both memory and compute to generate them. For a scene with 100 instanced cars, each in different color, we have to generate and store 100 identical BVHs.

That is why I initially thought about the new KHR_INSTANCE_TRANSFORM_AND_MATERIAL instance subtype accompanied by an array of materials in anari::Surface. It is memory efficient by not duplicating groups, thus exploiting full potential of instancing.

[jeffamstutz]

For raytracing engines like OSPRay the most intuitive place to keep object local BVH is Group object. It conveniently gathers all geometries and volumes that share the same Euclidean space. This means that for my initial example of a scene with two teapots, we have two Group objects and thus two identical BVHs, wasting both memory and compute to generate them. For a scene with 100 instanced cars, each in different color, we have to generate and store 100 identical BVHs.

That's a fair point and enough of an argument to keep the idea moving forward. There's a related issue in the wild with an existing ANARI backend implementing per-primitive materials as well (Cycles in the context of Blender). Proceeding with any material list extensions should make sure that these two ways of indexing into a material list (primID, geomID, instID, explicit parameter indexing, other schemes...) are designed together so that they are cohesive and don't make subtle differences in design choices.

After the v1.0 release, I think this would make a fine topic to put on the list for v1.1. Thanks!

[miroslawpawlowski]

I see two potential solutions for the problem above. To better illustrate it, let's have a scene with three instances like this:

We would like to use different Material for each Instance.

Proposal 1
The most simple and the most intuitive approach would be to just define Material as Instance parameter. Each Material would be equally applied to all surfaces within its Instance, e.g. Material I1 will overwrite both Material 1 and Material 2 and so on:

Proposal 2
More complex but also more powerful solution would be to add material array as Surface parameter. The array would be indexed by instance index, as ordered in the World, modulo material array size. This way we can retain variety of materials below Group.

[jeffamstutz]

A couple of comments:

• I very much like the first proposal because it is straightforward to reason about with simple edge cases to consider.
• Material lists get messy, because it's easier to get them wrong (indexing must match the size of the material list). I don't like automatic array truncation behavior as the required behavior, because it wastes computation on fixing an incorrect scene. OSPRay certainly is free to be "well behaved" with out-of-bounds array indexing, but I don't want that in the spec (that behavior then becomes required when it doesn't need to be).
• I also want to stay away from implicit indexing schemes involving instances, because then the surfaces you end up with are inherently tied to the instance list they exist in. When looking at the hierarchy, it is valuable for siblings objects in different worlds to not "fight" over how a lower level objects ought to look. In the case of the 2nd proposal, the instances in this particular world assume the surface has a material list constructed with the instanceIDs in mind, but those instances may need to exist in sparse locations within the instance list (i.e. it doesn't cleanly index [0-(n-1)])...immediately bringing in the need to indirect with another indexing scheme to make sure you get linear indexing.