ISET3d rendering

Relevant tutorials

• t_lensFisheye

---

Introduction to Rendering with ISET3d

The piWrite and piRender functions are often executed in series. The piWrite(thisR) command uses the information in the recipe to write the files PBRT will render. By default these files are written in the user's ISET3d/local directory. The location of these files is stored in the recipe itself.

The piRender(thisR) command invokes the Docker container that runs PBRT on these output files. The piRender command returns an ISETCam (or ISETBio) struct or various types of metadata. If the recipe includes a lens, the return will typically include an optical image of the irradiance at the film. If the recipe uses a pinhole (denoted by the 'perspective' camera class), the return will typically be a scene that defines the scene radiance.

This is a quick overview of how piRender command returns different types of radiance, irradiance, or metadata. More information is provided in the Rendering section of the wiki.

Example: Rendering a Scene

This example is a chess set that we found online. We cannot parse the file. Still, we can read it in. By default the camera is set to a 'perspective', which means a pinhole camera.

thisR = piRecipeDefault('scene name','chessSet');
thisR.set('film resolution',[300 200]);
thisR.set('rays per pixel',64);   % Number of rays set the quality.
>> thisR.get('camera')

ans = 

  struct with fields:

       type: 'Camera'
    subtype: 'perspective'
        fov: [1×1 struct]

piWrite followed by piRender produces this scene.

piWrite(thisR); 
scene = piRender(thisR);
sceneWindow(scene);

Metadata render types

By default, piRender returns both a scene and its depth map. You can plot the depth map from the 'Plot' pulldown in the scene window or by running scenePlot(scene,'depth map');

Using the 'render type' keyword, piRender can be set to return different quantities. The default is ('both') which means radiance and depth. Here are some other settings (check 'doc piRender' from time to time in case we update the function). NOTE: With v4, it is much more efficient to ask for multiple results at once, if possible, as otherwise the scene needs to be rendered again for each additional call.

piRender(thisR,'render type','radiance')
piRender(thisR,'render type','both')
piRender(thisR,'render type','depth')
piRender(thisR,'render type','illuminant')

You can also specify 'render type' as 'material' that returns an image where each pixel is a label for the material type.

NOTE: This is not working in v4 yet!

material = piRender(thisR,'render type','material');
ieNewGraphWin;
imagesc(material); colormap(hsv); axis image

Irradiance

When you add a lens to the render recipe, the returned value describes the light falling on the sensor after passing through the lens (instead of the scene's radiance). This is the spectral irradiance at the sensor. Here is how we render the chess set scene with a fisheye lens.

lensfile = 'fisheye.87deg.6.0mm.json';
thisR.camera = piCameraCreate('omni','lensFile',lensfile);
thisR.set('focus distance',0.45);
thisR.set('film diagonal',10);
thisR.set('aperture diameter',3);
piWrite(thisR);

oi = piRender(thisR,'render type','radiance');
oiWindow(oi);
oiSet(oi,'render flag','hdr');

There are many types of lenses, and even an entire repository devoted to creating, analyzing and editing lenses (isetlens). Many lens parameters can be set using the thisR.set() commands, and these will be described in the Camera section of the wiki.

Limitations

Any PBRT file and its auxiliary information (e.g., specified by the Include command) can be rendered by a piRender command. But as we describe in the introduction, not all PBRT files are equal. At this time, we cannot parse every PBRT text files into assets and materials in the recipe. If we cannot parse the assets, materials and lights, then our ability to control the assets is limited or non-existent. When we cannot parse, we just copy the scene and its related files and use piRender to create scene radiance or irradiance images.

We are working on expanding the set of PBRT files we can parse. Files the are exported from Blender and Cinema4D can be parsed. As we encounter different 'wild' files, with different organizations, we try to figure out how to parse those as well.