Render the result using `results.json`

[HynDufO]

Hi, I just came across the paper and attempted to reproduce the results.

I successfully followed your guide in README.md but am unsure about what to do with result.json to render the 3D model. Could you provide me with some guidance? Thanks in advance.

[fantaosha]

Hello, sorry for the late reply. You need to convert rotation matrix to exponential coordinates to render the 3D model with SMPL.

[HynDufO]

Hello, sorry for the late reply. You need to convert the rotation matrix to exponential coordinates to render the 3D model with SMPL.

Thanks for answering. Recently, I've tried various methods to render it. I've successfully used the rotation and shape parameters (related to joints and body shape) of the SMPL model but encountered difficulties with the camera and trans parameters. I'd like to ask some questions:

• Does the camera follow the format (fx, fy, cx, cy), where fx and fy are the focal lengths, and cx and cy are the principal points?

I attempted to render using pyrender, but the result didn't quite match the image:

fx = 1145.51133842318 
fy = 1144.77392807652
cx = 514.968197319863 
cy = 501.882018537695 

intrinsics = pyrender.IntrinsicsCamera(fx, fy, cx, cy)

# The last column is the `trans` array
camera_pose = np.array([
    [1.0, 0.0, 0.0, -0.8137451718577722],
    [0.0, 1.0, 0, -0.38184425690653306],
    [0.0, 0, 1.0, 4.098549799582818],
    [0.0, 0.0, 0.0, 1.0],
])
scene.add(intrinsics, pose=camera_pose)

I also attempted to convert rotation[0] to a rotation matrix and replace it in the camera_pose, but it didn't work.

I'm relatively new to this. If you could provide me with more guidance, I would greatly appreciate it.

[fantaosha]

Hello, sorry for the late reply. You need to convert the rotation matrix to exponential coordinates to render the 3D model with SMPL.

Thanks for answering. Recently, I've tried various methods to render it. I've successfully used the rotation and shape parameters (related to joints and body shape) of the SMPL model but encountered difficulties with the camera and trans parameters. I'd like to ask some questions:

• Does the camera follow the format (fx, fy, cx, cy), where fx and fy are the focal lengths, and cx and cy are the principal points?

I attempted to render using pyrender, but the result didn't quite match the image:

fx = 1145.51133842318 
fy = 1144.77392807652
cx = 514.968197319863 
cy = 501.882018537695 

intrinsics = pyrender.IntrinsicsCamera(fx, fy, cx, cy)

# The last column is the `trans` array
camera_pose = np.array([
    [1.0, 0.0, 0.0, -0.8137451718577722],
    [0.0, 1.0, 0, -0.38184425690653306],
    [0.0, 0, 1.0, 4.098549799582818],
    [0.0, 0.0, 0.0, 1.0],
])
scene.add(intrinsics, pose=camera_pose)

I also attempted to convert rotation[0] to a rotation matrix and replace it in the camera_pose, but it didn't work.

I'm relatively new to this. If you could provide me with more guidance, I would greatly appreciate it.

Hello, if I remember correctly, pyrender seems adopted a different convention for rendering. Can you post the rendering result got from pyrender? Thanks!

[HynDufO]

Hi, here are some more details.

I used the resulted rotation and shape to export hello_smpl.obj using the original SMPL code.

The full script to render is render.py. The result is:

I tried rotating the SMPL model following the code I found in the smplify-x repo for rendering. Uncomment the part:

rot = trimesh.transformations.rotation_matrix(
    np.radians(180), [1, 0, 0])
fuze_trimesh.apply_transform(rot)

The model is kinda flipped correctly, but not quite match with view in the original image:

And if I convert rotation[0] to a rotation matrix and replace it in the camera_pose, it rendered nothing. It might be a problem with the exported .obj in the first place but I'm unsure. Appreciate for your help!

[HynDufO]

Hello, if I remember correctly, pyrender seems adopted a different convention for rendering.

Could you suggest some alternatives for correct rendering? Thanks in advance.