What is the difference between rotmat and global_xform?

[xwg0508]

Hi , thank you for your code ! I want to ask the difference between rotmat(or rot6d) and global_xform?
From the code in amass.py , i see the global_xform was gotten by the following code:


SO What is the difference between rot6d and global_xform?Why we get global_xform through rot6d
AND in training and applying , the global_xform was seen as rotmat, such as the following code:

SO WHAT is the global_xform?

[c-he]

rot6d and rotmat are joint rotations relative to their parent joint, while global_xform is the rotation relative to the root joint. In training, I used global_xform since this global rotation representation performed better in our experiments. I should have given it a different name in the training code, so sorry for the confusion caused by my poor coding style :)

[ojss]

@c-he something is confusing me here. The model (generative) returns local_motion but that is again passed through global_to_local of the fk layer to get local_rotmat but this doesn't get used in the loss here.

The loss is between the 6d local_motion and input rotmat which is still the local (relative to parent) representation in the preprocessing code.

I'm not sure what I'm missing 😦

[c-he]

Basically we need to clarify the variable names first:

• global_motion and local_motion: they refer to the output of our motion field, where global and local are used to distinguish between the rotation of the root joint (global) and the rotations of other joints (local).
• global_to_local() in fk: in this context, global and local distinguish between rotations relative to the root joint (global) and those relative to the parent joint (local). To compute FK, we first need to convert the decoded rotations (rotmat_recon), which are relative to the root joint, into rotation matrices relative to their respective parent joints -- denoted as local_rotmat. Using these local rotation matrices, we can then apply the chain rule to compute joint positions.
  

  NeMF/src/nemf/generative.py

  Lines 191 to 207 in 6c98101

  	local_motion, global_motion = self.field(t, z_l, z_g) # (B, T, N)
  	rot6d_recon = local_motion[:, :, :n_joints * 6].contiguous() # (B, T, J x 6)
  	rot6d_recon = rot6d_recon.view(b_size, -1, n_joints, 6) # (B, T, J, 6)
  	rotmat_recon = rotation_6d_to_matrix(rot6d_recon) # (B, T, J, 3, 3)
  	if self.args.data.root_transform:
  	identity = torch.zeros(b_size, t.shape[1], 3, 3).to(self.device)
  	identity[:, :, 0, 0] = 1
  	identity[:, :, 1, 1] = 1
  	identity[:, :, 2, 2] = 1
  	rotmat_recon[:, :, 0] = identity
  	rotmat_recon[:, :, -2] = rotmat_recon[:, :, -4]
  	rotmat_recon[:, :, -1] = rotmat_recon[:, :, -3]
  	local_rotmat = self.fk.global_to_local(rotmat_recon.view(-1, n_joints, 3, 3)) # (B x T, J, 3. 3)
  	pos_recon, _ = self.fk(local_rotmat) # (B x T, J, 3)
  	pos_recon = pos_recon.contiguous().view(b_size, -1, n_joints, 3) # (B, T, J, 3)

During training, the geodesic loss is actually computed between rotation matrices relative to the root joint, see:

NeMF/src/nemf/generative.py

Line 102 in 6c98101

self.input_data['rotmat'] = rotation_6d_to_matrix(self.input_data['global_xform'])

Again, I'm sorry for the confusion caused by my poor naming style. Please let me know if you have any further questions.

[ojss]

@c-he ah I get it now thank you! I missed Line 102, this makes so much more sense now haha. Once again, appreciate your quick replies :)