Visualization results

[fufj]

Hi~ Thanks for your remarkable work.
I am reproducing your paper results including quantitative and qualitative results. However, difficulties are encountered in generating the visualization results as shown in Figure 6 of your paper. I have found taht two .py files in this repository, plot_depth_utils.py and visualization.py, may be related.
Would you mind providing some help for me?

[ruili3]

Hi,

Thanks for your interest :)

• We used the AbsRel error map in Fig.6, you may alter the save_concat_res() function in plot_depth_utils.py for error visualization.
• Unfortunately, the current codebase does not contain the original point cloud generation script. Our baseline work has a point cloud generation script, but it requires further debugging to work out. An alternative is to use other repos, eg. these functions (pcd, save_pcd) in Metric3D. It should work fine in our repo if you set the intrinsic properly.

[fufj]

Thank you so much. I will try it.

[757787182]

Hi~ Thanks for your remarkable work.
my loss of error chart looks like this,Would you mind providing some help for me?

[ruili3]

Hi~ Thanks for your remarkable work. my loss of error chart looks like this,Would you mind providing some help for me?

Did you use the script I mentioned above? It seems that you are using the original image and a different color scheme

[757787182]

thanks you for your reply ,i will try it

[xuhang2017]

Hello, I am very interested in your work, but I did not find the visualization code of the depth image in your code, it is convenient for you to provide it, thank you

[ruili3]

Hello, I am very interested in your work, but I did not find the visualization code of the depth image in your code, it is convenient for you to provide it, thank you

Hi, thanks for your interest. Have you tried using the following code as I mentioned above?

We used the AbsRel error map in Fig.6, you may alter the save_concat_res() function in plot_depth_utils.py for error visualization.
Unfortunately, the current codebase does not contain the original point cloud generation script. Our baseline work has a point cloud generation script, but it requires further debugging to work out. An alternative is to use other repos, eg. these functions (pcd, save_pcd) in Metric3D. It should work fine in our repo if you set the intrinsic properly.

Feel free if you encounter any difficulties

[xuhang2017]

Hello, I am very interested in your work, but I did not find the visualization code of the depth image in your code, it is convenient for you to provide it, thank you

Hi, thanks for your interest. Have you tried using the following code as I mentioned above?

We used the AbsRel error map in Fig.6, you may alter the save_concat_res() function in plot_depth_utils.py for error visualization.
Unfortunately, the current codebase does not contain the original point cloud generation script. Our baseline work has a point cloud generation script, but it requires further debugging to work out. An alternative is to use other repos, eg. these functions (pcd, save_pcd) in Metric3D. It should work fine in our repo if you set the intrinsic properly.

Feel free if you encounter any difficulties

Thank you so much. I will try it.

[kehuitt]

Hi, thanks for your remarkable work!
I'm attempting to visualize the results using the script mentioned above. However, I can not obtain the true visualized outcomes. The relevant revised code is presented below. Would you identify what needs modification?

# evaluate.py
for batch_idx, (data, target) in enumerate(self.data_loader):
    data, target = to(data, self.device), to(target, self.device)
    data["target"] = target

    with torch.no_grad():
        data = self.model(data)
        loss_dict = {"loss": torch.tensor([0])}
        loss = loss_dict["loss"]

    output = data["result"]

    pred_disp = output.cpu().numpy()
    gt_depth = target.cpu().numpy()
    img = data["keyframe"].cpu().numpy()
    self.save_visualization(batch_idx, pred_disp, img, gt_depth)

def save_visualization(self, batch_idx, pred_disp, img, gt_depth):
    """
    :param pred_disp: [batch_size, h, w]
    :param img: [batch_size, c, h, w]
    :param gt_depth: [batch_size, h, w]
    """

    output_dir  = "./visualization_results"
    if not os.path.exists(output_dir):
        os.mkdir(output_dir)

    for i in range(gt_depth.shape[0]):
        sample_pred_disp = np.transpose(pred_disp[i], (1, 2, 0))
        sample_gt_depth = np.transpose(gt_depth[i], (1, 2, 0))
        sample_img = np.transpose(img[i], (1, 2, 0))
        save_concat_res(output_dir, sample_pred_disp, sample_img, sample_gt_depth)

[ruili3]

Hi, thanks for your remarkable work! I'm attempting to visualize the results using the script mentioned above. However, I can not obtain the true visualized outcomes. The relevant revised code is presented below. Would you identify what needs modification?

evaluate.py

for batch_idx, (data, target) in enumerate(self.data_loader):
data, target = to(data, self.device), to(target, self.device)
data["target"] = target

with torch.no_grad():
    data = self.model(data)
    loss_dict = {"loss": torch.tensor([0])}
    loss = loss_dict["loss"]

output = data["result"]

pred_disp = output.cpu().numpy()
gt_depth = target.cpu().numpy()
img = data["keyframe"].cpu().numpy()
self.save_visualization(batch_idx, pred_disp, img, gt_depth)

def save_visualization(self, batch_idx, pred_disp, img, gt_depth):
"""
:param pred_disp: [batch_size, h, w]
:param img: [batch_size, c, h, w]
:param gt_depth: [batch_size, h, w]
"""

output_dir  = "./visualization_results"
if not os.path.exists(output_dir):
    os.mkdir(output_dir)

for i in range(gt_depth.shape[0]):
    sample_pred_disp = np.transpose(pred_disp[i], (1, 2, 0))
    sample_gt_depth = np.transpose(gt_depth[i], (1, 2, 0))
    sample_img = np.transpose(img[i], (1, 2, 0))
    save_concat_res(output_dir, sample_pred_disp, sample_img, sample_gt_depth)

Hi, could you please specify what issues specifically?

[kehuitt]

@ruili3 Hi, I utilized the save_concat_res(output_dir, sample_pred_disp, sample_img, sample_gt_depth) function from plot_depth_utils.py, and made revisions to the code in evaluate.py. Here is the result I obtained. (I have not modified any code in plot_depth_utils.py)
Except for the second picture, the other three pictures seem to be wrong, is there something wrong with the code I modified in eval.py? Thanks!!

[ruili3]

@ruili3 Hi, I utilized the save_concat_res(output_dir, sample_pred_disp, sample_img, sample_gt_depth) function from plot_depth_utils.py, and made revisions to the code in evaluate.py. Here is the result I obtained. (I have not modified any code in plot_depth_utils.py) Except for the second picture, the other three pictures seem to be wrong, is there something wrong with the code I modified in eval.py? Thanks!!

thanks for the feedback! it should work fine with the following changes:

• adding 0.5 to the input image and clipping the value between [0,1]: the "keyframe" was originally mapped to [-0.5, 0.5]
• transferring GT disp into metric depth following the same way as was done to the prediction: https://github.com/ruili3/dynamic-multiframe-depth/blob/b3d3f4eb00052c9d1f42c5f0abfaf896fceeb39e/depth_proc_tools/plot_depth_utils.py#L187C7-L187C85

feel free to let me know if further assistance is needed!