Can only estimate for a few seconds

[missTL]

The depth estimation measurement always crashes after a few seconds until it becomes completely dark.

[liu-ht23]

Are you using the fp16 to run inference? I also met with this problem when using fp16 or bp16 to run inference. But it is ok with fp32. Why do fp16 and bp16 fail and how to fix it?

[nagadomi]

If the issue happens only with fp16, it is often caused by -inf or +inf appearing somewhere and propagating, or turning into nan.

My modified code had a similar issue, so I addressed it, but the cause was a part that differs from the official code. I do not know whether the same issue exists in this official code.

[liu-ht23]

If the issue happens only with fp16, it is often caused by -inf or +inf appearing somewhere and propagating, or turning into nan.

My modified code had a similar issue, so I addressed it, but the cause was a part that differs from the official code. I do not know whether the same issue exists in this official code.

Want to know what was the issue you met and how did you address it?

[nagadomi]

Video-Depth-Anything/video_depth_anything/dpt_temporal.py

Lines 120 to 121 in 4f5ae23

	with torch.autocast(device_type="cuda", enabled=False):
	out = self.scratch.output_conv2(out.float())

I was running this line in fp16, which resulted in inf values depending on the input video..
In the original code, since AMP is used, this line runs in fp32 even when fp16 mode is enabled.

[liu-ht23]

Thanks so much! I found out that it was the problem with certain blocks and layers of DINOv2 and DPThead. So I manually set these blocks and layers to use fp32 and it can work normally now.