Loss scale

[JunyuanDeng]

Hello,

I am using your code to train our own dataset, but I found the loss become negative, is it normal? If so, could your provide a number that the loss would converge to?

Thanks for your futher reply!

[edexheim]

Hello,

Yes, the loss can become negative and that is normal. The loss is a log marginal likelihood of a Gaussian process, which can become negative. In Equation 8, for example, there is a Mahalanobis distance which must be positive (like in a standard MSE loss), but the log determinant can become negative, so the loss can be positive or negative.

Let me know if this helps.

Best,
Eric

[JunyuanDeng]

Hello, Eric

During the training, I found that even in the first 300 iteration, the complexity_term loss could be -5, the constant_term is 0.92 and the data_term is around 0.1~0.4, while the total loss could be -80. Is it normal to have a such low loss?

[JunyuanDeng]

Hello Eric,

I have another question. If I want to train this model on a more diverse dataset (including both indoor and outdoor scenes), rather than only ScanNet, do you think I need to modify anything to better handle the domain variation?

For example, the variance is currently implemented as a single learnable parameter—would it make more sense to have the model predict the variance (e.g., per image or per pixel) to adapt to different scenarios?

[edexheim]

Hi Junyuan,

Regarding the first question, a total loss of -80 seems reasonable if I recall correctly. If the complexity term is -5, this likely means the trace term in the VFE loss is around -75? One way that I would often sanity check the model is by running the active_depth_viz.py and depth_completion_viz.py to get a feel for how pixels are correlated and if adding points one-by-one seems reasonable in how the depth error is reduced.

Regarding the second question, in this version of the code I did not try with such different domains. The biggest difference with outdoor data is that the variance around the mean depth is likely larger than indoor data (such as driving where the road nearby is very close while points down the road are much further).

I had tried predicting variance per-pixel, which is quite challenging to optimize, as the model seemed to have trouble distinguishing whether to treat pixels as correlated or simply just assign high per-pixel variances to points that were unlikely under the current model. One sanity check would be to make a noise variance and signal variance per dataset, as this noise parameter is essentially some irreducible noise from the dataset (for example, from sensor noise). I had not tried a noise variance and signal variance per image, but it should be possible to predict that from the network as well. This would be more constrained than the per-pixel variance so this feels like a promising way to approach the problem.

Thanks for your interest in the work and let me know if you have more questions!