it doesn't converge

[bigdimboom]

at first I removed findMaxUpdateNorm. After a 5 iterations, it goes into inf. energy. Shouldn't the energy be decreasing? And then it crashes at the next data term estimation.

Or if I run the code with findMaxNorm enable, it will crash at 3rd iteration at findMaxNorm
**

**

[dgrzech]

what data are you running it on? how come regularisation energy isn't 0 at the first iteration of the solver?

[bigdimboom]

I tried disabling the reg-term but it's still not good. Cuda error is at "findMaxUpdatesNorm"

[dgrzech]

you didn’t answer either of my questions

the cost decreases from iteration 1 to 2, so the solver is working correctly. my guess is that you set the gradient descent step size too high

[bigdimboom]

I want to use your solver in my code to achieve dynamic-fusion like result. So fa, I have implemented a static TSDF library with ICP. Early on, I forgot to initialize the Jacobian matrix while porting your code to my implementation. That's why it did not start at 0. Thank you for pointing that out.

Right now I'm testing your code solely with the depth data captured by myself. However, it never converge even after 2048 iterations at first frame. I tried many configurations including the alpha size and the recommended ones in the paper that you pointed out in your previous response.

I suspect it might be relevant to the noise generated by the depth sensor, but interestingly it does not affect my static fusion.

[bigdimboom]

Interesting... Even though it never converges, the tracking somehow is good enough when there is no more than translations in the data(like you said the solver does work.). The canonical volume can even accumulate in a meaningful way only when it does not contain rotations in the data.

I also tried other step sizes from 0.01 to 0.3. Solver stops working at 0.4

I'm using a 128, 128, 128 grid with 4-10 mm per voxel.
Alpha is 0.05 - 0.1
Filter size is 7
lambda is 0.1
weight 0.2

Log:

iter. no. 1
data energy + w_reg * reg energy = 7646.78 + 0.2 * 0 = 7646.78
max. update norm 0.350222 at voxel (87, 48, 61)
iter. no. 50
data energy + w_reg * reg energy = 2797.01 + 0.2 * 1.05463e+06 = 213722
max. update norm 0.0398565 at voxel (127, 0, 112)
iter. no. 100
data energy + w_reg * reg energy = 2656.16 + 0.2 * 1.30884e+06 = 264424
max. update norm 0.0311408 at voxel (127, 0, 111)
iter. no. 150
data energy + w_reg * reg energy = 2599.94 + 0.2 * 1.46987e+06 = 296575
max. update norm 0.0289678 at voxel (127, 0, 110)
iter. no. 200
data energy + w_reg * reg energy = 2568.47 + 0.2 * 1.5869e+06 = 319948
max. update norm 0.0297711 at voxel (127, 0, 109)
iter. no. 250
data energy + w_reg * reg energy = 2546.59 + 0.2 * 1.67322e+06 = 337190
max. update norm 0.0220492 at voxel (127, 0, 109)
iter. no. 300
data energy + w_reg * reg energy = 2530.55 + 0.2 * 1.73912e+06 = 350354
max. update norm 0.0247185 at voxel (127, 0, 108)
iter. no. 350
data energy + w_reg * reg energy = 2518.33 + 0.2 * 1.79112e+06 = 360743
max. update norm 0.0193914 at voxel (127, 0, 108)
iter. no. 400
data energy + w_reg * reg energy = 2508.63 + 0.2 * 1.83303e+06 = 369114
max. update norm 0.0230706 at voxel (127, 0, 107)
iter. no. 450
data energy + w_reg * reg energy = 2500.57 + 0.2 * 1.86783e+06 = 376067
max. update norm 0.0184403 at voxel (127, 0, 107)
iter. no. 500
data energy + w_reg * reg energy = 2493.79 + 0.2 * 1.89636e+06 = 381766
max. update norm 0.0155475 at voxel (127, 0, 107)
iter. no. 550
data energy + w_reg * reg energy = 2487.84 + 0.2 * 1.92082e+06 = 386651
max. update norm 0.0117325 at voxel (127, 2, 103)
iter. no. 600
data energy + w_reg * reg energy = 2483.23 + 0.2 * 1.94245e+06 = 390973
max. update norm 0.0098311 at voxel (127, 2, 103)
iter. no. 650
data energy + w_reg * reg energy = 2479.33 + 0.2 * 1.96136e+06 = 394752
max. update norm 0.00923391 at voxel (127, 2, 102)
iter. no. 700
data energy + w_reg * reg energy = 2475.37 + 0.2 * 1.97828e+06 = 398130
max. update norm 0.00981192 at voxel (127, 2, 102)
iter. no. 750
data energy + w_reg * reg energy = 2471.89 + 0.2 * 1.99309e+06 = 401089
max. update norm 0.00664955 at voxel (127, 2, 102)
iter. no. 800
data energy + w_reg * reg energy = 2468.92 + 0.2 * 2.00564e+06 = 403596
max. update norm 0.00483364 at voxel (127, 2, 102)
iter. no. 850
data energy + w_reg * reg energy = 2466.35 + 0.2 * 2.01632e+06 = 405731
max. update norm 0.00396784 at voxel (127, 2, 102)
iter. no. 900
data energy + w_reg * reg energy = 2464.05 + 0.2 * 2.02548e+06 = 407559
max. update norm 0.00362555 at voxel (127, 2, 102)
iter. no. 950
data energy + w_reg * reg energy = 2461.94 + 0.2 * 2.03336e+06 = 409135
max. update norm 0.0032354 at voxel (127, 2, 102)
iter. no. 1000
data energy + w_reg * reg energy = 2460.08 + 0.2 * 2.04022e+06 = 410504
max. update norm 0.00412488 at voxel (127, 3, 101)
SOLVER REACHED MAX. NO. OF ITERATIONS WITHOUT CONVERGING
iter. no. 1
data energy + w_reg * reg energy = 3550.65 + 0.2 * 2.04039e+06 = 411629
max. update norm 0.283898 at voxel (85, 41, 61)
iter. no. 50
data energy + w_reg * reg energy = 2231.8 + 0.2 * 2.0562e+06 = 413471
max. update norm 0.0406142 at voxel (127, 0, 104)
iter. no. 100
data energy + w_reg * reg energy = 2198.59 + 0.2 * 2.06222e+06 = 414643
max. update norm 0.0362966 at voxel (127, 0, 105)
iter. no. 150
data energy + w_reg * reg energy = 2185.54 + 0.2 * 2.06657e+06 = 415499
max. update norm 0.0380663 at voxel (127, 0, 106)
iter. no. 200
data energy + w_reg * reg energy = 2177.17 + 0.2 * 2.07041e+06 = 416259
max. update norm 0.0247796 at voxel (127, 0, 107)
iter. no. 250
data energy + w_reg * reg energy = 2171.46 + 0.2 * 2.0736e+06 = 416891
max. update norm 0.0132635 at voxel (127, 0, 109)
iter. no. 300
data energy + w_reg * reg energy = 2166.9 + 0.2 * 2.07631e+06 = 417429
max. update norm 0.00937067 at voxel (127, 0, 110)
iter. no. 350
data energy + w_reg * reg energy = 2163.27 + 0.2 * 2.07877e+06 = 417917
max. update norm 0.00749873 at voxel (127, 4, 108)
iter. no. 400
data energy + w_reg * reg energy = 2160.23 + 0.2 * 2.08057e+06 = 418274
max. update norm 0.00592503 at voxel (127, 0, 111)
iter. no. 450
data energy + w_reg * reg energy = 2157.65 + 0.2 * 2.08197e+06 = 418551
max. update norm 0.00498483 at voxel (127, 5, 107)
iter. no. 500
data energy + w_reg * reg energy = 2155.44 + 0.2 * 2.08312e+06 = 418779
max. update norm 0.00444591 at voxel (127, 5, 107)
iter. no. 550
data energy + w_reg * reg energy = 2153.42 + 0.2 * 2.08412e+06 = 418978
max. update norm 0.00403678 at voxel (127, 5, 107)
iter. no. 600
data energy + w_reg * reg energy = 2151.44 + 0.2 * 2.08464e+06 = 419080
max. update norm 0.00387362 at voxel (127, 4, 109)
iter. no. 650
data energy + w_reg * reg energy = 2149.45 + 0.2 * 2.08471e+06 = 419091
max. update norm 0.00376256 at voxel (127, 4, 109)
iter. no. 700
data energy + w_reg * reg energy = 2147.63 + 0.2 * 2.08478e+06 = 419103
max. update norm 0.0033257 at voxel (127, 4, 109)
iter. no. 750
data energy + w_reg * reg energy = 2146.04 + 0.2 * 2.08474e+06 = 419093
max. update norm 0.00286596 at voxel (127, 4, 109)
iter. no. 800
data energy + w_reg * reg energy = 2144.6 + 0.2 * 2.08459e+06 = 419062
max. update norm 0.00256356 at voxel (127, 4, 109)
iter. no. 850
data energy + w_reg * reg energy = 2143.31 + 0.2 * 2.08465e+06 = 419073
max. update norm 0.00232556 at voxel (127, 4, 109)
iter. no. 900
data energy + w_reg * reg energy = 2142.09 + 0.2 * 2.08484e+06 = 419110
max. update norm 0.00211953 at voxel (127, 5, 108)
iter. no. 950
data energy + w_reg * reg energy = 2140.93 + 0.2 * 2.08484e+06 = 419108
max. update norm 0.00199594 at voxel (127, 5, 108)
iter. no. 1000
data energy + w_reg * reg energy = 2139.8 + 0.2 * 2.08468e+06 = 419076
max. update norm 0.00183652 at voxel (127, 5, 108)
SOLVER REACHED MAX. NO. OF ITERATIONS WITHOUT CONVERGING
iter. no. 1
data energy + w_reg * reg energy = 6906.06 + 0.2 * 2.08467e+06 = 423839
max. update norm 0.510061 at voxel (88, 46, 63)
iter. no. 50
data energy + w_reg * reg energy = 2941.66 + 0.2 * 2.08882e+06 = 420706
max. update norm 0.0442168 at voxel (41, 78, 120)
iter. no. 100
data energy + w_reg * reg energy = 2714.22 + 0.2 * 2.08955e+06 = 420625
max. update norm 0.0231331 at voxel (39, 78, 127)
iter. no. 150
data energy + w_reg * reg energy = 2639.57 + 0.2 * 2.0898e+06 = 420599
max. update norm 0.0112255 at voxel (127, 2, 110)
iter. no. 200
data energy + w_reg * reg energy = 2600.44 + 0.2 * 2.08997e+06 = 420594
max. update norm 0.0100529 at voxel (127, 0, 110)
iter. no. 250
data energy + w_reg * reg energy = 2576.55 + 0.2 * 2.0899e+06 = 420556
max. update norm 0.01044 at voxel (127, 0, 110)
iter. no. 300
data energy + w_reg * reg energy = 2559.94 + 0.2 * 2.09032e+06 = 420623
max. update norm 0.00768468 at voxel (127, 0, 109)

[dgrzech]

alpha >= 0.1 will definitely be too large. the reconstructions on youtube were run with a step size of the order of 1e-4. for more on convergence see the discussion in #2

re translation/rotation the problem with rotations would largely be due to the fact that, in the presence of rotation, the canonical tsdf is too different from the live projective sdf. the entire deformation field is estimated starting at frame 0, meaning that for parts of the volume that only come into view later on, the field will be very inaccurate, which it's impossible to fix, and the problem only gets worse at subsequent frames. you can see the same issue in the snoopy video on youtube