Issue on Control Point Position Optimization

[mikelyfei]

Hi Sebastian,

In pytests/tests/tf/test_tf_optimization.py line 106, derivative_tf_indices is defined as follows:

During training, I find that the position of each control point does not change. How can I optimize the position of each control point during training? What should I do with this and other parameters? Thank you.

[shamanDevel]

Hi, these are not actually the derivatives or the values you want to optimize. For forward-mode autodiff, you need to assign "optimization slots" for each variable you want to compute gradients for. This is because forward-mode autodiff scales linearly with the number of parameters and thus you want to minimize them. Only the parameters with a derivative-index >= 0 (and all must be unique) are optimized.

The actual TF values are in initial_tf, these are the values that are optimized.

[mikelyfei]

Hi Sebastian, may I know if I understand this correctly: if I let derivative_tf_indices = torch.tensor([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24]], dtype=torch.int32), both the RGBA values and positions of all control points, including the first and last points, of initial_tf are optimized? Is there a demo code in this repo for position optimization?

Besides, since I want to optimize the positions of the control points of initial_tf, the positions may be not in ascending order and even not positive after optimization. What can I do to make the optimized transfer function still valid? Does it affect gradient computation if I sort the rows of current_tf during optimization? Thank you!