The sampling results are integrated with PyTorch and incorporated into the automatic differentiation computation graph.

[Buantum]

TCI demonstrates remarkable capability in computing multivariate functional integrals, which I am eager to incorporate into my research endeavors. The current challenge lies in my loss function being composed of parameter-dependent multivariate integrals, presently addressed through Monte Carlo or Gauss-Legendre quadrature methods. These approaches permit direct manipulation within PyTorch classes, thereby enabling parameter optimization via PyTorch's computational graph.

Regarding the integration of Tensor Cross Interpolation with automatic differentiation, my conceptual framework involves:

Ensuring functional compatibility between TCI-defined and PyTorch-defined operations
Utilizing final index values obtained from TCI to perform resampling from PyTorch-defined functions
Defining compressed matrices within the PyTorch ecosystem
Ultimately executing integration operations (tensor contraction) within PyTorch
Should this functionality remain unimplemented, I would welcome the opportunity to contribute to its development. The TCI methodology holds significant promise for advancing variational methods in quantum physics research.

contact me with zhongwb5@mail.ustc.edu.cn

[shinaoka]

Thank you for the suggestion. Yes, it is possible. We are preparing a sample code; let's give ourselves some time to do so.

[YoshihiroMichishita]

Here is a provisional sample code. If there are any parts that are hard to read or unclear, feel free to ask. I plan to add it to the tutorial page eventually.

juliacall_test.ipynb.zip

[Buantum]

Wow！what a wonderfull work！

[Buantum]

Upon reviewing the code, I am truly amazed that it can optimize the integration of such an oscillatory function! I am thoroughly delighted with this outcome. A few days ago, I proposed a sampling method that requires defining the function as two distinct types. In the Julia package, is there a strict requirement for whether the function is defined in NumPy? Is it possible to perform a transformation on the output values of a Torch-defined function to generate a decomposition that produces TCI? When performing multivariate function variation with Torch, the function in question is often a neural network. Defining two identical neural networks places significant demands on memory, and the forward propagation also incurs computational time. However, if we simply match the output values of the network to the TCI data structure, we can leverage GPU acceleration for function computation, thereby reducing memory requirements.

thanks a lot!

[Buantum]

my computer is capable of executing the sample code for integration and decomposition. However, I am perplexed as to why this particular section is encountering a conversion issue.

————————————
It appears that qf is a NumPy array and cannot be directly executed in the current version of TCI I am using.

[YoshihiroMichishita]

Here are a few comments in response to your questions:

=========

1. The function passed to TCI must return a numpy.ndarray. This is because, on the Julia side, the return value is converted from a Python type to a Julia Array using PythonCall. However, since PythonCall does not support torch.Tensor, this conversion is only possible when the return type is a NumPy array.

2. It is possible to compute gradients by converting the output of a function defined in PyTorch.

3. Currently, it is not possible to compute gradients through TCI.crossinterpolate2 itself. Therefore, you need to first compute the pivots using a NumPy-based function, and then reconstruct the tensor train using the PyTorch version of the function to enable gradient computation.

=========
We cannot reproduce the error you have encountered...
We use python 3.11.9 and numpy 2.2.6.

[Buantum]

After rebuilding a new environment with python 3.11.9 and numpy 2.2.6, I have successfully run this code. Thanks!

"It is possible to compute gradients by converting the output of a function defined in PyTorch."
Does the TCI package, when sampling qf in the line tci_jl, ranks, error = crossinterpolate2(jl.Float64, qf, localdims_jl, maxbonddim=20, verbosity=1, loginterval=1), merely generate indices to invoke the qf function and await its return value, rather than reconstructing a function structure compatible with TCI based on qf? If it only generates indices, then it would be feasible to introduce a mechanism that transforms these indices into a torch-defined function, passes them into the torch-defined function, and subsequently converts the return value into the data structure required by TCI.

[YoshihiroMichishita]

Although we utilyze only the pivots(indices) in our sample code, tci_jl, ranks, error = crossinterpolate2(jl.Float64, qf, localdims_jl, maxbonddim=20, verbosity=1, loginterval=1) also reconstruct a function structure compatible with TCI based on qf.