pip install has issues detecting dependencies on conda

[masadcv]

If a system has pytorch pre-installed using conda - then poetry (pyproject.toml) does not detect it and installs pytorch using pip

This causes issues especially if there is a mismatch between CUDA version for pytorch from pip and the CUDA version available on system

To reproduce:

• Start a docker imagedocker run --gpus all -it --rm nvcr.io/nvidia/pytorch:21.10-py3
• Install FastGeodis using pip: pip install FastGeodis

The above gives the following error:

{rocessing FastGeodis-1.0.0rc2.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: torch>=1.8.0 in /opt/conda/lib/python3.8/site-packages (from FastGeodis==1.0.0rc2) (1.9.0a0+2ecb2c7)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.8/site-packages (from torch>=1.8.0->FastGeodis==1.0.0rc2) (3.7.4.3)
Building wheels for collected packages: FastGeodis
  Building wheel for FastGeodis (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for FastGeodis (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [69 lines of output]
      setup.py with torch 1.11.0+cu102
      BUILD_CPP=True, BUILD_CUDA=True, TORCH_VERSION=11100.
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-38
      creating build/lib.linux-x86_64-cpython-38/FastGeodis
      copying FastGeodis/__init__.py -> build/lib.linux-x86_64-cpython-38/FastGeodis
      running build_ext
      /tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/torch/_masked/__init__.py:223: UserWarning: Failed to initialize NumPy: numpy.core.multiarray failed to import (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:68.)
        example_input = torch.tensor([[-3, -2, -1], [0, 1, 2]])
      Traceback (most recent call last):
        File "/opt/conda/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/opt/conda/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/conda/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 244, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 229, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 123, in <module>
          setup(
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 177, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/command/build.py", line 24, in run
          super().run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
          self.build_extensions()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 410, in build_extensions
          self._check_cuda_version()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 787, in _check_cuda_version
          raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
      RuntimeError:
      The detected CUDA version (11.3) mismatches the version that was used to compile
      PyTorch (10.2). Please make sure to use the same CUDA versions.
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for FastGeodis
Failed to build FastGeodis
ERROR: Could not build wheels for FastGeodis, which is required to install pyproject.toml-based projects

[masadcv]

This is a known issue with poetry pyproject.toml and conda - see: cupy/cupy#4727

A possible workaround can be found here: cupy/cupy#4727 (comment)
is to install on conda systems where pytorch is pre-installed (e.g. docker images) using --no-build-isolation:

pip install FastGeodis --no-build-isolation

[Kin-Zhang]

Not sure why pip install --no-build-isolation have following issue:

(seflow) ➜  DeFlow git:(feature/benchmarks) ✗ pip install FastGeodis --no-build-isolation          
Collecting FastGeodis
  Using cached FastGeodis-1.0.5-cp38-cp38-linux_x86_64.whl
Requirement already satisfied: torch>=1.5.0 in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from FastGeodis) (2.0.0)
Requirement already satisfied: filelock in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (3.15.4)
Requirement already satisfied: typing-extensions in /home/kin/.local/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (4.12.2)
Requirement already satisfied: sympy in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (1.12.1)
Requirement already satisfied: networkx in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (3.1)
Requirement already satisfied: jinja2 in /home/kin/.local/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (3.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in /home/kin/.local/lib/python3.8/site-packages (from jinja2->torch>=1.5.0->FastGeodis) (2.1.1)
Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from sympy->torch>=1.5.0->FastGeodis) (1.3.0)
Installing collected packages: FastGeodis
Successfully installed FastGeodis-1.0.5
(seflow) ➜  DeFlow git:(feature/benchmarks) ✗ python -c "import FastGeodis"              
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages/FastGeodis/__init__.py", line 34, in <module>
    import FastGeodisCpp
ImportError: /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages/FastGeodisCpp.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c107WarningC1ENS_7variantIJNS0_11UserWarningENS0_18DeprecationWarningEEEERKNS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb

But when I install 1.0.4 no issue above.

[masadcv]

Hi @Kin-Zhang,
Thanks for reporting. This seems to me like a setup issue similar to reported here: #60

It seems like you are installing FastGeodis from cache, which means it is using a previously built version from the cache that possibly was built with another setup on your machine.

Could you try to force a rebuild:

pip install FastGeodis --no-build-isolation --no-cache-dir