Dynet GPU installation failure on Amazon Linux instances

[g-karthik]

Hi, I was able to install Dynet fine for CPU usage in an Amazon Linux p3.16x instance. For GPU usage, I ran the following command:

BACKEND=cuda pip install git+https://github.com/clab/dynet#egg=dynet

and I get the following error:

Looking in indexes: https://pypi.org/simple, https://pip.repos.neuron.amazonaws.com
Collecting dynet
  Cloning https://github.com/clab/dynet to /tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456
  Running command git clone --filter=blob:none -q https://github.com/clab/dynet /tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456
  Resolved https://github.com/clab/dynet to commit c418b09dfb08be8c797c1403911ddfe0d9f5df77
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in /home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages (from dynet) (1.21.5)
Requirement already satisfied: cython in /home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages (from dynet) (0.29.24)
Building wheels for collected packages: dynet
  Building wheel for dynet (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/ec2-user/anaconda3/envs/pytorch_p38/bin/python3.8 /home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmp5pz9ri55
       cwd: /tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456
  Complete output (33 lines):
  running bdist_wheel
  running build
  INFO:root:CMAKE_PATH='/home/ec2-user/anaconda3/envs/pytorch_p38/bin/cmake'
  INFO:root:MAKE_PATH='/usr/bin/make'
  INFO:root:MAKE_FLAGS='-j 64'
  INFO:root:EIGEN3_INCLUDE_DIR='/tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456/build/py3.8-64bit/eigen'
  INFO:root:EIGEN3_DOWNLOAD_URL='https://github.com/clab/dynet/releases/download/2.1/eigen-b2e267dc99d4.zip'
  INFO:root:CC_PATH='/home/ec2-user/anaconda3/envs/pytorch_p38/bin/x86_64-conda-linux-gnu-cc'
  INFO:root:CXX_PATH='/home/ec2-user/anaconda3/envs/pytorch_p38/bin/x86_64-conda-linux-gnu-c++'
  INFO:root:SCRIPT_DIR='/tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456'
  INFO:root:BUILD_DIR='/tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456/build/py3.8-64bit'
  INFO:root:INSTALL_PREFIX='/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/../../..'
  INFO:root:PYTHON='/home/ec2-user/anaconda3/envs/pytorch_p38/bin/python3.8'
  Traceback (most recent call last):
    File "/home/ec2-user/anaconda3/envs/pytorch_p38/bin/cmake", line 5, in <module>
      from cmake import cmake
  ModuleNotFoundError: No module named 'cmake'
  x86_64-conda-linux-gnu-c++ (crosstool-NG 1.24.0.133_b0863d8_dirty) 9.3.0
  Copyright (C) 2019 Free Software Foundation, Inc.
  This is free software; see the source for copying conditions.  There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  
  INFO:root:Creating build directory /tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456/build/py3.8-64bit
  INFO:root:Fetching Eigen...
  INFO:root:Unpacking Eigen...
  INFO:root:Configuring...
  Traceback (most recent call last):
    File "/home/ec2-user/anaconda3/envs/pytorch_p38/bin/cmake", line 5, in <module>
      from cmake import cmake
  ModuleNotFoundError: No module named 'cmake'
  /tmp/pip-build-env-5oy0twd_/overlay/lib/python3.8/site-packages/setuptools/dist.py:771: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
    warnings.warn(
  error: /home/ec2-user/anaconda3/envs/pytorch_p38/bin/cmake /tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456 -DCMAKE_INSTALL_PREFIX='/home/ec2-user/anaconda3/envs/pytorch_p38/lib/python3.8/site-packages/../../..' -DEIGEN3_INCLUDE_DIR='/tmp/pip-install-ao0f03yi/dynet_9d4d210ad7d640e0823785d1c0e51456/build/py3.8-64bit/eigen' -DPYTHON='/home/ec2-user/anaconda3/envs/pytorch_p38/bin/python3.8' -DBACKEND='cuda'
  ----------------------------------------
  ERROR: Failed building wheel for dynet
Failed to build dynet
ERROR: Could not build wheels for dynet, which is required to install pyproject.toml-based projects

I started python with this env activated and ran the command from cmake import cmake and there was no error. So it's not clear to me why it says it could not find a module named cmake in the pytorch_p38 conda env.

Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:59:51) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cmake import cmake
>>>

Any ideas would be greatly appreciated! @neubig