#236: Use pyzoltan to call Zoltan as a comparison algorithm in LBAF

[marcinwrobel1986]

THIS PR:

• added description in README.md regarding PyZoltan installation
• tested PyZoltan installation
• added docker usage to README.md
• removed not needed files from docker image

Closes #236 #275

[marcinwrobel1986]

@ppebay Please take a look and once the installation is ok, you can change the assignee and continue with this PR

[ppebay]

Thanks @marcinwrobel1986 but two questions:

• is the Zoltan binary actually linked?
• can you please add it to the dependencies and make sure it gets tested as part of CI? This will indeed now become an optional dependency.

[marcinwrobel1986]

Thanks @marcinwrobel1986 but two questions:

• is the Zoltan binary actually linked?
• can you please add it to the dependencies and make sure it gets tested as part of CI? This will indeed now become an optional dependency.

Ok, good remark, we could test it in CI. For Ubuntu there is an easy way to install Zoltan with the package manager, but it would make sense if we do it in "universal" way, for both MacOS and Linux.

[ppebay]

Two questions:
1- do we really have to Git clone pyzoltan? This seems rather convoluted to do that in addition to pip installing it.
2- I am still getting the following error:

Installing collected packages: pyzoltan
  Running setup.py install for pyzoltan ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for pyzoltan did not run successfully.
  │ exit code: 1
  ╰─> [11 lines of output]
      --------------------------------------------------------------------------------
      Unable to run mpic++ correctly, skipping parallel build
      --------------------------------------------------------------------------------
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/tmp/pip-install-2hnqbo4t/pyzoltan_b10f764ad6514ad88fed0d7b8c7f2b09/setup.py", line 346, in <module>
          setup_package()
        File "/private/tmp/pip-install-2hnqbo4t/pyzoltan_b10f764ad6514ad88fed0d7b8c7f2b09/setup.py", line 298, in setup_package
          raise RuntimeError(
      RuntimeError: There are no extension modules, Nothing to do!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pyzoltan

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Would this occur only on macOS?

[marcinwrobel1986]

Ad. 1. We are cloning PyZoltan, just to use the script provided there
Ad. 2. Please make sure you have OpenMPI installed
Here is the documentation
Some lines regardin MacOSX:
https://pyzoltan.readthedocs.io/en/latest/start.html#installing-mpi4py-and-zoltan-on-os-x

[ppebay]

I have MPICH installed... so this pyzoltan can only work with OpenMPI?

[marcinwrobel1986]

I have MPICH installed... so this pyzoltan can only work with OpenMPI?

I am not an expert, but you can noticed that they want one to install OpenMPI
sudo brew install open-mpi

[ppebay]

Yes I am installing it (via MacPorts actually) but I find it surprising that it would require a specific implementation of MPI (OpenMPI rather than MPICH).

[ppebay]

Still not working on macOS, even with OpenMPI installed:

python setup.py install
--------------------------------------------------------------------------------
Unable to run mpic++ correctly, skipping parallel build
--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "setup.py", line 423, in <module>
    setup_package()
  File "setup.py", line 369, in setup_package
    raise RuntimeError(
RuntimeError: There are no extension modules, Nothing to do!

[marcinwrobel1986]

Philippe, maybe if you have it installed, then just:

# One needs to export ZOLTAN as an environment variable
export ZOLTAN=$INSTALL_PREFIX

ZOLTAN environment variable is very important, and it's used when installing PyZoltan

One needs to install PyZoltan requirements with (this is from PyZoltan repo):

pip install -r <some-directory>/pyzoltan/requirements.txt

Last step is to install PyZoltan itself:

pip install pyzoltan --no-build-isolation

[ppebay]

Yes yes I did all of the above. Still failing. I think it's fair to say that their install process does not work on macOS.

[ppebay]

@lifflander if you have a chance, could you please try to check if pyzoltan's instructions are working with your version of macOS --- as you are probably using Homebrew rather than MacPorts?

Thanks!

[lifflander]

@lifflander if you have a chance, could you please try to check if pyzoltan's instructions are working with your version of macOS --- as you are probably using Homebrew rather than MacPorts?

Thanks!

I was able to get pyzoltan installed on the Mac. I think the key is having a proper version of OpenMPI installed. I ended up building everything with clang-5 and OpenMPI 3.

[marcinwrobel1986]

Hello @ppebay I added readme instruction for Docker usage.

[ppebay]

Please @tlamonthezie have a look at this