Binary incompatibility with Raysect when building with numpy 2.0

[vsnever]

NumPy 2.0 was recently released. Although Cherab builds without errors with NumPy 2.0, trying to run any tests results in errors such as:

ImportError: Failed to import test module: cherab.core
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/unittest/loader.py", line 470, in _find_test_path
    package = self._get_module_from_name(name)
  File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/runner/work/cherab-core/cherab-core/cherab/core/__init__.py", line 20, in <module>
    from .atomic import *
  File "/home/runner/work/cherab-core/cherab-core/cherab/core/atomic/__init__.py", line 22, in <module>
    from .interface import AtomicData
  File "cherab/core/atomic/interface.pyx", line 1, in init cherab.core.atomic.interface
    # Copyright 2016-2022 Euratom
  File "cherab/core/atomic/zeeman.pyx", line 1, in init cherab.core.atomic.zeeman
    # Copyright 2016-2018 Euratom
  File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/raysect/core/__init__.py", line 30, in <module>
    from .ray import *
  File "raysect/core/ray.pyx", line 1, in init raysect.core.ray
  File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/raysect/core/math/__init__.py", line 39, in <module>
    from .statsarray import StatsBin, StatsArray1D, StatsArray2D, StatsArray3D
  File "raysect/core/math/statsarray.pyx", line 1, in init raysect.core.math.statsarray
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Looks like NumPy 2.0 is binary incompatible with NumPy 1.x, which Raysect is built against. Until we have Raysect binaries built with NumPy 2.0, I suggest limiting the NumPy version to 1.x in setup.py, requirements.txt and ci.yml.

[jacklovell]

Yeah, this is sensible. We build Cherab wheels for PyPI using Numpy 1.x too (using the now-deprecated oldest-supported-numpy package), so will have the same problem. We can switch to specifying numpy~=2 for Python 3.9 and later since Numpy 2.0 produces a backward-compatible build, but for 3.7 (which UKAEA is still stuck on) and 3.8 would still need oldest-supported-numpy.

[vsnever]

When the Raysect binaries for NumPy 2.0 become available, we'll need to make the following changes:

pyproject.toml:

[build-system]
requires = [
    "setuptools>=62.3",
    "oldest-supported-numpy; python_version < '3.9'",
    "numpy~=2.0; python_version >= '3.9'",
    "cython~=3.0"
    "raysect==0.8.1",
]
build-backend="setuptools.build_meta"

requirements.txt:

cython~=3.0
numpy>=1.14,<2.0; python_version < '3.9'
numpy~=2.0; python_version >= '3.9'
scipy
matplotlib
raysect==0.8.1

setup.py:

    install_requires=[
        "cython~=3.0",
        "numpy>=1.14,<2.0; python_version < '3.9",
        "numpy~=2.0; python_version >= '3.9'"
        "scipy",
        "matplotlib",
        "raysect==0.8.1",
    ],

[munechika-koyo]

I found the incompatibility with numpy~=2.0 in the test script:

core/cherab/tools/tests/test_voxels.py

Lines 283 to 284 in 26595cb

	dx = coords[:, 0].ptp()
	dy = coords[:, 1].ptp()

.ptp() method seems to be deprecated, so it should be changed like:

dx, dy = np.ptp(coords, axis=0)

This change should not affect the package that depends on numpy<2.0, so would you mind if I apply the fix PR?

[munechika-koyo]

setup.py:

    install_requires=[
        "cython~=3.0",
        "numpy>=1.14,<2.0; python_version < '3.9",
        "numpy~=2.0; python_version >= '3.9'"
        "scipy",
        "matplotlib",
        "raysect==0.8.1",
    ],

Is it unnecessary to add cython for runtime?

[munechika-koyo]

This issue seems to be resolved by #486, so we should close it.