Doesn't seem to build on arm64 Windows (Python 3.12)

[da-woods]

I was trying to set up tests for Cython on ARM-Windows, and as part of that it requires line_profiler as a test dependency.

When it tries to install that, line_profiler fails to build. Full output is at https://github.com/cython/cython/actions/runs/17012093880/job/48229495248?pr=7091 (but link won't be good forever). Representative output of the failure is:

"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\bin\HostARM64\ARM64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DCYTHON_TRACE=0 -Ipython25.pxd -Iline_profiler -IC:\hostedtoolcache\windows\Python\3.12.10\arm64\include -IC:\hostedtoolcache\windows\Python\3.12.10\arm64\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.26100.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tpline_profiler/_line_profiler.cpp /Fobuild\temp.win-arm64-cpython-312\Release\line_profiler\_line_profiler.obj
  _line_profiler.cpp
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(437): error C2664: '__int64 _InterlockedCompareExchange64_acq(volatile __int64 *,__int64,__int64)': cannot convert argument 1 from 'volatile uintptr_t *' to 'volatile __int64 *'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(437): note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or parenthesized function-style cast
  C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\intrin0.inl.h(175): note: see declaration of '_InterlockedCompareExchange64_acq'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(437): note: while trying to match the argument list '(volatile uintptr_t *, uintptr_t, uintptr_t)'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(444): error C2664: '__int64 _InterlockedCompareExchange64_rel(volatile __int64 *,__int64,__int64)': cannot convert argument 1 from 'volatile uintptr_t *' to 'volatile __int64 *'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(444): note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or parenthesized function-style cast
  C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\intrin0.inl.h(177): note: see declaration of '_InterlockedCompareExchange64_rel'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(444): note: while trying to match the argument list '(volatile uintptr_t *, uintptr_t, uintptr_t)'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(454): error C2664: '__int64 _InterlockedCompareExchange64(volatile __int64 *,__int64,__int64)': cannot convert argument 1 from 'volatile uintptr_t *' to 'volatile __int64 *'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(454): note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or parenthesized function-style cast
  C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\intrin0.inl.h(174): note: see declaration of '_InterlockedCompareExchange64'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(454): note: while trying to match the argument list '(volatile uintptr_t *, uintptr_t, uintptr_t)'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(475): error C2664: 'long _InterlockedCompareExchange_acq(volatile long *,long,long)': cannot convert argument 1 from 'volatile int *' to 'volatile long *'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(475): note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or parenthesized function-style cast
  C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\intrin0.inl.h(182): note: see declaration of '_InterlockedCompareExchange_acq'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(475): note: while trying to match the argument list '(volatile int *, int, int)'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(482): error C2664: 'long _InterlockedCompareExchange_rel(volatile long *,long,long)': cannot convert argument 1 from 'volatile int *' to 'volatile long *'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(482): note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or parenthesized function-style cast
  C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\intrin0.inl.h(184): note: see declaration of '_InterlockedCompareExchange_rel'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(482): note: while trying to match the argument list '(volatile int *, int, int)'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(492): error C2664: 'long _InterlockedCompareExchange(volatile long *,long,long)': cannot convert argument 1 from 'volatile int *' to 'volatile long *'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(492): note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or parenthesized function-style cast
  C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\intrin0.inl.h(168): note: see declaration of '_InterlockedCompareExchange'
  C:\hostedtoolcache\windows\Python\3.12.10\arm64\include\internal\pycore_atomic.h(492): note: while trying to match the argument list '(volatile int *, int, int)'
  line_profiler/_line_profiler.cpp(16819): warning C4244: 'argument': conversion from 'Py_ssize_t' to 'int', possible loss of data
  line_profiler/_line_profiler.cpp(28665): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(28666): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(28667): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(29168): warning C4244: '=': conversion from 'Py_ssize_t' to 'long', possible loss of data
  line_profiler/_line_profiler.cpp(31597): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(31604): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(32206): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(32207): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(32208): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(32386): warning C4244: '=': conversion from 'Py_ssize_t' to 'long', possible loss of data
  line_profiler/_line_profiler.cpp(32636): warning C4551: function call missing argument list
  line_profiler/_line_profiler.cpp(35412): warning C4551: function call missing argument list
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.44.35207\\bin\\HostARM64\\ARM64\\cl.exe' failed with exit code 2

I think that's happening is that you're including the internal header pycore_atomic.h transiently, and it doesn't work in C++ mode.

It's possible (but I'm not sure) that leaving the standard library atomics on might help.

line_profiler/line_profiler/c_trace_callbacks.h

Lines 30 to 32 in 3e357a2

	# ifdef HAVE_STD_ATOMIC
	# undef HAVE_STD_ATOMIC
	# endif

(Note that Github Actions has a windows-11-arm runner so this is an easy way to test)

[da-woods]

I should say - I'm not seriously trying to run line_profiler on Windows+arm and I can easily just skip the Cython tests that depend on it, so prioritise accordingly.

[TTsangSC]

The HAVE_STD_ATOMIC (and _PyGC_FINALIZED) hack was lifted from python/cpython#108216, because:

• Essentially, the compiler needs the memory layout of the return value of PyThreadState_GetInterpreter(), from which the internal state of sys.monitoring can be retrieved.
• Said layout is only available in some internal/pycore_interp.h because PyInterpreterState is mean to be opaque on the user-end.
• However, including the required header file has unseen ramifications in terms of the atomicity stuff on Linux in Python 3.12 (see pipeline #998), in turn requiring the hack.

A fix may require more platform-specific use of such macros but should be possible I guess... will try to just

#ifdef __linux__
#endif

the hack and see how it goes.

[TTsangSC]

Seems to be working somewhat, see pipeline #1113. Just gotta iron out some wrinkles.

EDIT I misread the earlier pipeline outputs and was too optimistic. Sorry about that.

[da-woods]

Thanks for fixing this!

[TTsangSC]

You're welcome! Between this and #394 it seems that there is an uptick in interest of using the repo with ARM. Now that we seem to have fixed it for Windows, I reckon that the fix for Linux will not require more work than we've done with #391 (famous last words)...

Anyway, for that I think it may help if there's preexisting demo for how far away we're from making it work (say, if it does work code-wise and we're just a cibuildwheel config short of full hassle-free installation). Do you happen to have run pipelines where your tests involving line_profiler were run on ARM-Linux (preferably both glibc and musl)? If so, can you point me to them so that I can have a gander? Cheers!

EDIT historically we seem to have had issues with musl (#290), so maybe that will be a blocker. But it will be interesting to see the pipelines nonetheless (if you do have them lying around of course).

[da-woods]

Currently Cython is only running a few tests on ARM (windows+Linux) just to avoid using too much CI time (although of course all OS X tests are now ARM I think). We're just doing Python 3.12 (which is a completely arbitrary choice, but one which happened to be the one that didn't work on Windows).

It looks to build correctly (from source rather than from a binary wheel) but being Python 3.12 we can't actually emit profiling events from Cython so we skip the tests. That probably suggests that it'd be better to test 3.13 on ARM just as a more representative build.

(Here's a recent run on Linux Py3.12: https://github.com/cython/cython/actions/runs/18436760964/job/52531270428, but I don't think it'd be too informative).

[TTsangSC]

Thanks for the info!

[da-woods]

I've tried it on Python 3.13 Ubuntu: https://github.com/cython/cython/actions/runs/18441384957/job/52542214885. It looks to install fine from source and run our tests correctly. So it looks like line profiler builds fine under both 3.12 and 3.13 (but can't profile Cython code on 3.12, although probably works fine on Python code).

I think on Linux the Python just uses GCC atomics consistently, so you don't even have the pycore_atomic problem that you did on Windows.

I don't know about musl/glibc I'm afraid. It looks like Cython itself provides separate glibc and musl wheels (for both arm64 and x64) but I don't really know the details/compatibility behind that.

---

The test run itself isn't too interesting - you can see it installs from a tar.gz file and that it runs the test called "line_profile_test" without complaining, but that's about it.

[TTsangSC]

Ah yes I've also been watching your cython/cython#7212. Thanks for the report, this gives me more confidence that it will probably not take a phenomenal amount of work to make it happen.

Aw I see. Guess that we'll just have to figure that out when we let cibuildwheels loose on musllinux. The hope is that given #290 is a year old and a version behind at this point, we (or cpython or musl) have inadvertently fixed whatever bug that caused the issue in the first place.

Anyway thanks for the input!