build: add freethreaded python ci and wheels

[lgray]

• ci
• wheels
• tests

Will add wheels when they're fixed up by #283

[lgray]

Ah - right we are blocked because pydantic is not updated to be compatible with 3.13t at all.

Nope it's in beta now!

[lgray]

refreshing this PR since there's now beta2 of the freethreaded pydantic

[lgray]

@mgorny @nsmith- @henryiii

I noticed when it's doing the freethreaded windows build that the MS linker is looking for python313.lib, rather than python313t.lib that's specified as PythonLib in CMake.

Results in the error (I think):

LINK : fatal error LNK1104: cannot open file 'python313.lib' [C:\Users\runneradmin\AppData\Local\Temp\tmpu4rnegbz\build\_core.vcxproj]

I think that's the only blocker on windows.

[mgorny]

That looks like https://gitlab.kitware.com/cmake/cmake/-/issues/26016, which is supposedly fixed but I'm not sure in which CMake version.

[mgorny]

That looks like https://gitlab.kitware.com/cmake/cmake/-/issues/26016, which is supposedly fixed but I'm not sure in which CMake version.

Ah, sorry, now I see 3.30.3 — so supposedly it should work here.

[mgorny]

Sorry for thinking loudly. I see that CMake is doing the right thing:

2025-03-17T20:15:06.0037191Z   -- Found PythonInterp: C:/hostedtoolcache/windows/Python/3.13.1/x64-freethreaded/python.exe (found suitable version "3.13.1", minimum required is "3.7")
2025-03-17T20:15:06.0038736Z   -- Found PythonLibs: C:/hostedtoolcache/windows/Python/3.13.1/x64-freethreaded/libs/python313t.lib

So looks like python313.lib is coming from elsewhere. pybind11 perhaps?

[lgray]

No problem - think out loud all you want. I'm just not sure where to look :-)

[henryiii]

You need to use the modern FindPython, not the old one. FindPythonLibs / FindPythonInterp was "removed" (sort of) in CMake 3.27, so that's not what 3.30.3 is referring to.

[henryiii]

Right above here

correctionlib/CMakeLists.txt

Lines 16 to 21 in ea10902

	include(FetchContent)
	FetchContent_Declare(pybind11
	SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/pybind11
	CMAKE_ARGS "-DBUILD_TESTING=OFF -DPYBIND11_NOPYTHON=ON"
	)
	FetchContent_MakeAvailable(pybind11)

you should set(PYBIND11_FINDPYTHON ON). pybind11 3.0 will change the default.

[lgray]

Thanks @henryiii, giving it a try.

[lgray]

3.13t manylinux wheels are dying on tests because of missing awkward 3.13t wheel. Otherwise they build fine!

[lgray]

@henryiii I guess the best way to test this in freethreaded mode for races and such would to first just try pytest-parallel? Probably mark the dask tests to not be run in parallel?

[henryiii]

Yes, I believe so.

[lgray]

Nice - parallel tests seem to go, except on windows. Looks like some issue with parallel processing on windows to begin with.

[lgray]

I don't really understand this failure in macos.

Must be a threadsafety thing, but then why not in ubuntu?

[lgray]

Oh wow, it's definitely a thread safety issue (it passed in this latest commit!). Yikes!

[lgray]

@nsmith- Any first ideas on what needs a mutex around it?

[lgray]

For reference the error that crops up in the MT tests is:

self = <correctionlib.highlevel.CorrectionSet object at 0x2a9ea3d0150>

    def __iter__(self) -> Iterator[str]:
>       return iter(self._base)
E       TypeError: Object of type 'iterator' is not an instance of 'iterator'

/Library/Frameworks/PythonT.framework/Versions/3.13/lib/python3.13t/site-packages/correctionlib/highlevel.py:401: TypeError
________________________________ test_evaluator ________________________________

    def test_evaluator():
        with pytest.raises(RuntimeError):
            cset = core.CorrectionSet.from_string("{")
    
        with pytest.raises(RuntimeError):
            cset = core.CorrectionSet.from_string("{}")
    
        with pytest.raises(RuntimeError):
            cset = core.CorrectionSet.from_string('{"schema_version": "blah"}')
    
        with pytest.raises(RuntimeError):
            cset = core.CorrectionSet.from_string('{"schema_version": 2, "description": 3}')
    
        cset = core.CorrectionSet.from_string(
            '{"schema_version": 2, "description": "something", "corrections": []}'
        )
        assert cset.schema_version == 2
        assert cset.description == "something"
    
        cset = wrap(
            schema.Correction(
                name="test corr",
                version=2,
                inputs=[],
                output=schema.Variable(name="a scale", type="real"),
                data=1.234,
            )
        )
>       assert set(cset) == {"test corr"}
E       TypeError: Object of type 'iterator' is not an instance of 'iterator'

tests/test_core.py:47: TypeError

[nsmith-]

My best guess would be something related to this:

correctionlib/src/python.cc

Lines 106 to 108 in 093ce46

	.def("__iter__", [](const CorrectionSet &v) {
	return py::make_key_iterator(v.begin(), v.end());
	}, py::keep_alive<0, 1>())

is not threadsafe.

[lgray]

Digging around there seems to be something suspicious w.r.t. the py::keep_alive but I've not found an accurate description. make_key_iterator itself seems to be fine.

Will continue digging.

[lgray]

Darn, after all that the error is still there! @henryiii any ideas?

[nsmith-]

Is this saying there is no need to take the gil for the whole module? I think that's not true.
For example, CorrectionSet.from_file will need to create something reference-counted.

[lgray]

It appears to be bad naming for "free threading support" https://pybind11.readthedocs.io/en/stable/advanced/misc.html#free-threading-support

I can look into it further to see what it implies.

[henryiii]

That’s what Python calls it. If you need a lock, take a lock (like std::mutex), but that lock isn’t the GIL when there isn’t one.

[ikrommyd]

Should probably use https://github.com/Quansight-Labs/pytest-run-parallel. pytest-parallel is unmaintained and archived.

[ikrommyd]

I added this in a previous PR which needs changing too:

correctionlib/pyproject.toml

Line 99 in 122f3e5

skip = ["pp*-*", "cp313t-*", "cp314t-*"]

According to cibuildwheel, 314t are automatically built while 313t need to be requested.

[lgray]

Yep, still getting:

>       assert set(cset) == {"test corr"}
               ^^^^^^^^^
E       TypeError: Object of type 'iterator' is not an instance of 'iterator'

and I have absolutely no idea.

[lgray]

and dask still doesn't have freethreaded.

[ikrommyd]

@nsmith- @lgray should we just have 3.14 release and maybe have a free-threaded patch later?

[nsmith-]

I'll try to do a correctionlib sprint for some of the other feature requests soon. Maybe threaded can be part of that.
So in short, sure we could cut a release just for 3.14 support

[nsmith-]

@lgray there are a few unrelated improvements, would you be willing to spin those off to a new PR?