Add PEP 757 functions/types (int export/import) to the limited API 3.15?

[skirpichev]

Export API: https://docs.python.org/3.14/c-api/long.html#export-api
PyLongWriter API: https://docs.python.org/3.14/c-api/long.html#pylongwriter-api

We already have PyLong_FromNativeBytes()/PyLong_AsNativeBytes() (much more complex internally than PEP 757 API) to import/export big integers, but these functions aren't suitable for fast conversions between different big integer types. I think it's major obstacle for (eventually) using Limited API in the gmpy2.

Vote:

• @erlend-aasland
• @encukou
• @serhiy-storchaka (inactive)
• @vstinner
• @zooba

[vstinner]

Export API: https://docs.python.org/3.14/c-api/long.html#export-api
PyLongWriter API: https://docs.python.org/3.14/c-api/long.html#pylongwriter-api

This API was added to Python 3.14. You propose to add it to the limited C API of Python 3.15. IMO it makes sense.

The API was designed with the limited C API in mind. The API has 2 structures (PyLongLayout and PyLongExport), that's the main fragile part for the limited C API. If these structures have to change in the future, we may need to leave them unchanged, and add new structures. My hope is that PEP 757 design is future proof and was properly designed by the C API Working Group and the Steering Council.

---

I didn't follow PEP 757 usage outside CPython. Is it used in gmpy2, SAGE and Python-FLINT? If yes, does the API fit their needs?

[encukou]

+1

If these structures have to change in the future, we may need to leave them unchanged, and add new structures.

More precisely: we'd need to leave the structures unchanged, change the functions to convert & do extra allocations, and add new structs/functions.
(But we'd need to do that anyway for API compatibility, even if we don't add these to Limited API. Limited API makes the "old" stuff harder to deprecate/remove.)

Is it used in gmpy2, SAGE and Python-FLINT? If yes, does the API fit their needs?

I trust @skirpichev that it's useful for gmpy2.

SAGE uses PyLong internals, but from quick grepping around it looks like they could port to this API (and PyLong_*NativeBytes). I'm not sure about their needs around performance and compatibility with old Python versions.

fmpz doesn't seem too concerned about conversion performance: https://github.com/flintlib/python-flint/blob/e330bf5e071e08b4b9bd96b435a76206a0d1b5d5/src/flint/types/fmpz.pxd#L11-L19

[skirpichev]

I didn't follow PEP 757 usage outside CPython. Is it used in gmpy2, SAGE and Python-FLINT?

So far it's used by gmpy2 and python-gmp.

fmpz doesn't seem too concerned about conversion performance: https://github.com/flintlib/python-flint/blob/e330bf5e071e08b4b9bd96b435a76206a0d1b5d5/src/flint/types/fmpz.pxd#L11-L19

See plans to change that: flintlib/python-flint#159

There are no backports of this API to PyPy or old Cython versions, so such cases require a different code paths.

[vstinner]

Ping @serhiy-storchaka

[encukou]

ping @serhiy-storchaka

[vstinner]

ping @serhiy-storchaka

[skirpichev]

I think it's major obstacle for (eventually) using Limited API in the gmpy2.

Ah, sorry. I forgot another one: using heap types. My micro-benchmarks shows up to x1.25 slowdown from switching from static to heap type in case of small integers (1-3 limbs) :(

[encukou]

@serhiy-storchaka is inactive and counts as abstaining.
Please expose these in the limited API!

[vstinner]

PR adding the PEP 757 API to the stable ABI: python/cpython#143906.