Remove semi-public C-API for doing arithmetic with Py_complex type (original) (raw)

(Opened per a suggestion in the C-API WG issue.)

Historically, in CPython we have functions like _Py_c_sum and _Py_c_prod to do arithmetic with Py_complex (a custom type for complex numbers). Recent pr python/cpython#124829 added support for mixed-mode arithmetic and this API was extended with functions like _Py_cr_prod (i.e. when one argument is a real number). (JFR: this new C-API is hidden for now.)

It was suggested by @vstinner in python/cpython#124829 (comment) either to remove this from the public API or rename properly (e.g. like Py_complex_add() and Py_complex_add_real(), in the GNU GSL style).

Of course, people could use PyNumber_*() API. Main motivation to provide a different C-API for doing correct complex arithmetic — is speed. But with python/cpython#124829 — complex arithmetic in CPython follows to most implementations of the C99+ Annex G. So, users can use native double _Complex type instead of the Py_complex. Removing low-level arithmetic API & Py_complex struct possibly open a door to eventually switch CPython to use native complex type (double _Complex) internally (like for floats).

Proposal:

  1. deprecate _Py_c_*() functions
  2. add PyComplex_From/As*() methods to import/export from/to native complex type. (conditionally)
  3. deprecate Py_complex struct, PyComplex_FromCComplex() and PyComplex_AsCComplex().
  4. (in even more distant future) switch to using double _Complex internally

I see several arguments to keep these functions (from high to low priority):

  1. special arithmetic functions will make sense if we go beyond most Annex G implementations, i.e. add support for pure-imaginary numbers. This was recently suggested here. Probably, we can conclude that this proposal (which I’m fan of) has no enough support.
  2. not all Tier 1 PEP 11’s platforms have Annex G support: it’s Windows with MSVC. I don’t know if here are some plans to add such support. On another hand, this platform has a different native complex type, _Dcomplex, with some arithmetic primitives to do math with. People could use this type or some external library (like GSL) to do complex arithmetic.
  3. current Annex G has no special case for real/complex division (which we have in our implementation since python/cpython#124829). So, current CPython complex arithmetic has a small incompatibility with the C standard. This is something, going to be fixed by N3460 in upcoming standard.

CPython issue: Make _Py_c_sum(), _Py_c_diff(), etc (elementary operations on Py_complex) - part of the public API · Issue #128813 · python/cpython · GitHub
C-API WG issue: Remove public low-level API functions for Py_complex? · Issue #56 · capi-workgroup/decisions · GitHub

CAM-Gerlach (C.A.M. Gerlach) April 28, 2025, 7:11pm 2

@skirpichev Might this be better off moved to Core Development ? At least historically Ideas has typically been reserved for more speculative, PEP-sized ideas to improve the Python language and stdlib, while Core Development is for discussion of more practical immediate-term issues. Given the issue history you highlight, this is quite mature and ready to execute whatever decision is made, and smaller in scope and impact than a PEP. It will also reach a broader yet more focused audience of core developers and technically-inclined contributors than Ideas , which many/most core devs shy away from due to the prevalence of half-baked, impractical and repeated proposals.

skirpichev (Sergey B Kirpichev) April 29, 2025, 4:33am 3

Maybe, I was thinking about it first.
Though, last time I put post to that category it was moved to the ideas. So, I don’t trust my own judgment.

encukou (Petr Viktorin) April 29, 2025, 6:55am 4

deprecate _Py_c_*() functions
deprecate Py_complex struct, PyComplex_FromCComplex() and PyComplex_AsCComplex().

I’d recommend for soft-deprecation. There’s no security issues and not much maintenance burden beyond the initial implementation.
(IMO, we should keep our implementations at least until MSVC supports Annex G. There’s not much trouble in exposing the implementations; and if/when we do switch, the functions become one-liners with existing tests.)

add PyComplex_From/As*() methods to import/export from/to native complex type. (conditionally)

By “native complex type” you mean double _Complex, right?
If MSVC ever adds double _Complex, we’ll want these functions to use that.

Removing low-level arithmetic API & Py_complex struct possibly open a door to eventually switch CPython to use native complex type (double _Complex) internally (like for floats).

We don’t need to remove the existing struct & functions for that.
What we do need is make the PyComplexObject struct opaque (or just deprecate/remove it from public headers), and require existing users to go through the existing PyComplex_FromCComplex/PyComplex_AsCComplex.
(Your proposal to deprecate Py_complex doesn’t say what to do with PyComplexObject…)

skirpichev (Sergey B Kirpichev) April 29, 2025, 9:31am 5

Yes.

Sure, but for a while — these functions will be available only if platform supports Annex G.

Proposed API:

PyObject * PyComplex_FromNativeComplex(double _Complex v);
double _Complex PyComplex_AsNativeComplex(PyObject *op)

Ok, soft deprecation does make sense for me.

If so, we should add _Py_c_*() functions as public, together with new mixed-mode API (_Py_cr_*()) as public and soft-deprecate all that stuff.

Then the remaining thing to decide will be naming scheme. I suggest GSL-like style (as in the post). There was suggestion to use PyComplex_-prefix, but this looks wrong for me (as this API works with more low-level objects).

Well, in current docs we have here same situation as with the PyFloatObject: the PyComplexObject structure declared in docs, but no fields are documented. Probably, this should stay that way. The Py_complex struct will be soft-deprecated.

encukou (Petr Viktorin) April 29, 2025, 10:38am 6

Sounds good! (Maybe DoubleComplex rather than NativeComplex, if it’s already time for naming details?)

I don’t think we need to do that. Soft-deprecation is there to avoid breaking existing code; if we want people to use such functions we shouldn’t (soft-)deprecate them.

PyComplexObject.cval is public API that needs deprecation to remove. PEP 387 says:

Note that if something is not documented at all, it is not automatically considered private.

skirpichev (Sergey B Kirpichev) April 29, 2025, 4:26pm 7

I’m not sure that we want. This API seems redundant, unless we aren’t going beyond complex arithmetic, as specified in the Annex G of the upcoming C standard (without imaginary type).

Soft-deprecation does make sense, as it’s not hard to support that API and it might help poor Windows people.

Ok, then this field should be renamed to underscore-prefixed and the cval alias — documented and deprecated. This will disentangle Py_complex struct from the PyComplexObject.

h-vetinari (H. Vetinari) April 30, 2025, 4:36am 8

As hopeless as it may seem, people with a Microsoft account should upvote this issue to help tip the scales for MSVC to finally implement this.

skirpichev (Sergey B Kirpichev) April 30, 2025, 6:26am 9

I’m not sure which issue link is canonical. Here is another one.

Using a different compiler seems to be not a solution, as MSVC is a build requirement on Windows.

h-vetinari (H. Vetinari) April 30, 2025, 7:11am 10

Funnily enough, I have comments on both of these[1] :sweat_smile:

This topic came up recently: Introducing clang-cl as an alternative MSBuild backend

  1. I won’t comment on the UX of devcommunity. ↩︎

skirpichev (Sergey B Kirpichev) April 30, 2025, 9:00am 11

That looks interesting, but as @chris-eibl said: “it is a drop-in compiler for MSVC and brings also its own linker. But it uses the MSVC headers and links to the MSVC (static/dynamic) libraries.” The MSVC ships own “complex.h”, which is not compatible with the Annex G. So, I doubt this open door for support native complex types on Windows.

Though, using a different compiler on Windows seems possible, e.g. in an MSYS2 environment.

steve.dower (Steve Dower) April 30, 2025, 9:28am 12

Scales are indeed tipped by votes on issues like this.

I’m sure they’ll get duped eventually, which should combine the votes (it used to, at least), but voting for both won’t hurt!