[Python-Dev] No longer enable Py_TRACE_REFS by default in debug build (original) (raw)

Steve Dower steve.dower at python.org
Fri Apr 12 20:05:24 EDT 2019

On 12Apr.2019 1643, Nathaniel Smith wrote:

On Thu, Apr 11, 2019 at 8:26 AM Steve Dower <steve.dower at python.org> wrote:

On 10Apr2019 1917, Nathaniel Smith wrote: I don't know how many people use PyTRACEREFS, but if we can't find anyone on python-dev who uses it then it must be pretty rare. If dropping PyTRACEREFS would let us converge the ABIs and get rid of all the stuff above, then that seems like a pretty good trade! But maybe the Windows C runtime issue will foil this...

The very first question I asked was whether this would let us converge the ABIs, and the answer was "no".

Otherwise I'd have said go for it, despite the C runtime issues.

The reason we ship debug Python binaries is because debug builds use a different C Runtime, so if you do a debug build of an extension module you're working on it won't actually work with a non-debug build of CPython.

...But this is an important point. I'd forgotten that MSVC has a habit of changing the entire C runtime when you turn on the compiler's debugging mode. Technically they are separate options, but most project files are configured such that their Debug/Release switch affects both the compiler options (optimization) and the linker options (C runtime linkage). So how do other projects handle this? I guess historically the main target audience for Visual Studio was folks building monolithic apps, where you can just rebuild everything with whatever options you want, and compared to that Python extensions are messier. But Python isn't the only project in this boat. Do ruby, nodejs, R, etc., all provide separate debug builds with incompatible ABIs on Windows, and propagate that information throughout their module/package ecosystem?

Mostly I hear complaints about those languages not providing any help here. Python is renowned for having significantly better Windows support than any of them, so they're the wrong comparison to make in my opinion. Arguing that we should regress because other languages haven't caught up to us yet makes no sense.

The tools that are better than Python typically don't ship debug builds either, unless you specifically request them. But they also don't leak their implementation details all over the place. If we had a better C API, we wouldn't have users who needed to match ABIs.

For the most part, disabling optimizations in your own extension but using the non-debug ABI is sufficient, and if you're having to deal with other people's packages then maybe you don't have any choice (though I do know of people who have built debug versions of numpy before - turns out Windows developers are often just as capable as non-Windows developers when it comes to building things ;) ). And yes, they could also build CPython from source as well to get the debug ABI, or get the debug symbols, but I saw enough need that I decided it was worth the effort to just solve that problem. 250k downloads a month is enough to justify it for me.

Not to bring the packaging discussions to another venue, but maybe this is yet another area we need to stop pretending that we're able to solve every single problem with just the tools we already have available? People who want debug builds of packages can build them themselves, even numpy and scipy, they don't need us to preemptively do all their work for them. But we can (and should) help short-cut unnecessary effort or research by providing helpful tools and instruction.

Cheers, Steve

More information about the Python-Dev mailing list