[Python-Dev] No longer enable Py_TRACE_REFS by default in debug build (original) (raw)

Nathaniel Smith njs at pobox.com
Wed Apr 10 22:17:28 EDT 2019

On Wed, Apr 10, 2019 at 1:50 PM Steve Dower <steve.dower at python.org> wrote:

On 10Apr2019 1227, Nathaniel Smith wrote: > On Wed, Apr 10, 2019, 04:04 Victor Stinner <vstinner at redhat.com_ _> <mailto:vstinner at redhat.com>> wrote: > I don't think that I ever used sys.getobjects(), whereas many projects > use gc.getobjects() which is also available in release builds (not > only in debug builds). > > > Can anyone explain what pydebug builds are... for? Confession: I've > never used them myself, and don't know why I would want to. > > (I have to assume that most of Steve's Windows downloads are from folks > who thought they were downloading a python debugger.) They're for debugging :) In general, debug builds are meant for faster inner-loop development. They generally do incremental builds properly and much faster by omitting most optimisations, which also enables source mapping to be more accurate when debugging. Assertions are typically enabled so that you are notified when a precondition is first identified rather than when it causes the crash (compiling these out later means you don't pay a runtime cost once you've got the inputs correct - generally these are used for developer-controlled values, rather than user-provided ones). So the idea is that you can quickly edit, build, debug, fix your code in a debug configuration, and then use a release configuration for the actual released build. Full release builds may take 2-3x longer than full debug builds, given the extra effort they make at optimisation, and very often can't do minimal incremental builds at all (so they may be 10-100x slower if you only modified one source file). But because the builds behave functionally equivalently, you can iterate with the faster configuration and get more done.

Sure, I'm familiar with the idea of debug and optimization settings in compilers. I build python with custom -g and -O flags all the time. (I do it by setting OPT when running configure.) It's also handy that many Linux distros these days let you install debug metadata for all the binaries they ship – I've used that when debugging third-party extension modules, to get a better idea of what was happening when a backtrace passes through libpython. But --with-pydebug is a whole other thing beyond that, that changes the ABI, has its own wheel tags, requires special cases in packages that use ctypes to access PyObject* internals, and appears to be almost entirely undocumented.

It sounds like --with-pydebug has accumulated a big grab bag of unrelated features, mostly stuff that was useful at some point for some CPython dev trying to debug CPython itself? It's clearly not designed with end users as the primary audience, given that no-one knows what it actually does and that it makes third-party extensions really awkward to run. If that's right then I think Victor's plan of to sort through what it's actually doing makes a lot of sense, especially if we can remove the ABI breaking stuff, since that causes a disproportionate amount of trouble.

The reason we ship debug Python binaries is because debug builds use a different C Runtime, so if you do a debug build of an extension module you're working on it won't actually work with a non-debug build of CPython.

...But this is an important point. I'd forgotten that MSVC has a habit of changing the entire C runtime when you turn on the compiler's debugging mode. (On Linux, we generally don't bother rebuilding the C runtime unless you're actually debugging the C runtime, and anyway if you do want to switch to a debug version of the C runtime, it's ABI compatible so your program binaries don't have to be rebuilt.)

Is it true that if the interpreter is built against ucrtd.lib, and an extension module is built against ucrt.lib, then they'll have incompatible ABIs and not work together? And that this detail is part of what's been glommed together into the "d" flag in the soabi tag on Windows?

Is it possible for the Windows installer to include PDB files (/Zi /DEBUG) to allow debuggers to understand the regular release executable? (That's what I would have expected to get if I checked a box labeled "Download debug binaries".)

-n

-- Nathaniel J. Smith -- https://vorpus.org

More information about the Python-Dev mailing list