[Python-Dev] No longer enable Py_TRACE_REFS by default in debug build (original) (raw)

Victor Stinner vstinner at redhat.com
Fri Apr 12 21:35:42 EDT 2019

The very first question I asked was whether this would let us converge the ABIs, and the answer was "no".

The answer is yes and it's my primary goal.

See my first email: "This change makes the debug build ABI closer to the release build ABI".

To be honest, I am now lost in this long thread :-) I don't recall why I started to argue so much about the memory footprint, it's not really the main point here.

Victor

Otherwise I'd have said go for it, despite the C runtime issues. I don't see that in the thread... just Victor saying he isn't sure whether there might be other ABI incompatibilities lurking that he hasn't found yet. Did I miss something? I'm mostly interested in this because of the possibility of converging the ABIs. If you think that the C runtime thing isn't a blocker for that, then that's useful information. Though obviously we still need to figure out whether there are any other blockers :-).

>>>> The reason we ship debug Python binaries is because debug builds use a >>>> different C Runtime, so if you do a debug build of an extension module >>>> you're working on it won't actually work with a non-debug build of CPython. >>> >>> ...But this is an important point. I'd forgotten that MSVC has a habit >>> of changing the entire C runtime when you turn on the compiler's >>> debugging mode. >> >> Technically they are separate options, but most project files are >> configured such that their Debug/Release switch affects both the >> compiler options (optimization) and the linker options (C runtime linkage). > > So how do other projects handle this? I guess historically the main > target audience for Visual Studio was folks building monolithic apps, > where you can just rebuild everything with whatever options you want, > and compared to that Python extensions are messier. But Python isn't > the only project in this boat. Do ruby, nodejs, R, etc., all provide > separate debug builds with incompatible ABIs on Windows, and propagate > that information throughout their module/package ecosystem?

Mostly I hear complaints about those languages not providing any help here. Python is renowned for having significantly better Windows support than any of them, so they're the wrong comparison to make in my opinion. Arguing that we should regress because other languages haven't caught up to us yet makes no sense. The tools that are better than Python typically don't ship debug builds either, unless you specifically request them. But they also don't leak their implementation details all over the place. If we had a better C API, we wouldn't have users who needed to match ABIs. Do you happen to have a list of places where the C API leaks details of the underlying CRT? (I'm mostly curious because whenever I've looked my conclusion was essentially: "Well....... I don't see any places that are definitely broken, so maybe mixing CRTs is fine? but I have zero confidence that I caught everything, so probably better to play it safe?". At least on py3 – I know the py2 C API was definitely broken if you mixed CRTs, because of the exposed FILE*.) For the most part, disabling optimizations in your own extension but using the non-debug ABI is sufficient, and if you're having to deal with other people's packages then maybe you don't have any choice (though I do know of people who have built debug versions of numpy before - turns out Windows developers are often just as capable as non-Windows developers when it comes to building things ;) I'm not sure why you think I was implying otherwise? I'm sorry if you thought I was attacking your users or something. I did say that I thought most users downloading the debug builds were probably confused about what they were actually getting, but I didn't mean because they were stupid Windows users, I meant because the debug builds are so confusing that even folks on the Python core team are confused about what they're actually getting. -n -- Nathaniel J. Smith -- https://vorpus.org

Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com

-- Night gathers, and now my watch begins. It shall not end until my death. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20190413/8d193b73/attachment-0001.html>

More information about the Python-Dev mailing list