[Python-Dev] Python initialization and embedded Python (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Sat Nov 18 21:17:22 EST 2017

On 19 November 2017 at 01:45, Serhiy Storchaka <storchaka at gmail.com> wrote:

18.11.17 16:17, Nick Coghlan пише:

On 18 November 2017 at 10:01, Victor Stinner <victor.stinner at gmail.com> wrote:

I'm writing this email to ask if this change is an issue or not to embedded Python and the Python C API. Is it still possible to call "all" functions of the C API before calling PyInitialize()? It isn't technically permitted to call any of them, unless their documentation specifically says that calling them before PyInitialize is permitted (and that permission is only given for a select few configuration APIs in https://docs.python.org/3/c-api/init.html). The PyInitialize() is not complete. It mentions only PySetProgramName(), PySetPythonHome() and PySetPath(). But in other places it is documented that PySetStandardStreamEncoding(), PyImportAppendInittab(), PyImportExtendInittab() should be called before PyInitialize(). And the embedding examples call PyDecodeLocale() before PyInitialize(). PyMemRawMalloc(), PyMemRawFree() and PyInitFrozenExtensions() are called before PyInitialize() in PyFrozenMain(). Also these functions call PyMemRawStrdup(). Hence, the minimal set of functions that can be called before PyInitialize() is: * PySetProgramName() * PySetPythonHome() * PySetPath() * PySetStandardStreamEncoding() * PyImportAppendInittab() * PyImportExtendInittab() * PyDecodeLocale() * PyMemRawMalloc() * PyMemRawFree() * PyInitFrozenExtensions()

OK, in that case I think the answer to Victor's question is:

  1. Breaking calling Py_DecodeLocale() before calling Py_Initialize() is a compatibility break with the API implied by our own usage examples, and we'll need to revert the breakage for 3.7, and ensure at least one release's worth of DeprecationWarning before requiring either the use of an alternative API (where the caller controls the memory management), or else a new lower level pre-initialization API (i.e. making PyRuntime_Initialize a public API)
  2. We should provide a consolidated list of these functions in the C API initialization docs
  3. We should add more test cases to _testembed.c that ensure they all work correctly prior to Py_Initialize (some of them are already tested there, but definitely not all of them)

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list