[Python-Dev] Make extension module initialisation more like Python module initialisation (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Tue Aug 6 07:35:28 CEST 2013

On 6 August 2013 15:02, Stefan Behnel <stefan_ml at behnel.de> wrote:

Alternatives I see:

1) Expose a struct that points to the extension module's PyModuleDef struct and the init function and expose that struct instead. 2) Expose both the PyModuleDef and the init function as public symbols. 3) Provide a public C function as entry point that returns both a PyModuleDef pointer and a module init function pointer. 4) Change the minit function pointer in PyModuleDefbase from func(void) to func(PyObject*) iff the PyModuleDef struct is exposed as a public symbol. 5) Duplicate PyModuleDef and adapt the new one as in 4). Alternatives 1) and 2) only differ marginally by the number of public symbols being exposed. 3) has the advantage of supporting more advanced setups, e.g. heap allocation for the PyModuleDef struct. 4) is a hack and has the disadvantage that the signature of the module init function cannot be stored across reinitialisations (PyModuleDef has no "flags" or "state" field to remember it). 5) would fix that, i.e. we could add a proper pointer to the new module init function as well as a flags field for future extensions. A similar effect could be achieved by carefully designing the struct in 1). I think 1-3 are all reasonable ways to do this, although I don't think 3) will be necessary. 5) would be a clean fix, but has the disadvantage of duplicating an entire struct just to change one field in it. I'm currently leaning towards 1), with a struct that points to PyModuleDef, module init function and a flags field for future extensions. I understand that this would need to become part of the stable ABI, so explicit extensibility is important to keep up backwards compatibility. Opinions?

I believe a better option would be to migrate module creation over to a dynamic PyModule_Slot and PyModule_Spec approach in the stable ABI, similar to the one that was defined for types in PEP 384.

A related topic is that over on import-sig, we're currently tinkering with the idea of changing the way Python module imports happen to include a separate "ImportSpec" object (exact name TBC). The spec would contain preliminary info on all of the things that the import system can figure out without actually importing the module. That list includes all the special attributes that are currently set on modules:

__loader__
__name__
__package__
__path__
__file__
__cached__

(Note that the attributes on the spec may not be the same as those in the module's own namespace - for example, name and spec.name would differ in a module executed with -m, and path and spec.path would end up differing in packages that directly manipulated their path attribute during init execution)

The intent is to clean up some of the ad hoc hackery that was needed to make PEP 420 work, and reduce the amount of duplicated functionality needed in loader implementations.

If you wanted to reboot this thread on import-sig, that would probably be a good thing :)

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list