Issue 13431: Pass context information into the extension module init function (original) (raw)

Created on 2011-11-19 09:16 by scoder, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (10)
msg147931 - (view)	Author: Stefan Behnel (scoder) *	Date: 2011-11-19 09:16
This is a follow-up to issue 13429, which deals with setting __file__ on newly created extension modules earlier than it is currently the case. Currently, the module init function of extension modules lacks a way to find out the context in which it is being imported, e.g. its package path or its location in the file system. This makes it tricky for extension modules to do things like loading package resources or using relative imports at init time. This can be fixed by allowing the init function to take a context struct as argument, which would contain object pointers to the FQ package name and file path, and potentially other information. I think this would be backwards compatible to existing code, because C should allow the caller of the init function to pack additional arguments on the stack that the called function simply doesn't care about. From CPython 3.3 on, however, new and updated code could benefit from this feature.
msg147933 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2011-11-19 10:16
This approach is not correct C. You may pass an "incorrect" number of arguments only if the function is declared with an ellipsis, else the behavior is undefined (6.5.5.2p6). Your proposed approach works with most implementations of C, but Python shouldn't deliberately rely on undefined behavior.
msg147934 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2011-11-19 10:17
FWIW, I think that a PEP will be needed to redesign the module initialization for Python 4. Hopefully, this time people will then identify all requirements before the PEP is accepted (which unfortunately didn't happen for PEP 3121).
msg147935 - (view)	Author: Stefan Behnel (scoder) *	Date: 2011-11-19 10:47
Yes, that's unfortunate. I found the same paragraph in section 6.5.2.2p6 of the C99 standard now, so it seems that this idea isn't suitable for the Py3.x series. There's no Python 4 target version in the bug tracker, BTW. :)
msg147970 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2011-11-19 20:31
I really do propose to close this issue as "won't fix". By the time Python 4 comes along, things will have changed a lot; it's not clear that the value of keeping it open will outweigh the cost of keeping it open.
msg147976 - (view)	Author: Jesús Cea Avión (jcea) *	Date: 2011-11-20 05:38
What about providing a function that "init" would call to get this information, instead of a "hidden" parameter?.
msg147979 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2011-11-20 09:40
> What about providing a function that "init" would call to get this information, instead of a "hidden" parameter?. How would the function that init calls itself get the information, without a hidden parameter?
msg147998 - (view)	Author: Jesús Cea Avión (jcea) *	Date: 2011-11-20 15:55
Python interpreter already have that information internally. The initial proposal was to pass it to the init as an additional parameter. My suggestion is to export it as a new function that the init can call if VERSION >=3.3.
msg148000 - (view)	Author: Stefan Behnel (scoder) *	Date: 2011-11-20 16:29
The problem with having that information "internally" is that it's currently stored in local variables in the call chain from the dynamic library loader. Passing that information on into a callable function, without passing it as an argument into the init function, means, that it needs to get stored away in some global place, with all the drawbacks that this induces. That's what Martin was referring to. I agree with Martin that the idea of adding a parameter to the module init function is not worth pursuing before Python 4, so I'm closing this bug.
msg148009 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2011-11-20 21:29
> Python interpreter already have that information internally. The > initial proposal was to pass it to the init as an additional > parameter. My suggestion is to export it as a new function that the > init can call if VERSION >=3.3. Jesus: Please try to come up with a patch if you don't see the problem. Trust Stefan and me that there actually *is* a problem with your approach.