Issue 663074: codec registry and Python embedding problem (original) (raw)

Created on 2003-01-06 12:17 by lemburg, last changed 2022-04-10 16:06 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
python-2.3-codecs.patch niemeyer,2003-03-01 08:07
Messages (11)
msg13908 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2003-01-06 12:17
Found by Hartmut Ring: #include <Python.h> int main() { for (int i=0; i<2; i++) { Py_Initialize(); PyRun_SimpleString("a = u'\xe4'.encode('Latin-1')"); Py_Finalize(); } return 0; } First try runs fine, second try gives: Traceback (most recent call last): File "", line 1, in ? LookupError: no codec search functions registered: can't find encoding
msg13909 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2003-01-06 12:18
Logged In: YES user_id=38388 This is due to a flag used in codecs.c: /* Flag used for lazy import of the standard encodings package */ static int import_encodings_called = 0; The solution is to reset this flag in Py_Finalize().
msg13910 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2003-01-22 10:07
Logged In: YES user_id=21627 I think the various global variables maintained in codecs.c (_PyCodec_SearchCache, import_encodings_called, ...) need to become part of the interpreter state. Otherwise, a codec registered in one interpreter will be found in another. Also, it appears that import_encodings_called can be eliminated. Instead, a NULL value of _PyCodec_SearchPath could be used to indicate whether initialization has taken place.
msg13911 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2003-01-22 12:12
Logged In: YES user_id=38388 Good suggestion. Do you have time to cook up a patch ?
msg13912 - (view) Author: Gustavo Niemeyer (niemeyer) * (Python committer) Date: 2003-02-12 13:13
Logged In: YES user_id=7887 As discussed in python-dev, I'll work on that.
msg13913 - (view) Author: Gustavo Niemeyer (niemeyer) * (Python committer) Date: 2003-03-01 08:07
Logged In: YES user_id=7887 Here is a proposed solution. It has been tested with the standard regression tests, and in an environment with multiple interpreters.
msg13914 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2003-03-01 14:41
Logged In: YES user_id=6380 I can't review this beyond "it doesn't break the tests for me"; I suggest to assign to MAL for review. One note: I got a compiler warning implicit declaration of function `_PyCodecRegistry_Init' to fix, add static int _PyCodecRegistry_Init(void); /* Forward */ somewhere to the top of codecs.c.
msg13915 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2003-03-04 10:07
Logged In: YES user_id=38388 Same comments as Guido, plus: * in the interpreter state struct you should add the new fields to the end, not in the middle * the finalization phase is moved to a later point in the process; this could cause problems when finalizing codecs -- I guess we'll just have to see whether we get bug reports related to this; perhaps it's better to ZAP the dicts before ZAPping builtins etc. ?! Great work, Gustavo. Please check the patch in with the above changes. Thanks.
msg13916 - (view) Author: Gustavo Niemeyer (niemeyer) * (Python committer) Date: 2003-03-07 17:20
Logged In: YES user_id=7887 No problems. If you don't mind, I'd just to have a confirmation: are you sure I should move the new fields to the end, and leave a conditional member in the middle?
msg13917 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2003-03-07 17:36
Logged In: YES user_id=38388 Oh, I see what you mean. In that case, it's OK to add them just before the optional parts.
msg13918 - (view) Author: Gustavo Niemeyer (niemeyer) * (Python committer) Date: 2003-03-19 00:50
Logged In: YES user_id=7887 Commited as: Include/pystate.h: 2.23 Misc/NEWS: 1.699 Python/codecs.c: 2.21 Python/pystate.c: 2.24 Python/pythonrun.c: 2.182 Thanks for your support!
History
Date User Action Args
2022-04-10 16:06:06 admin set github: 37727
2003-01-06 12:17:28 lemburg create