The following program works fine under Mac OS 10.5.x. But in 10.6, it crashes Python, and displays Apple's crash reporter dialog. I've tried it on Python 2.6 and 2.5, both the 64-bit and 32-bit versions. The crash seems to happen any time urllib.urlopen is done in a thread. The same thing happens if I do it in a BaseHTTPServer.HTTPServer, using SocketServer.ThreadingMixIn. #!/usr/bin/python import threading, urllib class MyThread (threading.Thread): def run(self): c = urllib.urlopen("http://www.google.com") MyThread().start()
It seems that CoreFoundation doesn't like being loaded on a secondairy thread: #0 0x00007fff8301bb90 in __CFInitialize () #1 0x00007fff5fc0d5ce in __dyld__ZN16ImageLoaderMachO11doImageInitERKN11ImageLoader11LinkContextE () #2 0x00007fff5fc0d607 in __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkCon textE () #3 0x00007fff5fc0bcec in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj () #4 0x00007fff5fc0bc9d in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj () #5 0x00007fff5fc0bda6 in __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextE () #6 0x00007fff5fc08fbb in __dyld_dlopen () #7 0x00007fff84902d40 in dlopen () #8 0x000000010070a0d2 in py_dl_open () #9 0x00000001000bb4ed in PyEval_EvalFrameEx () (More stack frames follow...)
This probably means that the ctypes code in urllib.py needs to be ported to C, although we won't know if that helps until said C code is written :-( Doing that would be a good idea anyway, while trying to create a workaround I noticed that the ctypes code is invalid anyway because there are not enough ctypes annotations when running in 64-bit mode (where sizeof(int) != sizeof(long)) In 32-bit mode you can avoid the crash by calling urllib.urlopen or urllib.proxy_bypass once on the main thread, but you have to use a qualified name ("localhost.localdomain", not "localhost").
The attached patch seems to fix the issue, but needs further testing. Warning: the patch is not entirely clean, the patch contains an unrelated change to setup.py. The patch replaces some code that uses ctypes to read configuration data using the SystemConfiguration framework by a regular extension. With this patch the crasher goes away, I still have to do more testing and proper checking of the C code. The patch is for the trunk, I expect it will cleanly port to 2.6. It will also need porting to 3.x later on.
The attached patch was not 100% correct in the error handling in the set_proxies functions. That is fixed in the actually commit. Committed as r74962 (trunk), r74963 (2.6).
