[Pythonmac-SIG] Python 2.3.4 Installation (original) (raw)

tmk lists at netelligent.biz
Sun Nov 28 23:59:34 CET 2004

Thanks a lot for the thorough (and quick!) explanation Bob.

This is much appreciated.

= tmk =

On Nov 28, 2004, at 17:45, Bob Ippolito wrote:

On Nov 28, 2004, at 10:53 AM, tmk wrote:

First I must confess that I don't really understand the details of the problem of building extensions with multiple versions of Python installed on Panther... Very few people do :) Out of curiosity I just checked and the lib/python2.3/config/Makefile file from my compiled from source python 2.3.4 don't appear to be patched with the patch below. The correct linker settings are not adopted by any mainline version of Python 2.3. I had thought they were in 2.3.4, but they're not. Only Python 2.4 builds correctly by default. Also, Apple's build of Python 2.3.3 in the WWDC 2004 preview <http://www.opensource.apple.com/darwinsource/WWDC2004/python-11/> adopts a similar method to this patch, but slightly modified so that it does not depend on MACOSXDEPLOYMENTTARGET being set: (this is what a good /System/Library/Frameworks/Python.framework/Versions/2.3/lib/ python2.3/config/Makefile should look like, starting on or around line 98) LDSHARED= env MACOSXDEPLOYMENTTARGET=10.3 (CC)(CC) (CC)(LDFLAGS) -bundle -undefined dynamiclookup BLDSHARED= env MACOSXDEPLOYMENTTARGET=10.3 (CC)(CC) (CC)(LDFLAGS) -bundle -undefined dynamiclookup Am I correct in assuming that this is ok and that the patch below is only meant for the Apple-installed python 2.3? Unfortunately not, it recently came to my attention that it is VERY IMPORTANT TO PATCH THIS MAKEFILE. Here is some more information (listing what will happen with various permutations of common Python installations). I hope to never have to repeat this again by email :) Versions of Mac OS X prior to 10.3: This patch isn't at all compatible because dyld did not support this feature until Darwin 7. That sucks, but there's little that can be done about that! = Scenario 1 = [a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, unpatched Building extensions with [a] no longer works, because -framework Python locates [b]. Therefore, it actually ends up building extensions only compatible with [b]. Building extensions with [b] correctly creates extensions, but are only compatible with [b]. Loading correctly built extensions built for [a] from [b] or vice versa will crash the interpreter. = Scenario 2 = [a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, unpatched Building extensions with [a] will build extensions compatible with [b] (or any other Python 2.3.x, even non-framework builds). Building extensions with [b] correctly creates extensions, but are only compatible with [b]. Loading correctly built extensions built for [b] from [a] will crash the interpreter. = Scenario 3 = [a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, patched Building extensions with [a] no longer works, because -framework Python locates [b]. Therefore, it actually ends up building extensions only compatible with [b]. Building extensions with [b] will build extensions compatible with [a] (or any other Python 2.3.x, even non-framework builds). Loading correctly built extensions built for [a] from [b] will crash the interpreter. = Scenario 4 = [a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, patched Building extensions with [a] will build extensions compatible with [b] (or any other Python 2.3.x, even non-framework builds) and vice versa. = Scenario 5 = [a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, unpatched [c] Python 2.4.x in /Library Building extensions with [a] no longer works, because -framework Python locates [c]. These extensions are not compatible with ANY Python (built with 2.3 headers, linked with 2.4). Building extensions with [b] no longer works, because -framework Python locates [c]. These extensions are not compatible with ANY Python (built with 2.3 headers, linked with 2.4). Loading correctly built extensions built for [a] from [b] or vice versa will crash the interpreter. = Scenario 6 = [a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, unpatched [c] Python 2.4.x in /Library Building extensions with [b] no longer works, because -framework Python locates [c]. These extensions are not compatible with ANY Python (built with 2.3 headers, linked with 2.4). Loading correctly built extensions built for [b] from [a] will crash the interpreter. = Scenario 7 = [a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, patched [c] Python 2.4.x in /Library Building extensions with [a] no longer works, because -framework Python locates [c]. These extensions are not compatible with ANY Python (built with 2.3 headers, linked with 2.4). Loading correctly built extensions built for [a] from [b] will crash the interpreter. = Scenario 8 = [a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, patched [c] Python 2.4.x in /Library Building extensions with [b] will build extensions compatible with [a] (or any other Python 2.3.x, even non-framework builds) and vice versa. ... So obviously, it's very important to patch your Python! The sooner you patch, the less extensions you will build that are tied to a particular installation location. Note that nearly all precompiled extensions you will find for Python 2.3.0 are generally of the unpatched variety, so they aren't compatible with other installations of Python 2.3.x. Considering [c] can screw up the whole mix, even though it doesn't have it's own problems... it turns out that there is a fundamental flaw in the way gcc's compiler and linker finds headers and dylibs from frameworks! It can't take version numbers into account! The runtime does just fine, but link time does not.. This is pretty bad! It's not solvable without patching gcc either, because from the source files you say <FrameworkName/HeaderName.h> but HeaderName.h doesn't live in a directory named FrameworkName, it lives in a directory named Headers, so it's not possible to specify an -I that works correctly if you use this feature. There are a few workarounds to versioned framework issues, and they all suck. [1] Keep versioned frameworks in separate places and use -F appropriately. [2] Use -I with only "inside-framework" header names (i.e. <Python.h> instead of <Python/Python.h>) and link directly to the framework's dylib as if it were an .o file (i.e. /Library/Frameworks/Python.framework/Versions/2.3/Python instead of -framework Python) [3] Name your frameworks with the version number in them instead of using the versioning capability of frameworks (i.e. Python23.framework) Note that Python 2.4 uses a hybrid of [1] and [2]. When building the Python framework, it uses -F into the build directory, so it always links the interpreter correctly. All sources expect the headers to be available via "Python.h" and it uses an appropriate -I to make this work. When building extensions, it uses -undefined dynamiclookup, so it doesn't link directly to any Python. Also in a separate but related e-mail you also suggested that one would "also need to set the "MACOSXDEPLOYMENTTARGET" environment variable to "10.3" when using this patch."

Could you please elaborate on how/where one should set that environment variable (I set it in ~/.MacOSX/environment.plist). Setting it there is fine. You could also, instead, adopt a better patch like Apple has as stated above. -bob

More information about the Pythonmac-SIG mailing list