[Pythonmac-SIG] Python 2.3.4 Installation (original) (raw)

Bob Ippolito bob at redivi.com
Sun Nov 28 17:45:17 CET 2004

On Nov 28, 2004, at 10:53 AM, tmk wrote:

First I must confess that I don't really understand the details of the problem of building extensions with multiple versions of Python installed on Panther...

Very few people do :)

Out of curiosity I just checked and the lib/python2.3/config/Makefile file from my compiled from source python 2.3.4 don't appear to be patched with the patch below.

The correct linker settings are not adopted by any mainline version of
Python 2.3. I had thought they were in 2.3.4, but they're not. Only
Python 2.4 builds correctly by default. Also, Apple's build of Python
2.3.3 in the WWDC 2004 preview
<http://www.opensource.apple.com/darwinsource/WWDC2004/python-11/>
adopts a similar method to this patch, but slightly modified so that it
does not depend on MACOSX_DEPLOYMENT_TARGET being set:

(this is what a good
/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/ config/Makefile should look like, starting on or around line 98)

LDSHARED= env MACOSX_DEPLOYMENT_TARGET=10.3 (CC)(CC) (CC)(LDFLAGS) -bundle
-undefined dynamic_lookup BLDSHARED= env MACOSX_DEPLOYMENT_TARGET=10.3 (CC)(CC) (CC)(LDFLAGS) -bundle
-undefined dynamic_lookup

Am I correct in assuming that this is ok and that the patch below is only meant for the Apple-installed python 2.3?

Unfortunately not, it recently came to my attention that it is VERY
IMPORTANT TO PATCH THIS MAKEFILE. Here is some more information
(listing what will happen with various permutations of common Python
installations). I hope to never have to repeat this again by email :)

Versions of Mac OS X prior to 10.3: This patch isn't at all compatible
because dyld did not support this feature until Darwin 7. That sucks,
but there's little that can be done about that!

= Scenario 1 =

[a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, unpatched

Building extensions with [a] no longer works, because -framework Python
locates [b]. Therefore, it actually ends up building extensions only
compatible with [b]. Building extensions with [b] correctly creates extensions, but are only
compatible with [b]. Loading correctly built extensions built for [a] from [b] or vice versa
will crash the interpreter.

= Scenario 2 =

[a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, unpatched

Building extensions with [a] will build extensions compatible with [b]
(or any other Python 2.3.x, even non-framework builds). Building extensions with [b] correctly creates extensions, but are only
compatible with [b]. Loading correctly built extensions built for [b] from [a] will crash
the interpreter.

= Scenario 3 =

[a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, patched

Building extensions with [a] no longer works, because -framework Python
locates [b]. Therefore, it actually ends up building extensions only
compatible with [b]. Building extensions with [b] will build extensions compatible with [a]
(or any other Python 2.3.x, even non-framework builds). Loading correctly built extensions built for [a] from [b] will crash
the interpreter.

= Scenario 4 =

[a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, patched

Building extensions with [a] will build extensions compatible with [b]
(or any other Python 2.3.x, even non-framework builds) and vice versa.

= Scenario 5 =

[a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, unpatched [c] Python 2.4.x in /Library

Building extensions with [a] no longer works, because -framework Python
locates [c]. These extensions are not compatible with ANY Python
(built with 2.3 headers, linked with 2.4). Building extensions with [b] no longer works, because -framework Python
locates [c]. These extensions are not compatible with ANY Python
(built with 2.3 headers, linked with 2.4). Loading correctly built extensions built for [a] from [b] or vice versa
will crash the interpreter.

= Scenario 6 =

[a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, unpatched [c] Python 2.4.x in /Library

Building extensions with [b] no longer works, because -framework Python
locates [c]. These extensions are not compatible with ANY Python
(built with 2.3 headers, linked with 2.4). Loading correctly built extensions built for [b] from [a] will crash
the interpreter.

= Scenario 7 =

[a] Python 2.3.0 in /System/Library, unpatched [b] Python 2.3.x in /Library, patched [c] Python 2.4.x in /Library

Building extensions with [a] no longer works, because -framework Python
locates [c]. These extensions are not compatible with ANY Python
(built with 2.3 headers, linked with 2.4). Loading correctly built extensions built for [a] from [b] will crash
the interpreter.

= Scenario 8 =

[a] Python 2.3.0 in /System/Library, patched [b] Python 2.3.x in /Library, patched [c] Python 2.4.x in /Library

Building extensions with [b] will build extensions compatible with [a]
(or any other Python 2.3.x, even non-framework builds) and vice versa.

...

So obviously, it's very important to patch your Python! The sooner you
patch, the less extensions you will build that are tied to a particular
installation location. Note that nearly all precompiled extensions you
will find for Python 2.3.0 are generally of the unpatched variety, so
they aren't compatible with other installations of Python 2.3.x.

Considering [c] can screw up the whole mix, even though it doesn't have
it's own problems... it turns out that there is a fundamental flaw in
the way gcc's compiler and linker finds headers and dylibs from
frameworks! It can't take version numbers into account! The runtime
does just fine, but link time does not.. This is pretty bad! It's not
solvable without patching gcc either, because from the source files you
say <FrameworkName/HeaderName.h> but HeaderName.h doesn't live in a
directory named FrameworkName, it lives in a directory named Headers,
so it's not possible to specify an -I that works correctly if you use
this feature.

There are a few workarounds to versioned framework issues, and they all
suck.

[1] Keep versioned frameworks in separate places and use -F
appropriately. [2] Use -I with only "inside-framework" header names (i.e. <Python.h>
instead of <Python/Python.h>) and link directly to the framework's
dylib as if it were an .o file (i.e.
/Library/Frameworks/Python.framework/Versions/2.3/Python instead of
-framework Python) [3] Name your frameworks with the version number in them instead of
using the versioning capability of frameworks (i.e. Python23.framework)

Note that Python 2.4 uses a hybrid of [1] and [2]. When building the
Python framework, it uses -F into the build directory, so it always
links the interpreter correctly. All sources expect the headers to be
available via "Python.h" and it uses an appropriate -I to make this
work. When building extensions, it uses -undefined dynamic_lookup, so
it doesn't link directly to any Python.

Also in a separate but related e-mail you also suggested that one would "also need to set the "MACOSXDEPLOYMENTTARGET" environment variable to "10.3" when using this patch."

Could you please elaborate on how/where one should set that environment variable (I set it in ~/.MacOSX/environment.plist).

Setting it there is fine. You could also, instead, adopt a better
patch like Apple has as stated above.

-bob

More information about the Pythonmac-SIG mailing list