| msg96085 - (view) |
Author: Sridhar Ratnakumar (srid) |
Date: 2009-12-07 23:33 |
| When trying to build 2.7a1 I ran into the following problem on MacOSX 10.4 (Tiger): $ ./configure --enable-framework --enable-universalsdk [...] $ make [...] gcc -c -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk - fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes - I. -IInclude -I./Include -DPy_BUILD_CORE -o Python/pymath.o Python/pymath.c /var/tmp//ccGDYNCp.s:36:Invalid mnemonic 'fnstcw' /var/tmp//ccGDYNCp.s:56:Invalid mnemonic 'fldcw' lipo: can't open input file: /var/tmp//cc4UBrqs.out (No such file or directory) make: *** [Python/pymath.o] Error 1 |
|
|
| msg96086 - (view) |
Author: Sridhar Ratnakumar (srid) |
Date: 2009-12-07 23:34 |
| This error only occurs when I specify --enable-framework and --enable- universalsdk to ./configure. |
|
|
| msg96114 - (view) |
Author: Ned Deily (ned.deily) *  |
Date: 2009-12-08 08:55 |
| Works for me on 10.4.11 PPC (G3) machine rather than Intel (which shouldn't make a difference). But from your config.log, the gcc version is older than what I have on 10.4 (build 5250 vs build 5370). Suggest ensuring you have the most recent Xcode for 10.4 installed. |
|
|
| msg96126 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-12-08 12:54 |
| Do you get the same problem with py3k? |
|
|
| msg96127 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-08 13:01 |
| Could you also post the generated pyconfig.h? |
|
|
| msg96135 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-08 16:30 |
| This is due to a block starting at '#ifdef HAVE_GCC_ASM_FOR_X87' in Python/pymath.c. A simple patch to pymacconfig.h fixes the issue, I'll commit it once I've fully tested the result. |
|
|
| msg96136 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-08 16:37 |
| Fixed in r76712 (trunk), r76713 (3.2), r76714 (3.1). The python3 branches contained about half of the fix, I merged the trunk version into python3 anyway to keep all 3 branches in sync. The issue does not affect the 2.6 branch. |
|
|
| msg96144 - (view) |
Author: Sridhar Ratnakumar (srid) |
Date: 2009-12-08 17:36 |
| > Do you get the same problem with py3k? Not in 3.1.1 at least. > Fixed in r76712 (trunk) That fixes the issue. Thanks! Let me know if you still need pyconfig.h. |
|
|
| msg96145 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2009-12-08 17:48 |
| "PPC (G3) machine rather than Intel (which shouldn't make a difference)" ... but did in this case. Perhaps it's too early in the morning yet but I don't see why that error doesn't show up for me when building trunk on an Intel machine under 10.6, i.e. this gives no error: /usr/bin/gcc-4.0 -c -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Python/pymath.o Python/pymath.c (This is with one patch to configure to fix the test for Darwin 10) |
|
|
| msg96147 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-12-08 18:03 |
| Ned: maybe your pyconfig.h isn't defining HAVE_GCC_ASM_FOR_X87 in the first place, for some reason? I *do* get that error for trunk / 10.6 (with the 10.5 SDK; I don't have the 10.4 SDK installed): Mark-Dickinsons-MacBook-Pro:trunk dickinsm$ /usr/bin/gcc-4.0 -c -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk/ -I. -IInclude -o Python/pymath.o Python/pymath.c /var/folders/nH/nHvjwWGlGMWNLkYlpsol8U+++TI/-Tmp-//ccq7GcaH.s:11:Invalid mnemonic 'fnstcw' /var/folders/nH/nHvjwWGlGMWNLkYlpsol8U+++TI/-Tmp-//ccq7GcaH.s:26:Invalid mnemonic 'fldcw' lipo: can't open input file: /var/folders/nH/nHvjwWGlGMWNLkYlpsol8U+++TI/-Tmp-//ccDS89Qs.out (No such file or directory) |
|
|
| msg96152 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-08 20:14 |
| Ned: which version of Python do you build? The issue was only in the trunk (2.7), the other branches were fine (although the code in 3.1 and 3.2 was suboptimal). BTW. The issue affected universal builds on all OSX versions. The trunk should be fine now as well, as of r76712. Sridhar: I don't need pyconfig.h. I was thinking of a configure issue where your machine behaved differently from what I'd expect. I turned out to be a universal-binary related issue. Mark: what's the output of svnversion? I committed a fix in r76712, that fixed the issue for me on 10.6. |
|
|
| msg96153 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2009-12-08 20:38 |
| Ronald, it's a more complicated configure issue. I'm in the middle of pinning down the details. More shortly ... |
|
|
| msg96155 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-12-08 22:03 |
| Ronald: sorry, I wasn't clear. I only get that failure before your fix; afterwards, it's fine. |
|
|
| msg96165 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2009-12-09 09:11 |
| I agree that adding the definitions of HAVE_GCC_ASM_FOR_X87 to pymacconfig.h solves the problem. [mini-rant on] This problem is another illustration of how easy it is to inadvertently introduce significant bugs in the OS X multi-architecture support, bugs which can currently go undetected without a lot of diligence on everyone's part. We were lucky on this one. Thanks, Sridhar. [mini-rant off] After digging around a lot, I see now why I hadn't noticed it before. Always specifying --enable-universalsdk=/Path/To/sdk and default arch options (as the current installer builds do) causes configure to include -arch ppc and -arch i386 in CFLAGS and these get added to the gcc call in autoconf tests AC_TRY_COMPILE and AC_TRY_RUN. That meant the autoconf test for HAVE_GCC_ASM_FOR_X87 was *always* NO because the PPC arch compile always failed on the Intel assembler code test. And the compile failure is not obvious unless you examine configure.log in detail and notice that the result of the test was NO because of the multiple architectures. If configure is called with just --enable-universalsdk (no path) but on 10.5 or 10.6, the multiple archs still get added to CFLAGS, the test returns NO again, and the problem is still masked: CFLAGS='-arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk - g -O2' But if you make that same configure call on 10.4, the multiple archs are not added: CFLAGS='-g -O2' On a 10.4 PPC system (my case) the test *still* returns NO so the problem is *still* masked. It's only if you run the build with that configure on a 10.4 i386 system that the test will pass, causing HAVE_GCC_ASM_FOR_X87 to finally be defined. Then, during the make of the multi-architecture build, the root problem will finally be exposed, as it was here. That also means that, for multi-arch builds on OS X, the X87 control word code would never be used in the python.org installers or any other intel/ppc multi-architecture build. I hope that is a reasonably close to correct analysis but I could have a few details wrong. It's clearly easy to get wrong. Looking elsewhere in configure.in, there are other architecture-specific run-time tests that could give incorrect results in a multi-arch environment. The math tests immediately following appear to take a conservative approach so I'm guessing that, if there were conflicting results during the multiple arch compiles and runs, the code would work correctly though possibly sub-optimally. I didn't see any other obvious incorrect results although it is difficult to tell when so many configure tests get compile or build errors by design. There seem to be at least two issues brought out here: 1. How to make core developers and patch contributors more aware of the special requirements of multi-architecture builds? 2. What, if any, additions or changes to tests and testing process should be made to help catch problems like this (and others that might be lurking)? At a minimum, I would like to see a message added to configure.in that is output on OS X to warn that some of the autoconf derived values for architecture-specific SIZEOF and processor features are overridden in Include/pyconfig.h. Currently there doesn't seem to be any reference to this in configure.in and it so easy to be misled by the autoconf results. Adding comments to the source of the overridden tests would be great but a bit of work. Conditionally skipping the tests would be even better, if it could be made to work. A more systematic test process would also help. |
|
|
| msg96168 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-12-09 13:41 |
| A reply to Ned from the person who introduced this bug in the first place (sorry, folks): [Ned] > 1. How to make core developers and patch contributors more aware of the > special requirements of multi-architecture builds? I guess I was already aware of this, at least after Ned (IIRC), some months ago, detected essentially the same problem in py3k after we'd put the short float repr in. So this was really just sloppiness on my part. I did testing of the short float repr for single-architecture builds on OS X 10.4/PPC (32-bit), OS X 10.5/Intel (32-bit and 64-bit) and OS X 10.6/Intel (64-bit only), but still missed this. Something that I would have found helpful at one stage would be prominent instructions (somewhere, maybe in the python-dev FAQ?) for creating and testing multi-architecture builds on various OS X versions. Perhaps all those instructions need to do is to point to the Mac/BuildScript/README.txt? Even now, I'm still not 100% clear on exactly which selection of options should be supplied to the configure scripts, and why. > At a minimum, I would like to see a message added to configure.in that > is output on OS X to warn that some of the autoconf derived values for > architecture-specific SIZEOF and processor features are overridden in > Include/pyconfig.h. Do you mean Include/pymacconfig.h? If so, I agree such a warning would be good to have. As an aside, I *think* it's actually safe just to undefine HAVE_GCC_ASM_FOR_X87 unconditionally on OS X: the inline assembler that it enables is only ever used for machines which suffer from double rounding, which is a symptom of using the x87 FPU for basic arithmetic. gcc on OS X (either Apple gcc or GNU gcc) always seems to use SSE2 instructions for arithmetic, even for 32-bit builds, so the fnstcw and fldcw instructions aren't ever needed or used on OS X. |
|
|
| msg96173 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2009-12-09 16:58 |
| Mark, I wasn't ranting at you or Ronald or anyone in particular. It's no secret that Python's build system is complex and fragile in some areas. On the other hand, the build system is ambitious, covering a wide variety of platforms and variants, and is largely successful. In particular, supporting multi-architecture builds across multiple OS levels for OS X is, IMO, a Good Thing to be doing. But there is a cost for all that. It would be good to be able to scrape away some of the overall cruft that's accumulated over the many years. Hmm, a GSOC project to implement SCons builds for the current platforms and to facilitate cross-platform builds? Anyway, I did indeed mean Include/pymacconfig.h. |
|
|
| msg96182 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-09 21:38 |
| Ned: the best way to ensure universal builds don't get broken is through the buildbots. AFAIK there are no buildbots that create universal binaries at the moment. Sadly enough I don't have the resources to provide one. Mark: Mac/README explains how to build universal binaries and Mac/BuildScript/build-installer.py is used to build the default binary installer. To build a universal binary framework: ../configure \ --enable-framework \ # Creates a framework --enable-universalsdk # Create universal binary using 10.4 SDK make make install The build can be tweaked using a number of options: * Use an alternate SDK: --enable-universalsdk=PATH (I tend to use --enable-universalsdk=/ for my private builds, the 10.4u SDK is used in the default installers to ensure that users on 10.4 PPC can build universal extensions) * Select architectures to include in the build: --with-universal-archs=VALUE * Pick an alternate framework name: --with-framework-name=NAME * Pick a different deployment target: MACOSX_DEPLOYMENT_TARGET=X.Y (Default is 10.3 for 32-bit builds and 10.5 for builds that include a 64-bit architecture). BTW. I agree that the current configure script sucks, it is very hard to tweak it beyond adding simple feature tests. IIRC it has taken me several Python releases to reliably run the feature detection tests because some compiler flags didn't get exported correctly to the bits of the configure script that runs the feature tests. One way to improve on the script is to create a number of clearly labeled sections, such as: compiler/platform setup, architecture tests (SIZEOF_* detection, X86_FPU tests, ...), platform feature tests. The architecture tests could then be prepended with a comment block that explains how to deal with them for universal builds on OSX. |
|
|
| msg96183 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-09 21:39 |
| BTW. Am I correct when I state that this issue has been fixed and can be closed? |
|
|
| msg96244 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-12-11 11:29 |
| Yes, this can be closed as far as I'm concerned. |
|
|