Issue 1006238: cross compile patch (original) (raw)

Created on 2004-08-09 22:05 by goertzen, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (61)

msg46623 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2004-08-09 22:05

Here's a cross compile patch I prepared a while ago but never got around to submitting. I've been using it happily for months to cross compile python for embedded systems.

Below is a descriptive excerpt from the patch. Also note that the patch modifies configure.in, but not configure. You will need to regenerate configure with something like

autoconf configure.in >configure

This patch is inpsired from work by Klaus Reimer at http://www.ailis.de/~k/docs/crosscompiling/python.php

Cross Compiling
Python can be cross compiled by supplying different --host and --build
parameters to configure. (Python is compiled on the "build" system
and executed on the "host" system, in case you forgot :). Python is
tricky to cross compile because it needs to execute parts of itself
during construction. To work around this, make's VPATH feature is
used to compile a native python in the subdirectory "buildpython".
When parts of python need to be executed during construction, the
"buildpython" versions are used.
A consequence of using the VPATH feature is that you may not do a
cross compile build in the source directory. In other words, do this:
mkdir mydir
cd mydir
../Python/configure --host=powerpc-405-linux-gnu --build=i686-pc-linux-gnu
make
Cross compiling works well under linux, mileage may vary for other
platforms.
A few reminders on using configure to cross compile:
- Cross compile tools must be in the PATH.
- Cross compile tools must be prefixed with the host type
(ie powerpc-405-linux-gnu-cc, powerpc-405-linux-gnu-ranlib, ...)
- CC, CXX, AR, and RANLIB must be undefined when running configure and
make. Configure will detect them.
If you need a cross compiler, check out Dan Kegel's crosstool:
http://www.kegel.com/crosstool

msg46624 - (view)

Author: Jeff Epler (jepler)

Date: 2004-10-21 01:08

Logged In: YES user_id=2772

This patch applies cleanly to today's CVS head.

If building in the source directory while cross compiling fails, please make configure complain about it.

This patch makes the build process invoke make recursively, which is a big minus. I'd rather see pgen built with a HOSTCC and just assume python is available on $PATH for the setup.py step. There's no reason to build all the shared modules in buildpython, either, which would speed things up a fair bit.

On to how it worked for me: Not cross-compiling, this produces no unexpected test failures in "make test". (redhat9)

Cross compiling for i386-pc-bsdi2.1 everything goes fine until it tries to run buildpython and make shared modules, but shared modules don't work in the first place on bsdi2. I did not test the resulting Python binary.

Cross compiling for i386-redhat-linux (libc6) some extensions fail to build, but this could be because my header files for the cross-development environment are not complete. Running "make test" tries to invoke the "buildpython/python" and doesn't work. Running it manually I get some skips due to modules that did not build, but everything that did build seems to pass. (OK, so I wrote this before the tests completed, but they're off to a good start)

I currently cross-compile python 2.3 for win32 (mingw), and until recently cross-compiled it for bsdi2 and redhat6. However, I did not use configure or the included makefiles to do this (in order to integrate in a non-recursive build procedure that builds several packages), so this patch is unlikely to benefit me directly.

I don't think this patch is ready to be applied.

msg46625 - (view)

Author: Mike Frysinger (vapier)

Date: 2004-10-26 16:00

Logged In: YES user_id=114429

we've been using this with uClibc for a while now and it works great ...

ive personally built (on amd64) & deployed (on x86/ppc/arm/mips/sh4) python ... would great if this finally made it into the official sources :)

msg46626 - (view)

Author: Ned Ludd (solarx)

Date: 2005-04-09 20:43

Logged In: YES user_id=148412

Any progress with this on? python and perl are two of the last major things to overcome in the x-compile world.

msg46627 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2005-04-13 17:36

Logged In: YES user_id=843814

It still works for me, so I've had limited interest in working on it further.

I think a "real" cross compile patch will involve significant refactoring of distutils and the main setup.py script. Should this start with a PEP?

msg46628 - (view)

Author: Robsa (robsa)

Date: 2005-10-13 11:54

Logged In: YES user_id=1277505

Just thought I'd let you know that this patch works against Python 2.4.2. I had Python running on my custom AT91RM9200 board (ARM920T core) in about 20 minutes.

snaps for Daniel!!

msg46629 - (view)

Author: David Lambert (jdalambert)

Date: 2005-11-08 02:12

Logged In: YES user_id=845425

Hmm. not so much luck here. I get the following error after patching then autoconf. I am running on Fedora Core 4.

Any suggestions?

[dlambert@lambsys Python-2.4.2]$ autoconf configure.in

configure [dlambert@lambsys Python-2.4.2]$ ./configure configure: error: cannot run /bin/sh ./config.sub [dlambert@lambsys Python-2.4.2]$

msg46630 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2005-11-08 14:05

Logged In: YES user_id=843814

You can't configure in the source directory with the cross compile patch. This is explained in the directions.

msg46631 - (view)

Author: David Lambert (jdalambert)

Date: 2005-11-08 14:26

Logged In: YES user_id=845425

Thanks for the quick reply, and sorry for the confusion.

I DID try the cross compile in a sub directory. That failed with the same error. I then tried a non-cross build in the main directory, that also failed (which was my previous post). Here is my complete transcript after untarring Python:

[dlambert@lambsys Python-2.4.2]$ patch -p0 < ../python-patch can't find file to patch at input line 3 Perhaps you used the wrong -p or --strip option? The text leading up to this was:

|*** python-cvs-pristine/dist/src/README Fri Mar 5 08:33:21 2004 |--- python/dist/src/README Mon Apr 5 14:30:23 2004

File to patch: README patching file README Hunk #1 succeeded at 1130 (offset 30 lines). can't find file to patch at input line 48 Perhaps you used the wrong -p or --strip option? The text leading up to this was:

|*** python-cvs-pristine/dist/src/configure.in Sun Mar 21 17:45:41 2004 |--- python/dist/src/configure.in Mon Apr 5 16:15:07 2004

File to patch: configure.in patching file configure.in Hunk #2 succeeded at 609 (offset 58 lines). Hunk #3 succeeded at 3072 (offset 112 lines). can't find file to patch at input line 113 Perhaps you used the wrong -p or --strip option? The text leading up to this was:

|*** python-cvs-pristine/dist/src/Makefile.pre.in Thu Mar 18 01:51:27 2004 |--- python/dist/src/Makefile.pre.in Mon Apr 5 15:56:00 2004

File to patch: Makefile.pre.in patching file Makefile.pre.in Hunk #2 succeeded at 163 (offset 3 lines). Hunk #4 succeeded at 309 (offset 5 lines). Hunk #6 succeeded at 470 (offset 5 lines). Hunk #7 succeeded at 624 (offset 1 line). Hunk #8 succeeded at 839 (offset 7 lines). Hunk #9 succeeded at 923 (offset 1 line). Hunk #10 succeeded at 969 (offset 7 lines). can't find file to patch at input line 309 Perhaps you used the wrong -p or --strip option? The text leading up to this was:

|*** python-cvs-pristine/dist/src/setup.py Sun Mar 21 12:59:46 2004 |--- python/dist/src/setup.py Mon Apr 5 15:20:55 2004

File to patch: setup.py patching file setup.py Hunk #1 succeeded at 198 (offset -2 lines). patching file python/dist/src/config.guess patching file python/dist/src/config.sub [dlambert@lambsys Python-2.4.2]$ [dlambert@lambsys Python-2.4.2]$ [dlambert@lambsys Python-2.4.2]$ autoconf configure.in > configure [dlambert@lambsys Python-2.4.2]$ mkdir cross-build [dlambert@lambsys Python-2.4.2]$ cd cross-build [dlambert@lambsys cross-build]$ ../configure --host=arm-linux --build=i686-pc-li nux-gnu configure: error: cannot run /bin/sh ../config.sub [dlambert@lambsys cross-build]$

msg46632 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2005-11-08 15:09

Logged In: YES user_id=843814

patch isn't lying about wrong -p options. Use -p3 instead of -p0.

Cheers, Dan.

msg46633 - (view)

Author: David Lambert (jdalambert)

Date: 2005-11-08 16:06

Logged In: YES user_id=845425

Oops! All works fine now. Thanks :-)

msg46634 - (view)

Author: xudong (xudong888)

Date: 2006-01-05 16:55

Logged In: YES user_id=1420135

I am a Chinese and my English is very poor.I'm sorry if what I said is wrong.My question is What should I do after './configure'.Before this I have done patch and autoconf.

Thanks :-)

msg46635 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-01-05 17:24

Logged In: YES user_id=843814

After configure you run "make".

But did you use configure as the instructions say? You cannot just use "./configure".

Good luck, Dan.

msg46636 - (view)

Author: xudong (xudong888)

Date: 2006-01-06 10:57

Logged In: YES user_id=1420135

Thank you for your rapid answer.But I still can't solve my question.I can't find the whole instructions. Can you give me the whole instructions by EMAIL,my email is xudong888@163.com,thanks. My "host" system is mips embbed_linux,the release of the linux kernel is 2.4.xx and the CPU type is MIPS 4Kc.There are no python installed on this system.My "build" system is i86 and operate system is RedHat9.0,and has installed python2.4.1 and cross-compiling Tools mipsel-linux-gcc.I can cross compile C program for the "host" system.I have writed some program with python language.I can get the binary from python script by use freeze.py on the "build" system. Now I want to run the binary on the "host" system.Can you tell me how should I do. in addition,I don't know what parameter I should input to use this patch. finally,I'm sorry for my poor English.I like python very much,but I can't get help in chinese.

msg46637 - (view)

Author: xudong (xudong888)

Date: 2006-01-06 11:08

Logged In: YES user_id=1420135

If you have accounts of MSN,please add me.My accounts is xudong_888@hotmail.com. I am very impatient.I hope I can get some help from you.Thanks.

msg46638 - (view)

Author: xudong (xudong888)

Date: 2006-01-10 14:45

Logged In: YES user_id=1420135

I have resolved my question,now I can run python on the mips machine,and can run some binary build by python.Can I also cross compile pygame for python?But the pygame source does't has 'configure' and 'Makefiel' files,setup.py instead in source package.How should I do? Thanks :-)

msg46639 - (view)

Author: nicolas pinault (nico38)

Date: 2006-02-24 12:38

Logged In: YES user_id=1081828

Hi,

I try to cross compile Python-2.4.2 for an etrax processor. I have successfully applied the patch to Python sources. I then cd to a "MyPython" directory. I call ../Python-2.4.2/configure --build cris --host i386-linux It works ok (did not see an error) I all make I get the following error after a while : make : *** No rule to make target '@BUILDPGEN@', needed by '../Python-2.4.2/Include/graminit.h'. Stop.

Any Idea ? Nicolas

msg46640 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-02-24 14:51

Logged In: YES user_id=843814

Did you run autoconf after patching as instructed?

Also, your --build and --host options seem backwards. host is the arch you want python to run on, build is the arch you build python on.

Cheers, Dan.

msg46641 - (view)

Author: nicolas pinault (nico38)

Date: 2006-02-24 15:12

Logged In: YES user_id=1081828

You are right, I have misunderstood --build and --host signification.

And I also forgotten to run autoconf partialy because I don't have autoconf installed on my system. So I didn't pay attention it didn't run.

Thanks for your help. I'll let you know if this works in a few days.

Nicolas

msg46642 - (view)

Author: Nicolas (nico0438)

Date: 2006-02-24 19:55

Logged In: YES user_id=1330285

Hi again,

I finaly dad enought time to work on my project. So, I compiled autoconf tools for my system. Then, I rerun all the steps successfully.

Thanks again for your help.

Nicolas

msg46643 - (view)

Author: Nicolas (nico0438)

Date: 2006-02-24 20:54

Logged In: YES user_id=1330285

Me again, sorry...

What Should I get after compilation has finished ? Where are the files for my target ?

Nicolas

msg46644 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-02-24 21:14

Logged In: YES user_id=843814

I then cd to a "MyPython" directory. I call ../Python-2.4.2/configure ...

Then your files should be in your "MyPython" directory.

msg46645 - (view)

Author: Nicolas (nico0438)

Date: 2006-02-24 21:19

Logged In: YES user_id=1330285

Yes I guess.

But it seems that all I get is for the build system. If I try to execute on the build system ./python it runs. Also, there is no hostbuild subdirectory. Is it ok ?

Nicolas

msg46646 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-02-24 21:30

Logged In: YES user_id=843814

The build system python should be in "MyPython/buildpython", and the host python should just be in "MyPython"

Are you sure you applied the patch right? What you're getting is what would happen if you didn't apply the patch at all.

msg46647 - (view)

Author: Nicolas (nico0438)

Date: 2006-02-24 21:37

Logged In: YES user_id=1330285

If I try to aplly the patch again, I get messages saying a patch has allready been applied. To apply the patch I executed the following :

cd Python-2.4.2 patch <../python-patch.txt

msg46648 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-02-24 22:32

Logged In: YES user_id=843814

You didn't apply the patch properly. See the other messages about the use of the -p option. Start with a fresh source tree and use:

cd Python-2.4.2 patch -p3 <../python-patch.txt

msg46649 - (view)

Author: hugo (hugo_koopmans)

Date: 2006-03-04 20:02

Logged In: YES user_id=1229536

Hi there, I am trying to cross compile python on the fox board (etrax). now everything seemed to go well but when running the python binary is says: ./python: error while loading shared libraries: libstdc++.so.5: cannot load shared object file: No such file or directory

I only ftp-ed the python bin not the (much bigger) libpython2.4a .... seems this is a library archive of some kind ? do i need it? how do i use it then?

thanx a million if you can help me out.

hugo

msg46650 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-03-06 15:12

Logged In: YES user_id=843814

Hi hugo.

The problems you are having are less python related and more system-building/cross-development issues, and this is the wrong forum for hashing out this stuff out.

I recommend building a linux-from-scratch system (http://www.linuxfromscratch.org/) on an ordinary PC to learn how the insides of linux machine works and how to fix things when they go wrong.

Good luck, Dan.

msg46651 - (view)

Author: Nicolas (nico0438)

Date: 2006-03-07 21:30

Logged In: YES user_id=1330285

Hi again,

It seems I'm successfull in cross-compiling python. But I'm not sure of what I have to save on the host platform. There is python executable of course. But there is no lib directory with .so files and .py files in the directory where python is built. Can you tell me what files/directories you save on your host platform ? Thanks again for your support. Nicolas

msg46652 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-03-07 21:45

Logged In: YES user_id=843814

I use this command:

make install prefix=/my/temp/dir

msg46653 - (view)

Author: matt (mattcomms)

Date: 2006-07-04 09:08

Logged In: YES user_id=1543599

Hello, Wondering if your still offering some support.

As people have reported success with 2.4.2 I decided to use
that version. I have been carefull to follow the steps as
described but still having some difficulty.

The steps I take are:

patch -p3 < ../python-patch
autoconf configure.in >configure
mkdir cross-build
cd cross-build
../configure --host=cris-axis-linux-gnu --build=i486-slackware-linux

I also edit the LINKCC line in the makefile and include
-static (as I want to run on devboard with limited memory
and not to worry about shared libraries)

make throws lots of errors like:

*** WARNING: renaming "struct" since importing it failed:
build/lib.linux-i686-2.4/struct.so: cannot open shared
object file: No such file or directory
building 'regex' extension
cris-axis-linux-gnu-gcc -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC -fno-strict-aliasing -I.
-I/root/Python-2.4.2/./Include -I/usr/local/include
-I/root/Python-2.4.2/cross-build/Include
-I/root/Python-2.4.2/cross-build/buildpython
-c /root/Python-2.4.2/Modules/regexmodule.c -o
build/temp.linux-i686-2.4/regexmodule.o
cris-axis-linux-gnu-gcc -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC -fno-strict-aliasing -I.
-I/root/Python-2.4.2/./Include -I/usr/local/include
-I/root/Python-2.4.2/cross-build/Include
-I/root/Python-2.4.2/cross-build/buildpython
-c /root/Python-2.4.2/Modules/regexpr.c -o
build/temp.linux-i686-2.4/regexpr.o
cris-axis-linux-gnu-gcc -shared
build/temp.linux-i686-2.4/regexmodule.o
build/temp.linux-i686-2.4/regexpr.o -L/usr/local/lib -o
build/lib.linux-i686-2.4/regex.so

and

cris-axis-linux-gnu-gcc -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC -fno-strict-aliasing -I.
-I/root/Python-2.4.2/./Include -I/usr/local/include
-I/root/Python-2.4.2/cross-build/Include
-I/root/Python-2.4.2/cross-build/buildpython
-c /root/Python-2.4.2/Modules/_ssl.c -o
build/temp.linux-i686-2.4/_ssl.o
/root/Python-2.4.2/Modules/_ssl.c:30:25: openssl/rsa.h: No
such file or directory

make install prefix=/root/Python-2.4.2/cross-build

throws the same errors but finishes.

I then mount cross-build via nfs and run python

Python 2.4.2 (#2, Jun 28 2006, 18:35:28)
[GCC 3.2.1 Axis release R61/1.61] on linux2
Type "help", "copyright", "credits" or "license" for more
information.

import time
import time
Traceback (most recent call last):
File "", line 1, in ?
ImportError: /mnt/flash/Python/lib/python2.4/lib-dynload/time.so:
undefined symbol: PyExc_IOError

import socket
import socket
Traceback (most recent call last):
File "", line 1, in ?
File "/mnt/flash/Python/lib/python2.4/socket.py", line
45, in ?
import _socket
ImportError: /mnt/flash/Python/lib/python2.4/lib-dynload/_socket.so:
undefined symbol: PyObject_GenericGetAttr

Any hint as to what might be causing these errors would be
most appreciated.

Cheers

Matthew

msg46654 - (view)

Author: Daniel Goertzen (goertzen)

Date: 2006-07-04 16:48

Logged In: YES user_id=843814

This stuff is becoming hazy for me, but I'll try to offer some ideas:

Does it help if you put your build directory outside of the source directory?
Why is the system looking for struct.so when you told it to be static? I suspect static support is broken for a lot of modules. Does it go better when you take the static switch out? Assuming you have a few other apps in your system besides python (a shell, busybox maybe), I question wether static would actually save any memory. It perhaps is a little more complex to get all the libraries in their correct location, but I'll bet coaxing a python system to be static is way harder.
It is normal for some things to bomb out while compiling because the python build scripts always check the build system for library availability. In many cases you don't need those broken libraries. It is also often possible to hack up the install scripts to work properly for a cross compiled system. That said, simple things like struct should be working properly without hackery.

Cheers, Dan.

msg82681 - (view)

Author: Enji Cooper (ngie) *

Date: 2009-02-24 22:05

I can definitely chime in on this issue.

A (proper) testcase would be to do something like the following:

Example item -- this isn't what you'll be using...

CROSS_COMPILE_PREFIX=binos_c3.4.3-p1.mips64-octeon-linux-

AS="${CROSS_COMPILE}as"
CC="${CROSS_COMPILE}gcc"
CC="${CROSS_COMPILE}c++"
LD="${CROSS_COMPILE}ld"
NM="${CROSS_COMPILE}nm"
./configure --prefix=/usr/local
BUILD_PYTHON=/path/to/native/python
DESTDIR=/where/i/want/to/install/python
--build="${CROSS_COMPILE}gcc" -dumpmachine
--host=arch

NOTES:

arch(1) is a GNU specific utility, so it won't work outside of Linux!
I'll leave it as an exercise to the reader to specify the correct CFLAGS, CPPFLAGS, LDFLAGS, and LDLIBS.

Some testcases during the compile that fail if cross-compiling are (with 2.6.1):

chflags
lchflags
printf with %zd format support.

Thanks!

msg82682 - (view)

Author: Enji Cooper (ngie) *

Date: 2009-02-24 22:35

Am I correct in this understanding, or is Pgen unneeded after the grammar file is generated?

msg82741 - (view)

Author: Mike Frysinger (vapier)

Date: 2009-02-26 04:33

Garrett: your configure method is overly complicated. all you need to do is set --build=binos_c3.4.3-p1.mips64-octeon-linux. autoconf will figure out all the other toolchain settings.

msg82779 - (view)

Author: Roumen Petrov (rpetrov) *

Date: 2009-02-26 22:03

Mike, the python configure script fail to detect some of toolchain tools.

msg90706 - (view)

Author: Enji Cooper (ngie) *

Date: 2009-07-19 05:47

Coming back to this issue, I really want to resolve it on TRUNK and for it to make its way into 2.6.3 and 2.x trunk, as well as 3.0.2 and 3.x trunk. I am more than happy to sign a contributor agreement if this will push things along quicker.

Thanks, -Garrett

msg94775 - (view)

Author: Enji Cooper (ngie) *

Date: 2009-10-31 22:53

I'm trying to resolve this issue on:

2.6-releasemaint trunk 3.1-releasemaint py3k

first by resolving issues configure.in, but there are a TON of AC_TRY_RUN's, which means that this code cannot be cross-compiled as-is (25 on 2.x -- 27 on 3.x).

Is requiring the end-user to define the autoconf variables appropriately to their platform when running configure (when provided an error message telling them so), a longterm sustainable requirement? I know it isn't as user friendly, but this is a definite problem that either needs to be fixed in the autoconf tests (if possible) or the C code itself.

I wouldn't mind updating the INSTALL or README files to reflect this change either, if needed.

msg94776 - (view)

Author: Gregory P. Smith (gregory.p.smith) * (Python committer)

Date: 2009-10-31 23:14

Documenting the parameters needed to avoid all AC_TRY_RUNs is a good first
step for any that are not obvious how to convert from AC_TRY_RUN into something else.

msg94785 - (view)

Author: Mike Frysinger (vapier)

Date: 2009-11-01 12:50

AC_TRY_RUN is already documented: http://www.gnu.org/software/autoconf/manual/html_node/Obsolete-Macros.html#index-AC_005fTRY_005fRUN-1992

there are a bunch of distros out there (like OE and Gentoo) that have been maintaining cross-compile patches for python. i'm pretty sure the stuff in Gentoo works for 2.6.x, but i havent tried 3.1.x yet.

ive given up on pushing to upstream as this bug (among others)) shows that such attempts go nowhere

msg94792 - (view)

Author: Gregory P. Smith (gregory.p.smith) * (Python committer)

Date: 2009-11-01 17:35

Removing a toxic person from the cc list. Mike, please go harm some other all volunteer project.

msg94802 - (view)

Author: Enji Cooper (ngie) *

Date: 2009-11-01 19:55

On Sun, Nov 1, 2009 at 5:50 AM, Mike Frysinger <report@bugs.python.org> wrote:

Mike Frysinger <vapier@users.sourceforge.net> added the comment:

AC_TRY_RUN is already documented: http://www.gnu.org/software/autoconf/manual/html_node/Obsolete-Macros.html#index-AC_005fTRY_005fRUN-1992

there are a bunch of distros out there (like OE and Gentoo) that have been maintaining cross-compile patches for python. i'm pretty sure the stuff in Gentoo works for 2.6.x, but i havent tried 3.1.x yet.

ive given up on pushing to upstream as this bug (among others)) shows that such attempts go nowhere

Actually what Mike showed was helpful for me. I didn't realize that the 3rd argument to AC_TRY_RUN was for Canadian cross, aka cross-compiling.

My personal opinion on why past attempts have failed (and it's just my opinion) is probably because:

The change set wasn't incremental, thus the diff was large, and the checkin was rejected.
The patch was based on previous versions of python, which doesn't help the current trunk, release-maint* branches, etc.

I'm more than happy to steal existing code (if possible :)..), but it should be well designed so longterm maintenance can be eased, and the cross-compile issue can be resolved in a correct manner.

It took me 2 months to rewrite the Makefile infrastructure for LTP -- this should be a lot simpler and less painful to resolve (in terms of autotools input files, Makefile, etc). setup.py and distutils is something that I need to defer to someone more seasoned in the python internals (at least for mentoring) s.t. it can be resolved on all branches.

First comes first, I'll propose some changes for cross-compilation dealing with some of the AC_TRY_RUN tests -- there are some tests that can be turned into preprocessor defines and/or AC_TRY_COMPILES [the sizeof(pthread_t), etc], then I'll look at the other tests and propose appropriate action for them.

If needed individuals in the python org. aren't aware of this work, it probably should be brought to their attention sometime in the next couple of weeks, because I need to make sure core team members are aware of these changes so that they can get reviewed and checked into the project in a timely manner (my group needs to upgrade from 2.4.2 to python 2.6.x in the next couple months; this is a stopgap item for us because we use a cross-compilation environment).

All the best, -Garrett

msg94805 - (view)

Author: Mike Frysinger (vapier)

Date: 2009-11-01 20:57

Gregory: there's no need to be a dick. i'm pointing out the obvious -- bugs have been open literally for years with zero assistance/feedback from anyone who can actually get things merged. people have posted patches, but no one has said "xxx needs to be done in order to get merged". you havent posted anything here either (assuming you're someone who can actually get things merged and not just comment in a tracker). if you can at least do something with trackers, you should start by marking 1597850 as a dupe of this one. or you can simply prove my point by continuing to contribute nothing.

the basic required changes are simple -- fix the few autoconf tests. getting automatic cross-detection (building a host python/pygen automatically) isnt nearly as important as long as people have a way to tell the build system to use a different python/pgen for build purposes. last i looked, these simple changes were pretty trivial to move across major versions of python.

i fixed the chflags specific check a long time ago (as i imagine others have as well): http://sources.gentoo.org/dev-lang/python/files/python-2.6-chflags-cross.patch

same goes for the printf %zd test: http://sources.gentoo.org/dev-lang/python/files/python-2.5-cross-printf.patch

however, unless these trivial baby steps can be made, worrying about the next step (properly cross-compiling modules and such) is a complete waste of time as these require diving into the python-specific build system which does see a lot of churn over versions.

msg94813 - (view)

Author: Enji Cooper (ngie) *

Date: 2009-11-01 23:20

Ok. Taking a look at trunk...

The following could be converted to AC_TRY_COMPILE statements for the 3rd AC_TRY_RUN tuple:

$ac_enable_profiling : 697
$ac_cv_no_strict_aliasing_ok : 921
$ac_cv_opt_olimit_ok : 1070
$ac_cv_olimit_ok : 1092
$ac_cv_pthread_is_default : 1126
$ac_cv_kpthread : 1163
$ac_cv_pthread : 1225
$ac_osx_32bit : 1569
$ac_cv_pthread_system_supported : 2229
$ac_cv_have_size_t_format : 3959

The following can just be converted to AC_TRY_COMPILE:

$ipv6 : 2278
$ac_cv_have_chflags : 2663
$ac_cv_have_lchflags : 2693

The following will need to be sorted out, as to what needs to be done here, as they are legitimate runtime only tests:

$ac_cv_little_endian_double : 3249
$ac_cv_big_endian_double : 3271
$ac_cv_mixed_endian_double : 3299
$ac_cv_x87_double_rounding : 3354
$ac_cv_broken_sem_getvalue : 3395
$ac_cv_tanh_preserves_zero_sign : 3430
$ac_cv_wchar_t_signed : 3510
$ac_cv_rshift_extends_sign : 3597
$ac_cv_broken_nice : 3714
$ac_cv_broken_poll : 3735
$ac_cv_working_tzset : 3772

Taking a look at py3k, most of the offsets are the same -- some have changed, but the only the test which doesn't exist in trunk is the following:

$ac_cv_broken_mbstowcs : 3872

Again, this is a valid runtime test, so it needs to be sorted out what should be done here with cross-compilation.

msg94816 - (view)

Author: Gregory P. Smith (gregory.p.smith) * (Python committer)

Date: 2009-11-02 02:03

these two have been merged and applied to trunk.

""" i fixed the chflags specific check a long time ago (as i imagine others have as well): http://sources.gentoo.org/dev-lang/python/files/python-2.6-chflags- cross.patch

same goes for the printf %zd test: http://sources.gentoo.org/dev-lang/python/files/python-2.5-cross- printf.patch """

msg94817 - (view)

Author: Mike Frysinger (vapier)

Date: 2009-11-02 03:20

the chflags is specifically documented as needing a runtime test:

On Tru64, chflags seems to be present, but calling it will

exit Python

which is why i left the default of AC_TRY_RUN but cross-compile falls back to a simple link test. btw, a compile test is not valid when trying to see if a function exists. you'll get a successful compile (and warning about implicit function), but no error because you didnt finally link the object with the undefined reference.

somewhat similar are the compiler checks (profile/pthread/alias/etc...). some compilers do different things when linking and compiling (like gcc and -pthread), so sticking to a LINK in the fallback of the RUN is better, although not always perfect. some flags are accepted/ignored by compilers and issue only warnings about the unknown flags, not errors. but this issue probably isnt worth worrying about considering the code in there today suffers from this edge case (and if no one is complaining, then forget about it).

in terms of making sure all AC_TRY_RUN's have cross-compile fallbacks, i only worried about the ones that actually get exercised. the two i posted fixes for are the only ones ive seen people (and myself) actively hit.

the ipv6 should def have a LINK fallback, and it should try using in6addr_any as that is often an exported symbol (which is missing when ipv6 doesnt exist).

the double endian checks could easily be made into a compile test with a creative grep. pick a double value that expands into a funky ascii string and then grep the object file for a match. otherwise, a char swapped ascii string indicates it's big endian.

the wchar/rshift signed tests can be made into a compile-only test by creative use of arrays (like autoconf does now with compile sizeof() tests). main() { char foo[(((wchar_t) -1) < ((wchar_t) 0)) ? 1 : -1]; } compilers will portably abort when array size is negative, and this expression should be a constant.

i dont think any of the "broken" ones need to be sorted out as they already have cross-compile fall backs and there isnt much to be done in figuring out if the run time env is broken.

thanks Greg for the commits !

msg97387 - (view)

Author: Jacob Godserv (javaJake)

Date: 2010-01-07 23:28

This bug affects me as well. Adding myself to CC.

msg97388 - (view)

Author: Jacob Godserv (javaJake)

Date: 2010-01-07 23:32

The stage of this bug could be changed to "patch review", since a patch is available.

msg110395 - (view)

Author: Mark Lawrence (BreamoreBoy) *

Date: 2010-07-15 22:42

As a cross compile patch and being a feature request this must target 3.2.

msg132104 - (view)

Author: Éric Araujo (eric.araujo) * (Python committer)

Date: 2011-03-25 15:49