| msg269116 - (view) |
Author: Erik Bray (erik.bray) *  |
Date: 2016-06-23 15:20 |
| When building Python on Cygwin, both a libpython-X.Y.dll and a libpython-X.Y.dll.a are created (see https://cygwin.com/cygwin-ug-net/dll.html). The latter is an "import library" consisting of stubs for functions in the DLL so that it can be linked to statically when building, for example, extension modules. The odd bit is that in the altbininstall target if the (DLLLIBRARY)variable(whichreferencesthe.dllitself)isdefinedthenonlyitisinstalled,while(DLLLIBRARY) variable (which references the .dll itself) is defined then only it is installed, while (DLLLIBRARY)variable(whichreferencesthe.dllitself)isdefinedthenonlyitisinstalled,while(LDLIBRARY) (which in this cases references the .dll.a import library) is *not* installed, except in (prefix)/lib/pythonX.Y/config,whichisnotnormallyonthelinkersearchpath,orevenincludedbypython−config−−ldflags.Thereforestaticlinkingtolibpythonfails,unlessthesearchpathisexplicitlymodified,orasymlinkiscreatedfrom(prefix)/lib/pythonX.Y/config, which is not normally on the linker search path, or even included by python-config --ldflags. Therefore static linking to libpython fails, unless the search path is explicitly modified, or a symlink is created from (prefix)/lib/pythonX.Y/config,whichisnotnormallyonthelinkersearchpath,orevenincludedbypython−config−−ldflags.Thereforestaticlinkingtolibpythonfails,unlessthesearchpathisexplicitlymodified,orasymlinkiscreatedfrom(prefix)/lib/pythonX.Y/config/libpython.dll.a to (prefix)/lib.InfacttheMakefilealreadydoesthelatter,againnotif(prefix)/lib. In fact the Makefile already does the latter, again not if (prefix)/lib.InfacttheMakefilealreadydoesthelatter,againnotif(DLLLIBRARY) is defined. In fact Cygwin's own package for Python manually creates the latter symlink in its install script. But I would consider it a bug in Python's Makefile that it doesn't handle this. I first asked on Python-dev about this back in April but got no reply (understandably) probably due to the lack of core devs who can comment on Cygwin :) |
|
|
| msg276398 - (view) |
Author: Masayuki Yamamoto (masamoto) * |
Date: 2016-09-14 08:25 |
| Hello, Eric. I tried to build the '_speedups' C extension module of simplejson (third party library) using the python built on cygwin. As you pointed out, the build of '_speedups' C extension module at the stage of the link failed. build log: Python3.5.2+ has be used. It has applyed patches to known issues. $ ./configure --prefix=/opt/py35 --with-threads --with-dbmliborder=gdbm --with-system-ffi --with-system-expat $ make && make altinstall $ LANG=C /opt/py35/bin/pip3.5 install simplejson (snip) running build_ext building 'simplejson._speedups' extension creating build/temp.cygwin-2.6.0-i686-3.5 creating build/temp.cygwin-2.6.0-i686-3.5/simplejson gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/py35/include/python3.5m -c simplejson/_speedups.c -o build/temp.cygwin-2.6.0-i686-3.5/simplejson/_speedups.o gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-2.6.0-i686-3.5/simplejson/_speedups.o -L/opt/py35/lib/python3.5/config -L/opt/py35/lib -lpython3.5m -o build/lib.cygwin-2.6.0-i686-3.5/simplejson/_speedups.cpython-35m.dll /usr/lib/gcc/i686-pc-cygwin/5.4.0/../../../../i686-pc-cygwin/bin/ld: cannot find -lpython3.5m collect2: error: ld returned 1 exit status *************************************************************************** WARNING: The C extension could not be compiled, speedups are not enabled. Failure information, if any, is above. I'm retrying the build without the C extension now. *************************************************************************** (snip) In the log, -L option specified to "/opt/py35/lib/python3.5/config". But, the current python has "/opt/py35/lib/python3.5/config-3.5m" instead of "/opt/py35/lib/python3.5/config". And libpython3.5m.dll.a library exists in the config-3.5m. I think that to succeed building a C extension module if -L options is changed to "/opt/py35/lib/python3.5/config-3.5m". I'm going to try to find the place that add -L option at build time. |
|
|
| msg276402 - (view) |
Author: Masayuki Yamamoto (masamoto) * |
Date: 2016-09-14 08:56 |
| I'm so sorry that I misspelled your name, Erik. |
|
|
| msg276669 - (view) |
Author: Masayuki Yamamoto (masamoto) * |
Date: 2016-09-16 03:01 |
| I found -L option addition at Lib/distutils/command/build_ext.py:222. This place is a process that finalize options for the build_ext command. Comment in there: for extensions under Cygwin and AtheOS Python's library directory must be appended to library_dirs After that I searched Makefile rule for installing (LDLIBRARY)into"(LDLIBRARY) into "(LDLIBRARY)into"(prefix)/lib/python3.5/config-3.5m" to look for way of simplifying code. And I came across the (LIBPL)configurationvariablethatisusedbyinstallationoftheconfigurationforbuildinglibrary.HenceImadeapatchthataddstolibrarydirsthe(LIBPL) configuration variable that is used by installation of the configuration for building library. Hence I made a patch that adds to library_dirs the (LIBPL)configurationvariablethatisusedbyinstallationoftheconfigurationforbuildinglibrary.HenceImadeapatchthataddstolibrarydirsthe(LIBPL) configuration variable instead of the "$(prefix)/lib/python$(VERSION)/config". I succeeded to build '_speedups' C extension module of simplejson using new python that is applied this patch. Erik, Would you be able to verify this patch? Thanks Build log: $ patch -p1 <3.5-issue27374-distutils-build_ext.patch $ ./configure --prefix=/opt/py35 --with-threads --with-dbmliborder=gdbm --with-system-ffi --with-system-expat $ make touch && make -j2 && make altinstall $ /opt/py35/bin/python3.5 -V Python 3.5.2+ $ LANG=C /opt/py35/bin/pip3.5 install -v simplejson (snip) running build_ext building 'simplejson._speedups' extension creating build/temp.cygwin-2.6.0-i686-3.5 creating build/temp.cygwin-2.6.0-i686-3.5/simplejson gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/py35/include/python3.5m -c simplejson/_speedups.c -o build/temp.cygwin-2.6.0-i686-3.5/simplejson/_speedups.o gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-2.6.0-i686-3.5/simplejson/_speedups.o -L/opt/py35/lib/python3.5/config-3.5m -L/opt/py35/lib -lpython3.5m -o build/lib.cygwin-2.6.0-i686-3.5/simplejson/_speedups.cpython-35m.dll running install_lib (snip) |
|
|
| msg277225 - (view) |
Author: Erik Bray (erik.bray) * |
Date: 2016-09-22 13:55 |
| Masayuki--concerning your above comments, I think this is similar to, if not the same as #13756 |
|
|
| msg277254 - (view) |
Author: Masayuki Yamamoto (masamoto) * |
Date: 2016-09-23 08:38 |
| I don't know why Cygwin packaging script (cygport) doesn't copy direct libpythonX.Ym.dll.a to /usr/lib. Hence I wrote the patch that doesn't conflict with cygport script. In current makefile rule for Cygwin, Install (LDLIBRARY)into(LDLIBRARY) into (LDLIBRARY)into(LIBPL). Therefore I prefer symlink from (LIBPL)thancopyingdirectlibraryfileto(LIBPL) than copying direct library file to (LIBPL)thancopyingdirectlibraryfileto(LIBDIR). What do you think of makefile rule addition that is same effect (symlink from LIBPL) as cygport script? |
|
|
| msg404426 - (view) |
Author: Irit Katriel (iritkatriel) *  |
Date: 2021-10-20 11:09 |
| See also Issue19241. |
|
|