Issue 4359: at runtime, distutils uses buildtime files (original) (raw)

Issue4359

Created on 2008-11-19 23:52 by a.badger, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (11)
msg76083 - (view)	Author: Toshio Kuratomi (a.badger) *	Date: 2008-11-19 23:52
When using some distutils functions, distutils attempts to use buildtime files like Makefile and pyconfig*.h as data sources. For instance, this snippet:: from distutils.command.install import install from distutils.core import Distribution dist = Distribution({"name": "foopkg"}) cmd = install(dist) cmd.ensure_finalized() There's two reasons this should change. 1) Some Linux distributions separate the python runtime and buildtime files and put the buildtime files in a -devel package. Depending on these buildtime files means that the -devel package can be needed for running python scripts. For instance, here's the traceback that occurs when the previous commands are run without python-devel on Fedora Linux:: Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.5/distutils/cmd.py", line 117, in ensure_finalized self.finalize_options() File "/usr/lib/python2.5/distutils/command/install.py", line 273, in finalize_options (prefix, exec_prefix) = get_config_vars('prefix', 'exec_prefix') File "/usr/lib/python2.5/distutils/sysconfig.py", line 493, in get_config_vars func() File "/usr/lib/python2.5/distutils/sysconfig.py", line 352, in _init_posix raise DistutilsPlatformError(my_msg) distutils.errors.DistutilsPlatformError: invalid Python installation: unable to open /usr/lib/python2.5/config/Makefile (No such file or directory) 2) keeping the information in a Makefile and *.h files and then having regular expressions pull the information out is fragile and not what the tools were meant for. Using a defined data format is much better. The variables necessary for building extensions should be placed in a data file of some sort. This can be built by the configure script at the same time as it's substituting variables into the Makefile and pyconfig files. xml is good for interoperability and we have good modules in the std library for that now. .ini is less verbose and we have modules to deal with that as well.
msg94707 - (view)	Author: Andrew McNabb (amcnabb)	Date: 2009-10-30 15:21
I've noticed this, too, and I agree with Toshio's observations. Tarek, do you have any opinions?
msg94711 - (view)	Author: Dave Malcolm (dmalcolm)	Date: 2009-10-30 15:54
For Fedora, I've fixed this (I hope) by special-casing those two files: %{_libdir}/python%{pybasever}/config/Makefile /usr/include/python2.6/pyconfig-{32|64}.h making them part of the core python package. See downstream bug here: https://bugzilla.redhat.com/show_bug.cgi?id=531901 (BTW, downstream, we rename pyconfig.h and generate a stub that #includes either pyconfig- 32/64 so that both 32 and 64-bit devel packages can be installed on 32-bit hosts; we patch distutils so that it parses the wordlength-specific file)
msg94712 - (view)	Author: Dave Malcolm (dmalcolm)	Date: 2009-10-30 15:56
> For Fedora, I've fixed this (I hope) by special-casing those two files: Sorry; for "fixed", read "addressed"; this covers part 1 of the issue, but not part 2.
msg95269 - (view)	Author: Tarek Ziadé (tarek) *	Date: 2009-11-15 01:11
This is a problem indeed. One solution would be to generate a module in the stdlib that contains all these info, when configure is called. as a matter of fact, I am currently working in a branch to add a module called "sysconfig" to the stdlib, that contains installation paths extracted from distutils/sysconfig and site.py, so the stdlib has only one place to handle those. This module will basically be the last spot to look for data in makefile and pyconfig.h, so maybe we could inject in it a condensed version of these files, in the form of a dict. (I'll send a mail on python-dev about this)
msg95271 - (view)	Author: Tarek Ziadé (tarek) *	Date: 2009-11-15 02:16
see http://mail.python.org/pipermail/python-dev/2009-November/094232.html (notice that the dependency in install can be removed easily because it just reads variables from sys and does not require to import sysconfig)
msg96578 - (view)	Author: Tarek Ziadé (tarek) *	Date: 2009-12-18 19:21
I am closing this since Fedora has fixed the issue on their side. On a side note: I am working on this new sysconfig module, in a branch called tarek_sysconfig
msg96594 - (view)	Author: Toshio Kuratomi (a.badger) *	Date: 2009-12-19 01:42
Hey tarek, the main thrust of this bug for me was storing the data in an inappropriate format and not having an API to get at it; things that I think the sysconfig branch will address. Does it make sense to have a bug to track that progress? Does it make sense for that bug to be this one or a new one?
msg115842 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2010-09-08 02:11
I think the future sysconfig module (http://bitbucket.org/tarek/distutils2/src/tip/docs/design/wiki.rst) addresses part of the issue: It will use a configparser file to store installation directories. There is nothing about other build-time variables, though; this could be added to finally remove the brittle Makefile parsing in sysconfig. There is no bug to track this work that I know of.
msg116145 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2010-09-12 01:08
#9807 has been recently opened.
msg119348 - (view)	Author: Éric Araujo (eric.araujo) *	Date: 2010-10-21 22:49
... and #9807 spawned #9878.