Issue 16907: Distutils fails to build extension in path with spaces (original) (raw)
Created on 2013-01-09 12:22 by klo.uo, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (10)
Author: klo uo (klo.uo)
Date: 2013-01-09 12:22
I noticed this issue, while trying to compile Cython extension, when source file is in path with spaces. Extension wouldn't compile because LIBDIR is passed to MinGW g++ (though not to gcc) or MSVC compilers unquoted. I tracked the problem to "distutils/build_ext.py" in get_ext_fullpath() function.
I patched it, this way:
======================================== --- build_ext.bak +++ build_ext.py @@ -647,6 +647,11 @@ package = '.'.join(modpath[0:-1]) build_py = self.get_finalized_command('build_py') package_dir = os.path.abspath(build_py.get_package_dir(package)) + try: + from win32api import GetShortPathName + package_dir = GetShortPathName(package_dir) + except: + pass # returning # package_dir/filename return os.path.join(package_dir, filename)
which is just one way to do it.
Author: Ramchandra Apte (Ramchandra Apte) *
Date: 2013-01-09 13:17
Can you fix the bare except?
Author: klo uo (klo.uo)
Date: 2013-01-09 13:22
Can you fix the bare except?
======================================== ... except ImportError: pass
Author: Serhiy Storchaka (serhiy.storchaka) * 
Date: 2013-01-09 14:24
get_ext_fullpath() has no relations to this bug. The bug is probably in spawn() functions, perhaps in _nt_quote_args() or something like. It looks as Windows specific bug (Posix passes command arguments as a list, not joining it into string).
Author: Martin v. Löwis (loewis) *
Date: 2013-01-09 16:27
klo.uo: can you kindly provide a working (or, rather, failing) example? A trivial "hello-world" kind of package could do, along with a report what path you unpacked it in, and what error you get.
Your patch is not applicable to Python, since it requires the win32 extensions, which we cannot assume to be present.
Author: Éric Araujo (eric.araujo) *
Date: 2013-01-09 18:06
I think this is a duplicate report; could you please search for an existing open report for the same bug?
Author: klo uo (klo.uo)
Date: 2013-01-09 18:15
klo.uo: can you kindly provide a working (or, rather, failing) example? A trivial "hello-world" kind of package could do, along with a report what path you unpacked it in, and what error you get.
As mentioned in opening thread, this doesn't happen with Cython's
"hello world" example, as gcc somehow doesn't trigger this problem.
Maybe because it's handled by Cython's own distutils copies, maybe it's
handled by Numpy's distutils version or it could be by Python's
distutils.
I noticed this issue while using Numpy with Cython. Here is simple example:
C:\Documents and Settings\klo\My Documents\code\python\misc\test.pyx:
cimport numpy
def sum(x): cdef numpy.ndarray[int, ndim=1] arr = x cdef int i, s = 0 for i in range(arr.shape[0]): s += arr[i] return s
C:\Documents and Settings\klo\My Documents\code\python\misc\setup.py:
from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext from numpy.distutils.misc_util import get_numpy_include_dirs
setup( cmdclass = {'build_ext': build_ext}, ext_modules = [Extension("test", ["test.pyx"], include_dirs=get_numpy_include_dirs())] )
command line: python setup.py build_ext --inplace
MinGW result:
... g++ -shared build\temp.win32-2.7\Release\test.o -LC:\Python27\libs -LC:\Python27\PCbuild -lpython27 -lmsvcr90 -o C:\Documents and Settings\klo\My Documents\code\python\misc\test.pyd Found executable C:\MinGW\bin\g++.exe g++.exe: error: and: No such file or directory g++.exe: error: Settings\klo\My: No such file or directory g++.exe: error: Documents\code\python\misc\test.pyd: No such file or directory error: Command "g++ -shared build\temp.win32-2.7\Release\test.o -LC:\Python27\libs -LC:\Python27\PCbuild -lpython27 -lmsvcr90 -o C:\Documents and Settings\klo\My Documents\code\python\misc\test.pyd" failed with exit status 1
Similar result if I use MSVC compiler:
MSVC result:
building 'test' extension c:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Python27\lib\site-packages\numpy\core\include -IC:\Python27\include -IC:\Python27\PC /Tctest.c /Fobuild\temp.win32-2.7\Release\test.obj Found executable c:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\cl.exe c:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Python27\libs /LIBPATH:C:\Python27\PCbuild /EXPORT:inittest build\temp.win32-2.7\Release\test.obj /OUT:C:\Documents and Settings\klo\My Documents\code\python\misc\test.pyd /IMPLIB:build\temp.win32-2.7\Release\test.lib /MANIFESTFILE:build\temp.win32-2.7\Release\test.pyd.manifest /MANIFEST Found executable c:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe LINK : fatal error LNK1181: cannot open input file 'and.obj' error: Command "c:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Python27\libs /LIBPATH:C:\Python27\PCbuild /EXPORT:inittest build\temp.win32-2.7\Release\test.obj /OUT:C:\Documents and Settings\klo\My Documents\code\python\misc\test.pyd /IMPLIB:build\temp.win32-2.7\Release\test.lib /MANIFESTFILE:build\temp.win32-2.7\Release\test.pyd.manifest /MANIFEST" failed with exit status 1181
So issue is with unquoted path for output file, and not for LIBDIR as I wrote previously (perhaps I misread the logs, as I was trying to fix similar issue for Theano, at the same time)
Your patch is not applicable to Python, since it requires the win32 extensions, which we cannot assume to be present.
Yes, I provided patch that worked for me temporarily, as I'm not familiar with distutils, and that's as far as I went. But source issue is probably elsewhere, as Serhiy suggested.
Author: klo uo (klo.uo)
Date: 2013-01-09 18:21
I found two similar issues:
- distutils compiler not handling spaces in path to output/src files: http://bugs.python.org/issue4508
with patches for unix compiler and cygwin compiler
- Distutils does not put quotes around paths that contain spaces when compiling with MSVC: http://bugs.python.org/issue13765
with some patch for MSVC
Problem seems to be deeper than provided patches, as evident also from my report
Author: Mark Lawrence (BreamoreBoy) *
Date: 2014-07-30 18:03
Of the two issues mentioned in #4508 is still open but #13765 has been closed "not a bug".
Author: Zachary Ware (zach.ware) *
Date: 2016-01-18 21:59
This isn't obviously a distutils bug, but sounds strikingly similar to #4508. Closing as a duplicate of #4508.
History
Date
User
Action
Args
2022-04-11 14:57:40
admin
set
github: 61111
2016-01-18 21:59:58
zach.ware
set
status: open -> closed
superseder: distutils compiler not handling spaces in path to output/src files
assignee: eric.araujo ->
nosy: + zach.ware
messages: +
resolution: duplicate
stage: needs patch -> resolved
2014-07-30 18:03:59
BreamoreBoy
set
nosy: + BreamoreBoy
messages: +
versions: + Python 3.4, Python 3.5
2013-01-09 18:21:32
klo.uo
set
messages: +
2013-01-09 18:15:50
klo.uo
set
messages: +
2013-01-09 18:06:22
eric.araujo
set
messages: +
2013-01-09 16:27:58
loewis
set
nosy: + loewis
messages: +
2013-01-09 14:24:33
serhiy.storchaka
set
nosy: + serhiy.storchaka
messages: +
components: + Windows
type: behavior
stage: needs patch
2013-01-09 13:22:14
klo.uo
set
messages: +
2013-01-09 13:17:22
Ramchandra Apte
set
nosy: + Ramchandra Apte
messages: +
2013-01-09 12:22:42
klo.uo
create