| msg171875 - (view) |
Author: Masami HIRATA (msmhrt) |
Date: 2012-10-03 09:09 |
| I'm trying to install C extension modules inside a venv. It works outside a venv and inside a virtualenv-1.8.2 but breaks inside the venv. OS: Windows 7 Starter Edition SP1 (32-bit) Python: 3.3.0 (python-3.3.0.msi) Compiler: Microsoft Visual C++ 2010 Express SP1 |
|
|
| msg171953 - (view) |
Author: Vinay Sajip (vinay.sajip) *  |
Date: 2012-10-04 14:23 |
| Comparing the link command which works: C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:c:\python33\Libs /LIBPATH:C:\temp\venv2\libs /LIBPATH:C:\temp\venv2\PCbuild /EXPORT:PyInit__regex build\temp.win32-3.3\Release\Python3\_regex.obj build\temp.win32-3.3\Release\Python3\_regex_unicode.obj /OUT:build\lib.win32-3.3\_regex.pyd /IMPLIB:build\temp.win32-3.3\Release\Python3\_regex.lib /MANIFESTFILE:build\temp.win32-3.3\Release\Python3\_regex.pyd.manifest with the one that fails: C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\temp\venv\libs /LIBPATH:C:\Python33 /LIBPATH:C:\temp\venv \PCbuild /EXPORT:PyInit__regex build\temp.win32-3.3\Release\Python3\_regex.obj build\temp.win32-3.3\Release\Python3\_regex_unicode.obj /OUT:build\lib.win32-3.3\ _regex.pyd /IMPLIB:build\temp.win32-3.3\Release\Python3\_regex.lib /MANIFESTFILE:build\temp.win32-3.3\Release\Python3\_regex.pyd.manifest indicates that the link error happens because the library path is incorrectly specified as c:\Python33 rather than c:\Python33\libs. I will investigate further. |
|
|
| msg171960 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2012-10-04 15:07 |
| A little more investigation suggests that the disparity is due to some code in virtualenv's version of distutils.__init__.py, which adds the "\Libs" entry which allows the virtualenv version to work. Adding Carl Meyer as nosy, as I'd like his advice on what the best approach to the fix would be. |
|
|
| msg171961 - (view) |
Author: Carl Meyer (carljm) * |
Date: 2012-10-04 15:20 |
| On cursory inspection, I agree that this is precisely what the "if win32" block in `virtualenv_embedded/distutils-init.py` is intended to fix, and it seems to me the correct fix is likely to just make the equivalent fix directly in distutils: change the library_dirs-building code in `distutils.command.build_ext:finalize_options` (under the "if os.name == 'nt'" block) to build the path relative to `sys.base_exec_prefix` rather than `sys.exec_prefix`. |
|
|
| msg171962 - (view) |
Author: Carl Meyer (carljm) * |
Date: 2012-10-04 15:21 |
| (Actually, to match virtualenv's fix it should add the paths based on both exec_prefix and base_exec_prefix, if they are different.) |
|
|
| msg171970 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2012-10-04 16:29 |
| Since I expect it may be some time before the next Python release is out, ISTM a temporary workaround would be to add a line following line 192 of Lib\distutils\command\build_ext.py, which reads self.library_dirs.append(os.path.join(sys.exec_prefix, 'libs')) The new line should read self.library_dirs.append(os.path.join(sys.base_exec_prefix, 'libs')) Which will result in adding both directories to the library search path. This is a little untidy in the case when you're not in a venv, but it should still work. With that change, the link command now looks like C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\temp\venv\libs /LIBPATH:C:\Python33\libs /LIBPATH:C:\Python33 /LIBPATH:C:\temp\venv\PCbuild /EXPORT:PyInit__regex build\temp.win32-3.3\Release\Python3\_regex.obj build\temp.win32-3.3\Release\Python3\_regex_unicode.obj /OUT:build\lib.win32-3.3\_regex.pyd /IMPLIB:build\temp.win32-3.3\Release\Python3\_regex.lib /MANIFESTFILE:build\temp.win32-3.3\Release\Python3\_regex.pyd.manifest and the command succeeds. |
|
|
| msg171972 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2012-10-04 16:33 |
| If you like, you can make the added line conditional on "if sys.base_exec_prefix != sys.prefix", which is the form the actual fix is likely to take. Thus: if sys.base_exec_prefix != sys.prefix: self.library_dirs.append(os.path.join(sys.base_exec_prefix, 'libs')) |
|
|
| msg172063 - (view) |
Author: Masami HIRATA (msmhrt) |
Date: 2012-10-05 09:34 |
| I have tested the workaround and it works correctly. Please see attached log file. |
|
|
| msg172068 - (view) |
Author: Masami HIRATA (msmhrt) |
Date: 2012-10-05 10:28 |
| btw, it seems to me that "-IC:\Python33\include -IC:\Python33\include" should be "-IC:\Users\msmhrt\mypython\3.3.0\include -IC:\Python33\include". What do you think about it? |
|
|
| msg172094 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2012-10-05 16:00 |
| > I have tested the workaround and it works correctly. > Please see attached log file. Thanks for the update. > btw, it seems to me that "-IC:\Python33\include -IC:\Python33\include" > should be "-IC:\Users\msmhrt\mypython\3.3.0\include -IC:\Python33\include". Could well be a similar problem; I'll look into it. |
|
|
| msg173570 - (view) |
Author: Masami HIRATA (msmhrt) |
Date: 2012-10-22 23:12 |
| > Could well be a similar problem; I'll look into it. Is there any progress? |
|
|
| msg173584 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2012-10-23 08:45 |
| Sorry, not yet - I haven't been able to spend much time looking at it, but hopefully I will be able to before too long. |
|
|
| msg173607 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2012-10-23 12:49 |
| Added fixes in my sandbox repo. Please verify. |
|
|
| msg173624 - (view) |
Author: Masami HIRATA (msmhrt) |
Date: 2012-10-23 16:39 |
| I attached new log file. (generated with 3.3.0 + your patch) |
|
|
| msg173627 - (view) |
Author: Vinay Sajip (vinay.sajip) * |
Date: 2012-10-23 18:06 |
| So you agree that it's working? The include path has the venv's include in it, and the c:\Python33\libs inclusion allows the linking to work. I'll commit the changes to 3.3, 3.4 and close the issue shortly. |
|
|
| msg173635 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2012-10-23 19:27 |
| New changeset 43f537339107 by Vinay Sajip in branch '3.3': Issue #16116: Now uses corrected include and library paths when building C extensions in a venv. http://hg.python.org/cpython/rev/43f537339107 New changeset 5e9f656c3d67 by Vinay Sajip in branch 'default': Closes #16116: Merged fix from 3.3. http://hg.python.org/cpython/rev/5e9f656c3d67 |
|
|
| msg173636 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2012-10-23 20:11 |
| >+- Issue #16116: Fix include and library paths to be correctwhen building C Spaces. |
|
|
| msg173645 - (view) |
Author: Masami HIRATA (msmhrt) |
Date: 2012-10-23 22:40 |
| > So you agree that it's working? Yes, the patch works correctly. Thank you! |
|
|