[Numpy-discussion] Build questions, atlas, lapack... (original) (raw)

Fernando Perez fperez.net at gmail.com
Thu Jun 8 18:48:27 EDT 2006

Hi all,

I'm starting the transition of a large code from Numeric to numpy, so I am now doing a fresh build with a lot more care than before, actually reading all the intermediate messages. I am a bit puzzled and could use some help.

This is all on an ubuntu dapper box with the atlas-sse2 packages (and everything else recommended installed). By running as suggested in the scipy readme:

python ~/tmp/local/lib/python2.4/site-packages/numpy/distutils/system_info.py

I get the following message at some point:

==================================== atlas_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/atlas,/usr/lib/sse2 ) looking libraries f77blas,cblas,atlas in /usr/local/lib but found None looking libraries f77blas,cblas,atlas in /usr/local/lib but found None looking libraries lapack_atlas in /usr/local/lib but found None looking libraries lapack_atlas in /usr/local/lib but found None looking libraries f77blas,cblas,atlas in /usr/lib/atlas but found None looking libraries f77blas,cblas,atlas in /usr/lib/atlas but found None looking libraries lapack_atlas in /usr/lib/atlas but found None looking libraries lapack_atlas in /usr/lib/atlas but found None ( paths: /usr/lib/sse2/libf77blas.so ) ( paths: /usr/lib/sse2/libcblas.so ) ( paths: /usr/lib/sse2/libatlas.so ) ( paths: /usr/lib/sse2/liblapack_atlas.so ) looking libraries lapack in /usr/lib/sse2 but found None looking libraries lapack in /usr/lib/sse2 but found None looking libraries f77blas,cblas,atlas in /usr/lib but found None looking libraries f77blas,cblas,atlas in /usr/lib but found None looking libraries lapack_atlas in /usr/lib but found None looking libraries lapack_atlas in /usr/lib but found None system_info.atlas_info ( include_dirs = /usr/local/include:/usr/include ) ( paths: /usr/include/atlas_misc.h,/usr/include/atlas_enum.h,/usr/include/atlas_aux.h,/usr/include/atlas_type.h ) /usr/local/installers/src/scipy/numpy/numpy/distutils/system_info.py:870: UserWarning:

Could not find lapack library within the ATLAS installation.

warnings.warn(message) ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/atlas,/usr/lib/sse2 ) FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib/sse2'] language = c define_macros = [('ATLAS_WITHOUT_LAPACK', None)]

====================================

What I find very puzzling here is that later on, the following goes by:

lapack_atlas_info: ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/atlas,/usr/lib/sse2 ) looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/local/lib but found None looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/local/lib but found None looking libraries lapack_atlas in /usr/local/lib but found None looking libraries lapack_atlas in /usr/local/lib but found None looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib/atlas but found None looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib/atlas but found None looking libraries lapack_atlas in /usr/lib/atlas but found None looking libraries lapack_atlas in /usr/lib/atlas but found None ( paths: /usr/lib/sse2/liblapack_atlas.so ) ( paths: /usr/lib/sse2/libf77blas.so ) ( paths: /usr/lib/sse2/libcblas.so ) ( paths: /usr/lib/sse2/libatlas.so ) ( paths: /usr/lib/sse2/liblapack_atlas.so ) looking libraries lapack in /usr/lib/sse2 but found None looking libraries lapack in /usr/lib/sse2 but found None looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib but found None looking libraries lapack_atlas,f77blas,cblas,atlas in /usr/lib but found None looking libraries lapack_atlas in /usr/lib but found None looking libraries lapack_atlas in /usr/lib but found None system_info.lapack_atlas_info ( include_dirs = /usr/local/include:/usr/include ) ( paths: /usr/include/atlas_misc.h,/usr/include/atlas_enum.h,/usr/include/atlas_aux.h,/usr/include/atlas_type.h ) ( library_dirs = /usr/local/lib:/usr/lib ) ( paths: /usr/lib/atlas,/usr/lib/sse2 ) FOUND: libraries = ['lapack_atlas', 'f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib/sse2'] language = c define_macros = [('ATLAS_WITH_LAPACK_ATLAS', None)]

==============================================

Does the second mean that it /is/ finding the right libraries? Since the first search in atlas_info is also printing

( paths: /usr/lib/sse2/liblapack_atlas.so )

I don't quite understand why it then reports the warning.

For reference, here's the content of the relevant directories on my system:

============================================== longs[sse2]> ls /usr/lib/sse2 libatlas.a libcblas.a libf77blas.a liblapack_atlas.a libatlas.so@ libcblas.so@ libf77blas.so@ liblapack_atlas.so@ libatlas.so.3@ libcblas.so.3@ libf77blas.so.3@ liblapack_atlas.so.3@ libatlas.so.3.0 libcblas.so.3.0 libf77blas.so.3.0 liblapack_atlas.so.3.0

longs[sse2]> ls /usr/lib/atlas/sse2/ libblas.a libblas.so.3@ liblapack.a liblapack.so.3@ libblas.so@ libblas.so.3.0 liblapack.so@ liblapack.so.3.0

==============================================

In summary, I don't really know if this is actually finding what it wants or not, given the two messages.

Cheers,

f

ps - it's worth mentioning that the sequence:

python ~/tmp/local/lib/python2.4/site-packages/numpy/distutils/system_info.py

gets itself into a nasty recursion where it fires the interactive session 3 times in a row. And in doing so, it splits its own output in a funny way:

[...]

blas_opt_info:

Starting interactive session

Tasks: i - Show python/platform/machine information ie - Show environment information c - Show C compilers information c - Set C compiler (current:None) f - Show Fortran compilers information f - Set Fortran compiler (current:None) e - Edit proposed sys.argv[1:].

Task aliases: 0 - Configure 1 - Build 2 - Install 2 - Install with prefix. 3 - Inplace build 4 - Source distribution 5 - Binary distribution

Proposed sys.argv = ['/home/fperez/tmp/local/lib/python2.4/site-packages/numpy/distutils/system_info.py']

Choose a task (^D to quit, Enter to continue with setup):

msg:

( library_dirs = /usr/local/lib:/usr/lib ) FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['/usr/lib/sse2'] language = c define_macros = [('NO_ATLAS_INFO', 2)]

=================

I tried to fix it, but the call sequence in that code is convoluted enough that after a few 'import traceback;traceback.print_stack()' tries I sort of gave up. That code is rather (how can I say this nicely) pasta-like :), and thoroughly uncommented, so I'm afraid I won't be able to contribute a cleanup here.

I think this tool should run by default in a mode with NO attempt to fire a command-line subsystem of its own, so users can simply run

python /path/to/system_info > system_info.log

for further analysis.

More information about the NumPy-Discussion mailing list