| msg30331 - (view) |
Author: Skip Montanaro (skip.montanaro) *  |
Date: 2006-10-21 18:02 |
| I noticed a test_sqlite test failure on my Mac G5 the other day while trying to set up a buildbot for sqlalchemy. Everything runs fine on my G4 powerbook. Both machines run Mac OSX 10.4.8 with Apple's gcc 4.0.0, build 5026. I whittled the problem down to just having test_ctypes and test_sqlite enabled, then further whittled it down to just having Lib/ctypes/test/ test_find.py available (all other ctypes tests eliminated). More detailed problem descriptions are in these two postings to python-dev: http://article.gmane.org/gmane.comp.python.devel/84478 http://article.gmane.org/gmane.comp.python.devel/84481 Skip |
|
|
| msg30332 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2006-10-21 18:04 |
| Logged In: YES user_id=44345 Another article here: http://article.gmane.org/gmane.comp.python.devel/84487 |
|
|
| msg30333 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-03-18 17:20 |
| Thomas, I assigned this to you simply so you can cast your eye on the problem and see if you think there might be anything ctypes-related in the problem I've reported. I've ignored it up 'til now, but I bumped up against it again trying to get the Pybots SQLAlchemy test suite running on my Mac. During my previous encounter with this problem Ronald Oussouren reported on a possible connection with GLUT: http://mail.python.org/pipermail/python-dev/2006-October/069523.html Assuming there is no obvious ctypes connection you can determine (it's not clear to me that there is any current directory funny business going on) just assign it back to me and I'll try to find a pysqlite expert to take a look. Thx, Skip |
|
|
| msg30334 - (view) |
Author: Thomas Heller (theller) *  |
Date: 2007-03-22 20:18 |
| Skip, I do not have a G5 machine available, only a G4 and the problem doesn't appear there. For someone with a G5 it should not be too difficult to reproduce (or investigate), I think that running './python Lib/test/regrtest.py test_ctypes test_sqlite' should suffice. However, if loading the GLUT library changes the cwd, and sqlite fails with a changed working dir, I'm not sure where the problem really is and how it should be fixed. It could be said that test_ctypes should restore the cwd after the tests (but maybe this should be done by the test framework?), or it could be said that test_sqlite should set the cwd in case the test relies on it. I'll assign back to you because I have no way to test it on a machine where the problem appears. |
|
|
| msg30335 - (view) |
Author: Thomas Heller (theller) * |
Date: 2007-05-04 14:04 |
| FYI, this PyOpenGL bug may be related: https://sourceforge.net/tracker/?func=detail&atid=105988&aid=1712741&group_id=5988 |
|
|
| msg30336 - (view) |
Author: ted pollari (pollari) |
Date: 2007-06-13 20:19 |
| FWIW, I'm getting an identical error on a MacBook Pro with a Core Duo processor |
|
|
| msg30337 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2007-06-29 19:11 |
| I've just noticed the same failure on an iBook G4, with OS X 10.4.10, sqlite version 3.4.0, and Python built with gcc 4.2.0 (from macports). (Python 2.6a0 (trunk:56123M, Jun 29 2007, 13:22:18) [GCC 4.2.0] on darwin) |
|
|
| msg30338 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2007-06-29 23:37 |
| And also on Python 2.5, compiled under Apple's GCC. (Still OS X 10.4.10/G4) The following four lines of Python code are enough to cause the problem for me. Interestingly, if the second and third lines are exchanged (so that the two import commands come first), then no exception is raised. Python 2.5.1 (r251:54863, May 10 2007, 20:59:25) [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from ctypes import CDLL >>> CDLL('/System/Library/Frameworks/GLUT.framework/GLUT').glutWireTetrahedron <_FuncPtr object at 0x514500> >>> import sqlite3 >>> sqlite3.connect(":memory:") Traceback (most recent call last): File "", line 1, in sqlite3.ProgrammingError: library routine called out of sequence |
|
|
| msg30339 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-06-30 02:39 |
| Fails for me as well (Python 2.6a0, PowerBook G4, 10.4.10, Apple's GCC 4.0.1). Thomas, can you reproduce with the lines marketdickinson used? I don't see any directory changing going on: % python Python 2.6a0 (trunk:54264M, Mar 10 2007, 15:19:48) [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.getcwd() '/Users/skip' >>> from ctypes import CDLL>>> CDLL('/System/Library/Frameworks/GLUT.framework/GLUT').glutWireTetrahedron<_FuncPtr object at 0x106c6c0> >>> os.getcwd() '/Users/skip' >>> import sqlite3 >>> sqlite3.connect(":memory:") Traceback (most recent call last): File "", line 1, in ProgrammingError: library routine called out of sequence >>> os.getcwd() '/Users/skip' Pass it back if you still can't reproduce the problem. Meanwhile I've subscribed to the pysqlite mailing list and will ask there. Skip |
|
|
| msg30340 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-06-30 02:45 |
| One other minor(?) data point. I am using a version of sqlite3 which I compiled and installed, not Apple's older version: >>> sqlite3.version, sqlite3.sqlite_version ('2.3.3', '3.3.8') ... % pwd /Users/skip/local/lib/python2.6/lib-dynload % otool -L _sqlite3.so _sqlite3.so: /Users/skip/local/lib/libsqlite3.0.dylib (compatibility version 9.0.0, current version 9.6.0) /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 93.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.7) It appears those of us who can reproduce the problem all have slightly different installs (2.5 v. 2.6, Apple's GCC v. DarwinPorts, Apple's sqlite3 v. DarwinPorts v. homebuilt, G4 v. G5 v. Intel architecture). |
|
|
| msg30341 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2007-06-30 03:32 |
| I think your minor(?) data point is a major data point. I'm using the Macports version of sqlite3. Furthermore, after hiding the /usr/lib version with fibonacci:/usr/lib dickinsm$ sudo mv libsqlite3.0.dylib libsqlite3.0.dylib.temp I can't reproduce the problem any more. I tried sprinkling a few printf statements through the sqlite3 source to see what was going on. Bizarrely, it looks as though the Apple-supplied sqlite3SafetyCheck() routine is being called from the Macports createCollation(); or at least that's the only way that I can understand the results I'm getting. Is this possible? Mark |
|
|
| msg30342 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2007-06-30 03:56 |
| It looks as though GLUT is a red herring. I can reproduce the problem with a variety of frameworks in place of GLUT, for example Quartz, Cocoa, AppKit, CoreData, as in the following lines. import ctypes ctypes.CDLL('/System/Library/Frameworks/CoreData.framework/CoreData') import sqlite3 sqlite3.connect(":memory:") Furthermore, I notice that the GLUT framework has a library dependency on the AppKit framework, AppKit on CoreData, and CoreData depends on the Apple-supplied libsqlite3, and that Quartz and Cocoa similarly ultimately depend on libsqlite3. Coincidence? I can reproduce the problem with a variety of frameworks in place of GLUT, for example Quartz, Cocoa, AppKit, CoreData. (The .glutWireTetrahedron is unnecessary.) |
|
|
| msg30343 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-06-30 10:42 |
| I'll be damned... Based on your last response I ran Python under ktrace then looked at the kdump output. As it's marching along it sees a mention of /usr/lib/libsqlite3.so in /System/Library/Frameworks/CoreData.framework/Versions/A/CoreData then just a few lines later I see 20023 python NAMI "/usr/lib/libsqlite3.0.dylib" 20023 python RET open 3 20023 python CALL fstat(0x3,0xbfffc770) 20023 python RET fstat 0 20023 python CALL pread(0x3,0xbfffcbd0,0x1000,0) then a bit later I see what I had wanted to see: 20023 python NAMI "/Users/skip/local/lib/python2.6/lib-dynload/_sqlite3.so" 20023 python RET open 6 20023 python CALL fstat(0x6,0xbfff9c00) 20023 python RET fstat 0 20023 python CALL pread(0x6,0xbfffa060,0x1000,0) ... 20023 python NAMI "/Users/skip/local/lib/libsqlite3.0.dylib" 20023 python RET open 6 20023 python CALL fstat(0x6,0xbfff99b0) 20023 python RET fstat 0 20023 python CALL pread(0x6,0xbfff9e10,0x1000,0) I can't execute the CDLL call if I've renamed /usr/lib/libsqlite.3.0.dylib. (I suppose that stands to reason.) Now the question becomes whether or not the Apple-provided library can be replaced with something more modern. Presumably every application that uses CoreData (most Apple things) gets it if during startup. Thomas, for the purposes of the Python regression test suite do you think you can find something other than GLUT to dynamically load? Mark, thanks for the sleuthing. Skip |
|
|
| msg30344 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-06-30 11:15 |
| I submitted a bug report to Apple about the problem, though I don't really expect a response. (The other bug I filed in 2004 has never received any response as far as I can tell.) Maybe a more recent version of SQLite will be delivered with Leopard... |
|
|
| msg30345 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-06-30 13:17 |
| Thomas, Here's a possible modification to ctypes.test.test_find which appears to be the only place that OpenGL (and thus CoreData on the Mac?) is accessed. On Darwin I replace the GL find stuff with Tcl find stuff. It's a bit ugly, but it might solve the problem. Skip File Added: test_find.diff |
|
|
| msg30346 - (view) |
Author: Thomas Heller (theller) * |
Date: 2007-07-04 20:01 |
| Skip, thanks for the analysis and test patch. IMO loading the GLUT library does not add significant value to the ctypes tests, so wouldn't it be best to skip the GLUT lib on OSX? See the attached patch "test_find without GLUT on OSX" (note that I didn't actually test the patch, and I don't have access to OSX G5 anyway). Maybe the problem and the analysis should be reported to the PyOpenGL project... File Added: test_find-no-osx-glut.diff |
|
|
| msg30347 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-07-05 01:31 |
| Certainly the patch is simpler. ;-) If loading GLUT doesn't add any value on OSX, would it add any value on other platforms? Do other tests have to find libraries successfully to pass? If so, maybe test_find should be dispensed with altogether, however if test_find has value we should figure out a library to load on OSX. |
|
|
| msg30348 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2007-07-12 10:41 |
| This certainly sounds like someone is fooling around with the -force_flat_namespace link flag, which could result in picking up the wrong version of symbols. I haven't checked yet if this is indeed the case here, and if so who is doing this. |
|
|
| msg30349 - (view) |
Author: Thomas Heller (theller) * |
Date: 2007-07-13 09:47 |
| Skip, IMO loading GLUT does not add any value (I just added it because GLUT is related to the OpenGL libraries, and because it worked [That's what I thought at least]). Finding and loading the other libs adds value because they must be loaded with RTLD_GLOBAL on some platforms, so I would like to keep that part. So, I suggest to remove the GLUT part from the tests altogether. |
|
|
| msg30350 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2007-07-13 15:05 |
| Go for it then. You are, after all, the expert. I'm crossing my fingers that this will finally solve the community buildbot failures I *always* get on mac os x when trying to test out sqlalchemy. Skip |
|
|
| msg30351 - (view) |
Author: Thomas Heller (theller) * |
Date: 2007-07-13 16:51 |
| Implemented in SVN rev. 56350. |
|
|
| msg62259 - (view) |
Author: Skip Montanaro (skip.montanaro) * |
Date: 2008-02-10 21:41 |
| I'm reopening this. I am seeing the same behavior now on my MacBook Pro running Mac OS X 10.5.1. Looking at the crash report in my ~/Library/Logs/CrashReporter directory I see both /usr/lib and /opt/local/lib versions of libsqlite3: 0x1188000 - 0x11defef +libsqlite3.0.dylib ??? (???) /opt/local/lib/libsqlite3.0.dylib 0x911f6000 - 0x9127dff7 libsqlite3.0.dylib ??? (???) <273efcb717e89c21207c851d7d33fda4> /usr/lib/libsqlite3.0.dylib I was able to work around this by disabling the MacPorts version of sqlite3. Others might not be so lucky. I don't see any references to sqlite in the ctypes test code. Must be an indirect reference. Skip |
|
|
| msg66592 - (view) |
Author: Humberto Diógenes (hdiogenes) * |
Date: 2008-05-10 22:53 |
| Skip, I had the same problem as you, with the same solution. On a MacBook Pro running Mac OS X 10.5.2, running test_ctypes and test_sqlite together would kill my machine. Found another installation of sqlite3 on /usr/local, removed it and now the test works fine. |
|
|
| msg87277 - (view) |
Author: Thomas Heller (theller) * |
Date: 2009-05-05 20:01 |
| So, should this be closed as some kind of 'works for me' + 'wont fix' + 'third-party-bug' in Apple's sqlite? |
|
|
| msg87278 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-05-05 20:11 |
| Probably. I do remember investigating this further, but I don't entirely remember what I found. I think Ronald's comment about flat namespaces turned out to be highly relevant though. |
|
|
| msg114842 - (view) |
Author: Mark Lawrence (BreamoreBoy) * |
Date: 2010-08-24 22:26 |
| Refer to and . |
|
|