| msg73204 - (view) |
Author: Reed O'Brien (robrien) |
Date: 2008-09-14 01:30 |
| or in FreeBSD? 2.6rc1 and 3.0b3 both fail test_array on FreeBSD7 amd64 test_array passes in 2.5.2 on the same machine but fails test_list the same as test_array... *** Signal 9 |
|
|
| msg73209 - (view) |
Author: Andrew I MacIntyre (aimacintyre) *  |
Date: 2008-09-14 10:56 |
| 2.6rc1 test_array passes on FreeBSD 6.3 i386. I don't have a 7.0 box (either i386 or amd64) accessible at the moment to cross check. Can you run the test on its own in verbose mode to get a bit more of an idea where its blowing up? - ie: user@box$ ./python -E -tt Lib/test/regrtest.py -v test_array from the build directory. |
|
|
| msg73225 - (view) |
Author: Reed O'Brien (robrien) |
Date: 2008-09-14 14:40 |
| 2.6rc2 and Python-3.0b3 test_array detail test_alloc_overflow (test.test_array.DoubleTest) ... Killed Fills swap space and dumps core. 2.5.2 test_list test_addmul (test.test_list.ListTest) ... ok test_append (test.test_list.ListTest) ... ok Killed The FreeBSD ports patches fix this in 2.5.2. Specifically patching seq_tests.py to limit test_bigrepeat() to if sys.maxint <= 2147483647. no other tests fail; so I don't know immediately what else is patched. Although there are about 25 patches for the 2.5 port. |
|
|
| msg73256 - (view) |
Author: Andrew I MacIntyre (aimacintyre) * |
Date: 2008-09-15 12:15 |
| I've temporarily got a 7.0 amd64 system running and can confirm what you see. I checked out the 2.5.2 port patches you mentioned, but all the ones that seemed related are already in the 2.6rc1 sources. On a hunch, I used ulimit -v is used to set a process virtual memory size limit, and the test completed satisfactorily (I set it to 1048756, ie 1GB). I'm as yet none the wiser as to what to do with this info... |
|
|
| msg73257 - (view) |
Author: Andrew I MacIntyre (aimacintyre) * |
Date: 2008-09-15 12:44 |
| On FreeBSD 7.0 i386, test_array passes without ulimit -v needing to be set (ie its still the default "unlimited". |
|
|
| msg73299 - (view) |
Author: Mark Dickinson (mark.dickinson) *  |
Date: 2008-09-16 13:10 |
| Does the attached patch help fix this at all? I encountered possibly the same problem on a 64-bit Mac build of Python, and this patch fixed the problem for me. |
|
|
| msg73335 - (view) |
Author: Andrew I MacIntyre (aimacintyre) * |
Date: 2008-09-17 16:15 |
| Mark, your patch will probably get the test to pass, but the underlying reason the test is failing appears to be unexpected behaviour of the platform malloc(). FreeBSD 7.0 introduced a new malloc() implementation that relies on mmap() and this is behaving differently to the malloc() implementation in FreeBSD 6.3 which relied on sbrk(). I have posted a query about the new malloc()'s behaviour to a FreeBSD forum and will report what I find out. |
|
|
| msg73367 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2008-09-18 08:23 |
| Thanks, Alan. I realised my answer was a shallow one after reading (too late!) the thread you started on python-dev. However, I have a suspicion that that particular array test (test_alloc_overflow) was merely meant to test the code in newarrayobject, at around line 427 of arraymodule.c, which looks like: /* Check for overflow */ if (nbytes / descr->itemsize != (size_t)size) { return PyErr_NoMemory(); } and that the test dated from an era when it was fairly safe to assume that a size_t was at most 32 bits. I'd guess that test_alloc_overflow was never intended to be a test of OS malloc failure behaviour. So the array test is wrong, and I think this patch should be applied anyway. I admit this doesn't help with the much more interesting question of what's going on with malloc on FreeBSD. |
|
|
| msg73368 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2008-09-18 08:24 |
| s/Alan/Andrew/. Need more coffee. Apologies. |
|
|
| msg73379 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2008-09-18 12:57 |
| The patch looks fine to me, except that the failure message is now out of date. |
|
|
| msg74076 - (view) |
Author: Andrew I MacIntyre (aimacintyre) * |
Date: 2008-09-30 13:35 |
| On FreeBSD 7.0 amd64, applying Mark's patch does get the test to pass. As noted on python-dev, the response I got from the author of the new malloc() package in FreeBSD 7.x indicates that as of 7.1 the allocator defaults to using sbrk() rather than mmap() (strategy selection is tunable with an environment variable) so the problematic behaviour should be masked once 7.1 is released. Martin: I'm missing something regarding your comment about the failure message - the exception text seems literally correct (though not particularly useful), though I haven't spent a lot of time trying to understand how test_array works. What would you suggest? |
|
|
| msg74078 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2008-09-30 13:56 |
| The 2**16 in the error message is wrong. Maybe the error message should be something like: "Attempt to create array of size larger than maxsize failed to raise MemoryError." A bit verbose, but more accurate. There's another error in that patch, too. The line: b * maxsize//3 + 1 should be b * (maxsize//3 + 1) (I was mistakenly reading the '*' as '*=', as in the first test.) |
|
|
| msg74096 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2008-09-30 19:48 |
| > The 2**16 in the error message is wrong. As he says. |
|
|
| msg74113 - (view) |
Author: Andrew I MacIntyre (aimacintyre) * |
Date: 2008-10-01 03:30 |
| Yes, the bleeding obvious became so after some sleep. Committed to trunk as r66708 with the extra fix and a more abbreviated failure message, which I hope is still semantically the same as Mark's suggested text. Will forward port as appropriate to py3k. |
|
|
| msg107979 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2010-06-17 01:18 |
| Was the fix forward ported or is this an issue for 3.1/2? |
|
|
| msg185413 - (view) |
Author: Georg Brandl (georg.brandl) * |
Date: 2013-03-28 10:00 |
| Closing this old pending issue due to lack of activity. |
|
|