[Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite (original) (raw)

Tim Peters tim at zope.com
Tue Sep 16 12:58:21 EDT 2003

[Jeremy Hylton]

I disabled the KEEPALIVESIZELIMIT, as you suggested, and that also fixed the problem.

[Tim

We don't know why, though, nor do we have a small test case, right?

[Jeremy]

Not a small test case, no, but I have boiled it down a bit. If you run the tests from a Zope checkout on the Zope-27-branch, you can provoke the crash with "test.py Template."

Sorry, I cannot. I tried a fresh Zope-2_7-branch, on Windows, with

released Python 2.3, release build
current CVS Python, release build
current CVS Python, debug build

"test.py Template" gave the same result in all cases: 77 tests run, 1 failure (AssertionError: HTML Output Changed) in checkStringExpressions.

But note that Py_UNICODE is unsigned short on this box, and reading a trash one of those as an index isn't nearly as likely to reach into unmapped memory as on a box where you get 4 bytes of trash.

... On second thought, the test could be correct even if it does read up heap trash: the

unicodelatin1[unicode->str[0]] == unicode part is a very strong condition. If str[0] does contain heap trash, it can't pass, but it could cause Purify (etc) to whine about reading uninitialized memory. If we don't care about that,

It could also cause a segmentation violation depending on the value of the stack trash it reads. At least, that's what appears to be going wrong.

Right, I meant the condition is very strong if the unicode->str[0] part is known to lie in range(256). unicode_latin1 is statically declared to contain 256 elements, so if the trash index is in range(256) then the index resolves to legitimate memory.

unicode->length == 1 && (unsigned int)unicode->str[0] < 256U &&_ _unicodelatin1[unicode->str[0]] == unicode

would be a safe and correct guard.

That sounds better.

To worm around something that shouldn't happen at all, sure .

Still, it doesn't look to me like the code expected that str could contain uninitialized heap trash at this point, so I'd like someone who thinks they understand this code to think about how that could happen (it apparently is happening, so "beats me" isn't good enough ).

I certainly don't understand the code yet.

Me neither, but offhand didn't see anything glaringly wrong. If some internal Unicode operation decided to allocate a short Unicode string, but freed it before filling in any of the string bytes, I suppose the keepalive optimization would retain a Unicode object with uninitialized str space on the unicode free list. A subsequent _PyUnicode_New could grab that and try to boost its ->str size. That could explain it.

More information about the Python-Dev mailing list