| msg93295 - (view) |
Author: Carl Friedrich Bolz-Tereick (Carl.Friedrich.Bolz) * |
Date: 2009-09-29 14:35 |
| When unmarshalling a hand-written string it is possible to break the invariants of longs: Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> x = marshal.loads('l\x02\x00\x00\x00\x00\x00\x00\x00') >>> print x 00000 >>> x == 0 False >>> bool(x) True >>> x + 1 1L I would expect this to raise an error instead. |
|
|
| msg93298 - (view) |
Author: Mark Dickinson (mark.dickinson) *  |
Date: 2009-09-29 15:14 |
| Thanks for the report; I'll take a look. Though in general, I think this comes under the heading of 'Don't do that, then.' I suspect there are many ways to produce odd behaviour by feeding malformed input to marshal. See the big pink boxes at: http://docs.python.org/library/marshal.html :-) |
|
|
| msg93299 - (view) |
Author: Carl Friedrich Bolz-Tereick (Carl.Friedrich.Bolz) * |
Date: 2009-09-29 15:25 |
| Yes, I know :-). I thought I'd report it anyway, as it does more than "just" give you random behavior, but actually produces a broken object. |
|
|
| msg93302 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-09-29 16:39 |
| So there are two options: (1) raise an exception, or (2) strip leading zero digits and produce a properly normalized Python long. Do you have any strong preferences? I'm inclined towards (2). How did you encounter this in the first place? |
|
|
| msg93303 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-09-29 16:55 |
| On second thoughts, I think Carl was right in the first place: it's better to raise an exception. If marshal.loads is getting input that doesn't precisely match the marshal.dumps format, then something's wrong somewhere. Looks like we missed the window for 2.6.3. :-( |
|
|
| msg93306 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-09-29 17:39 |
| Here's a patch for Python 2.6. |
|
|
| msg93308 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-09-29 17:54 |
| And a patch for trunk. (Significantly different, because marshal for Python longs was reworked when 30-bit long digits were added.) |
|
|
| msg93310 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-09-29 18:00 |
| Changing versions: 3.1 and 3.2 also need fixing (the trunk patch should be easily mergeable); it's not a security issue (AFAIK), so won't be fixed in 2.5. |
|
|
| msg93319 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-09-29 19:26 |
| Fixed in r75141 (trunk), r75145 (py3k) and r75146 (release31-maint). I'll apply the 2.6 fix once the release26-maint branch is unfrozen. |
|
|
| msg93411 - (view) |
Author: Carl Friedrich Bolz-Tereick (Carl.Friedrich.Bolz) * |
Date: 2009-10-01 16:19 |
| [...] > How did you encounter this in the first place? I was working on PyPy's marshaler, broke it in such a way that it was producing the bad input that I gave above. Then I fixed it, but didn't kill my .pyc files. So a few days later I got a long zero with the properties above in PyPy (which was quite annoying to debug). Then I figured that CPython likely has the same problem. Thanks a lot for fixing this! Carl Friedrich |
|
|
| msg93481 - (view) |
Author: Mark Dickinson (mark.dickinson) * |
Date: 2009-10-03 08:16 |
| 2.6 fix applied in r75203. |
|
|