Issue 16592: stringlib_bytes_join doesn't raise MemoryError on allocation failure (original) (raw)

Created on 2012-12-02 01:27 by pitrou, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (5)
msg176767 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2012-12-02 01:27
>>> l = [b''] * (100*1024*1024) [104918914 refs] >>> d = b''.join(l) Traceback (most recent call last): File "", line 1, in SystemError: error return without exception set (you'll have to adjust the list size based on your system memory size)
msg176770 - (view)	Author: Roundup Robot (python-dev)	Date: 2012-12-02 06:56
New changeset 9af5a2611202 by Christian Heimes in branch 'default': Issue #16592: stringlib_bytes_join doesn't raise MemoryError on allocation failure http://hg.python.org/cpython/rev/9af5a2611202
msg176771 - (view)	Author: Christian Heimes (christian.heimes) *	Date: 2012-12-02 06:59
Antoine, on Unix you can restrict the address space of a program to test the issue without almost crashing and OOMing your box. ;) >>> import resource >>> resource.setrlimit(resource.RLIMIT_AS, (1024*1024*100, 1024*1024*100)) >>> l = [b''] * (100*1024*70) >>> d = b''.join(l) Traceback (most recent call last): File "", line 1, in MemoryError I wonder why I don't see a memory error in Python 3.3 or earlier. Any idea?
msg176778 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2012-12-02 09:49
> I wonder why I don't see a memory error in Python 3.3 or earlier. Any idea? See issue #15958. Now join() accept arbitrary buffer objects, which can be modified during iteration, and therefore need a temporary list of Py_buffers.
msg176779 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2012-12-02 09:54
> Antoine, on Unix you can restrict the address space of a program to > test the issue without almost crashing and OOMing your box. ;) Ah, thanks, but I didn't crash and OOM it, since it has no swap: memory errors get raised quickly here :)