| msg27412 - (view) |
Author: Fazal Majid (majid) |
Date: 2006-02-03 02:25 |
| On UNIX, mmapmodule.c makes a call to dup(2) prior to mapping the file descriptor into memory (oddly enough, it doesn't map the new fd resulting from dup(), but the old one). The close method does not release this file descriptor, however, leading to a leak. If mmap is used repeatedly, the process will eventually run out of file descriptors, as can easily be shown with this test case: import sys, os, mmap for i in xrange(2000): f = os.open('/dev/zero', os.O_RDWR) m = mmap.mmap(f, 10000) os.close(f) a = m[0:100] m[100:200] = 'F' * 100 m.close() |
|
|
| msg27413 - (view) |
Author: Fazal Majid (majid) |
Date: 2006-02-03 02:27 |
| Logged In: YES user_id=110477 SourceForge munged the formatting of the test case, so I am attaching a copy as well. |
|
|
| msg27414 - (view) |
Author: Fazal Majid (majid) |
Date: 2006-02-03 02:44 |
| Logged In: YES user_id=110477 The following patch closes the file descriptor when the mmap object's close method is called, and seems to resolve this problem. |
|
|
| msg27415 - (view) |
Author: Keith Dart (kdart) |
Date: 2006-02-03 14:51 |
| Logged In: YES user_id=16527 I would also like to add that the following no longer works in Python 2.4: _buf = mmap.mmap(-1, 8192, flags=mmap.MAP_PRIVATE|mmap.MAP_ANONYMOUS, prot=mmap.PROT_READ |
mmap.PROT_WRITE ) This is because the fd value of -1 raises an exception because it is being dup-ed inside the mmap module. But the fd is ignored in anonymous mmaps and does not need to be dup-ed. This worked in older Python (2.3). Is the dup() call really necessary? |
|
| msg27416 - (view) |
Author: Fazal Majid (majid) |
Date: 2006-02-03 17:43 |
| Logged In: YES user_id=110477 From my reading of the source, the fd is kept around just to support the mmap.size() and mmap.resize() methods. I tend to agree, tying down a fd is wasteful. |
|
|
| msg27417 - (view) |
Author: Neal Norwitz (nnorwitz) *  |
Date: 2006-02-05 03:25 |
| Logged In: YES user_id=33168 A similar fix was checked in to 41368 (2.4) and 41366 on 01 Nov 2005. Thanks. There shouldn't be a problem using 2.4 SVN HEAD or 2.5 HEAD. |
|
|