| msg26755 - (view) |
Author: liturgist (liturgist) |
Date: 2005-10-28 20:26 |
| Creating an mmap object does not accept a length parameter of zero on Linux FC4 and Cygwin. However, it does on Windows XP. $ ls -al t.dat -rw-r--r-- 1 pwatson mkgroup-l-d 64 Oct 28 10:13 t.dat $ python Python 2.4.1 (#1, May 27 2005, 18:02:40) [GCC 3.3.3 (cygwin special)] on cygwin Type "help", "copyright", "credits" or "license" for more information. >>> import mmap >>> fp = open('t.dat', 'rb') >>> b = mmap.mmap(fp.fileno(), 0, access=mmap.ACCESS_READ) Traceback (most recent call last): File "", line 1, in ? EnvironmentError: [Errno 22] Invalid argument >>> b = mmap.mmap(fp.fileno(), 64, access=mmap.ACCESS_READ) >>> b.size() 64 ===== $ ls -al .profile -rwxrwxr-x 1 pwatson pwatson 1920 Jul 22 06:57 .profile $ python Python 2.4.1 (#1, Jul 19 2005, 14:16:43) [GCC 4.0.0 20050519 (Red Hat 4.0.0-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import mmap >>> fp = open('.profile', 'rb') >>> b = mmap.mmap(fp.fileno(), 0, access=mmap.ACCESS_READ) Traceback (most recent call last): File "", line 1, in ? EnvironmentError: [Errno 22] Invalid argument >>> b = mmap.mmap(fp.fileno(), 1920, access=mmap.ACCESS_READ) >>> b.size() 1920 |
|
|
| msg26756 - (view) |
Author: liturgist (liturgist) |
Date: 2005-10-28 21:09 |
| Logged In: YES user_id=197677 It would be helpful to creating cross-platform code for all platforms to have the same requirements. If this is marked as Not-A-Bug, then how about changing it to a documentation bug so that and example could be provided? fp = open(fn, 'rb') b = mmap.mmap(fp.fileno(), os.stat(fp.name).st_size, access=mmap.ACCESS_READ) |
|
|
| msg26757 - (view) |
Author: Josiah Carlson (josiahcarlson) *  |
Date: 2005-11-08 06:17 |
| Logged In: YES user_id=341410 On Windows and Linux, I have been able to map positive, nonzero portions of files for quite some time. The only use-case I can see in using an mmap of size 0 is if the file was just created, and a user wanted to immediately resize the mmap to be larger. This can be worked-around with a simple fp.write('\0'), followed by a mmap call with size 1. Resize as desired/necessary. |
|
|
| msg26758 - (view) |
Author: liturgist (liturgist) |
Date: 2005-11-08 15:16 |
| Logged In: YES user_id=197677 If the file was created by another process or specified as a parameter, the code might not be free to write into it. Asking os.stat() for the file size works on both platforms. I have not tested on any other platform, but I do not see any reason it would not work. fp = open(fn, 'rb') b = mmap.mmap(fp.fileno(), os.stat(fp.name).st_size, access=mmap.ACCESS_READ) Even if the file were just created, surely asking os.stat() for the file size would not have more overhead than writing to the file. Does it? |
|
|
| msg26759 - (view) |
Author: Josiah Carlson (josiahcarlson) * |
Date: 2005-11-08 18:11 |
| Logged In: YES user_id=341410 According to the documentation, in Windows, passing a 'length' of 0 will map the entire file. On 'Unix', it does not discuss what happens when you use a length of 0. The behavior on undefined arguments in the case of linux could almost be expected (though perhaps should be clarified in documentation), and being that the 'cygwin' platform isn't Windows (it's a unix-like environment in Windows), expecting it to perform like Windows, is probably Wasn't your bug report that you could not map 0 bytes from the file, or that passing '0' did not do what you expected on Linux and/or cygwin? If so, then maybe the documentation should be updated. To respond to "wouldn't os.stat(filename).st_size be good to pass during mmap creation time?" Of course, I do that myself, and I would expect most sane people to do the same. os.fstat(fp.fileno()).st_size also works. So, what is your bug report again, and why is this bug report open? |
|
|
| msg26760 - (view) |
Author: liturgist (liturgist) |
Date: 2005-11-08 19:15 |
| Logged In: YES user_id=197677 The bug report is still right here in this listing. What are the possible solutions? 1) Not have the Windows version map the entire file length. Could break existing code, so not likely. 2) Have the UNIX version map the entire file when a length of 0 is passed. Since the fp is passed, the code can get length from os.fstat(fp.fileno()).st_size. There could be some code handling this exception whose operation would change if this modification were made. 3) Modify the documentation to be more explicit about valid and invalid length values on the UNIX version. Providing and example using fp.fstat(fp.fileno()).st_size would be very helpful. I see the third option as the only viable path at this point. Letting the Windows path use zero as a magic indicator is an impediment to writing cross-platform code. An example in the documentation of how it could be done is sorely needed. |
|
|
| msg26761 - (view) |
Author: Josiah Carlson (josiahcarlson) * |
Date: 2005-11-08 19:37 |
| Logged In: YES user_id=341410 You could just pass fp.fstat(fp.fileno()).st_size to all of your instances, not just the unix ones. Right now, Python exposes the underlying mmap implementation. Windows automatically handles the statting/etc., for using the full size of the file in an mmap. Other platforms don't. While it would seem reasonable to have the unix specific implementation automatically perform an os.fstat(), actually calling the requisite code in os from mmapmodule.c is a pain in the ass (in my experience), and duplicating the code in mmapmodule.c is a poor idea. I would suggest you offer a documentation patch in the patch tracker to state the behavior of Unix mmap and suggested cross-platform workarounds. |
|
|
| msg26762 - (view) |
Author: Georg Brandl (georg.brandl) *  |
Date: 2005-12-18 10:57 |
| Logged In: YES user_id=1188172 This was addressed with patch #1144555, which has been applied to the 2.5 branch only (as it is a new feature). Closing as "Out of Date". |
|
|