Issue 9837: ZipFileExt.read() reads more data than requested (original) (raw)

Issue9837

Created on 2010-09-12 09:28 by Black.Dew, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
a.py Black.Dew,2010-09-12 14:03 simple test
Messages (4)
msg116158 - (view) Author: Black Dew (Black.Dew) Date: 2010-09-12 09:28
ZipFileExt.read() can return more data than requested, unlike file and other file-like objects. This function calls read1() in a loop, passing the original requested size even if part of the data was already read thus reading and returning more than the caller requested. This should be fixed by requesting n-len(buf) or something similar, or at least properly documented.
msg116184 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-12 12:44
Indeed. In most cases, though, read1(n) will return exactly n bytes and the loop will therefore end immediately with the right buf size. Do you have a simple test case?
msg116197 - (view) Author: Black Dew (Black.Dew) Date: 2010-09-12 14:03
If i follow the logic in read1() correctly it will do that only for files with very low compression ratios (the original sample where i noticed that problem was actually a chunk of encrypted data inside the zip). From the comments referring to "with at most one read() system call" I would also assume it's OS-specific. Anyway, I've attached a small file that reproduces this behavior on my system (win xp 32bit). When i run it i get "AssertionError: Read returned 4097 bytes (4096 requested)"
msg116203 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-12 14:58
Ok, thank you. I've fixed the issue and added a test in r84737 (3.x) and r84739 (2.7).
History
Date User Action Args
2022-04-11 14:57:06 admin set github: 54046
2010-09-12 14:58:36 pitrou set status: open -> closedresolution: fixedmessages: +
2010-09-12 14:03:34 Black.Dew set files: + a.pymessages: +
2010-09-12 12:44:39 pitrou set messages: + versions: + Python 3.2
2010-09-12 09:38:39 r.david.murray set nosy: + pitrou
2010-09-12 09:28:13 Black.Dew create