| msg19256 - (view) |
Author: Randy Wiser (randyw) |
Date: 2003-12-03 19:16 |
| Calling socket.recv(size) with size starting at 3000000 and decreasing by approximately 1024 each time, quickly raises a MemoryError exception, although the process memory use does not exceed 30 Mbytes! Verified on Python 2.1, 2.3 and 2.3.2 (under Windows XP Professional ONLY version 5.1.2600 hotfix KB824146 if it matters :-) Cannot reproduce (test runs many iterations without fail) under Windows 2000 Professional or Server. To reproduce on XP, run attached files is separate shell windows: c:\> python hostMemErrSrv.py waiting for client on 3021 Connected by ('127.0.0.1', 1217) . . . . . . . . . . . . . . . . c:\> python hostMemErr.py MemoryError, hit return to exit. Traceback (most recent call last): File "hostMemErr.py", line 117, in ? main() File "hostMemErr.py", line 108, in main data = p.dataRecv(sz) File "hostMemErr.py", line 63, in dataRecv tmp = self.dataSocket.recv (nleft) File "", line 1, in recv MemoryError |
|
|
| msg19257 - (view) |
Author: Tim Peters (tim.peters) *  |
Date: 2003-12-03 19:28 |
| Logged In: YES user_id=31435 Python doesn't do anything differently across different flavors of Windows, so you're probably looking at a Microsoft bug, or an MS design limitation. Note that doing recv() with an argument in the millions isn't a sane thing to do to begin with. Unassigned from me, and reduced priority (I'm not going to do anything more on this; Randy, try writing equivalent code directly in C; if the same thing happens there, as appears likely, then your complaint should go to Microsoft). |
|
|
| msg19258 - (view) |
Author: Randy Wiser (randyw) |
Date: 2003-12-04 01:16 |
| Logged In: YES user_id=215159 A quick test with Python 2.3 -i shows the following, so I'm guessing if at first WinXP claims no memory, try try again. Thanks for the quick response Tim. We'll try not to ask WinXP for 3 Mil. of memory unless we really need it. python -i hostMemErr.py MemoryError, hit return to exit. Traceback (most recent call last): File "hostMemErr.py", line 117, in ? main() File "hostMemErr.py", line 108, in main data = p.dataRecv(sz) File "hostMemErr.py", line 63, in dataRecv tmp = self.dataSocket.recv (nleft) MemoryError >>> s= 'z' * 3000000 Traceback (most recent call last): File "", line 1, in ? MemoryError >>> s= 'z' * 3000000 >>> s= 'z' * 3000000 >>> s= 'z' * 3000000 >>> s= 'z' * 30000000 >>> ^Z |
|
|
| msg19259 - (view) |
Author: Tim Peters (tim.peters) * |
Date: 2003-12-15 01:47 |
| Logged In: YES user_id=31435 It's not crazy to ask WinXP for 3MB of memory in general, it's simply weird to pass a number that large to .recv() specifically. Sockets are a low-level facility, working directly with system network buffers, whose sizes are measured in KB rather than MB. Since we don't have Microsoft's socket implementation to stare at, we can't be sure, but it's possible that XP's implementation goes insane when fed unreasonable arguments. If so, I expect they'll eventually fix that. Regardless, I don't see anything here that's plausibly Python's doing (other than reporting back that the system malloc() returned NULL), so I'm closing this as 3rd-party, wontfix. |
|
|
| msg114267 - (view) |
Author: geremy condra (debatem1) |
Date: 2010-08-18 19:12 |
| I can sporadically reproduce this on Linux as well, repeatedly passing values larger than 2**20 to recv. Doesn't seem to happen with regularity, but I have better than 11Gb of unused RAM on hand. Is there a good reason for this? |
|
|