| msg128753 - (view) |
Author: Guy Kisel (Guy.Kisel) |
Date: 2011-02-17 20:20 |
| Tested in Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)] on win32 Exception thrown: OverflowError: modification time overflows a 4 byte field Steps to reproduce: 1. Set system time to 2/8/2106 or later and modify a .py file (or use a utility to change the date modified directly). 2. Try to run or import the .py file. This date is 2^32 seconds after the Unix epoch. |
|
|
| msg128758 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2011-02-17 21:38 |
| The problem occurs on import (import bla reads bla.py), when Python tries to create bla.pyc. The problem is that Python stores the timestamp as 4 bytes in .pyc files, whereas time_t is 64 bits on Windows (at least on Windows XP with Visual Studio). To support bigger timestamps, we have to change the file format of .pyc files. It cannot be done in Python 2.7, I propose to do it in Python 3.3 See also #5537 and #4379: other issues with 64 bits => 32 bits timestamps. |
|
|
| msg128761 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2011-02-17 22:08 |
| > To support bigger timestamps, we have to change the file format > of .pyc files. write_compiled_module(), check_compiled_module() and other functions use the marshal module to read/write binary file, but marshal has no function for int64_t, only for long (which may be 32 bits, especially on Windows). I don't know if Python has a builtin 64 bits integer type. There is PY_LONG_LONG, but this type is optional. A possible solution is to always store timestamp as 64 bits signed integer, but reject timestamp > 2^32 (as currently) if we don't have 64 bits integer type (like PY_LONG_LONG). Something like: #ifdef PY_LONG_LONG write_timestamp64(t); #else if (t << 32) { error; } write_timestamp32(t); write_long32(0); /* emulate 64 bits in big endian */ #endif |
|
|
| msg131594 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2011-03-21 01:17 |
| By the way, a simpler fix would be just to not fail if the .pyc file cannot be created (but emit a warning / write an error to stderr). |
|
|
| msg151875 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-01-24 01:35 |
| Fixing this is much easier than Victor's suggestion, we just have to ignore the higher bits of the timestamp (that is, store it modulo 2**32). This is enough for the purposes of testing the freshness of a pyc file. And by avoiding modifying the pyc format, we can also ship the fix in bugfix releases. |
|
|
| msg151887 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-01-24 09:19 |
| Here is a patch for 3.2. importlib doesn't have the issue, so I just added a test. |
|
|
| msg151914 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2012-01-24 16:10 |
| LGTM |
|
|
| msg151915 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2012-01-24 16:54 |
| New changeset a2f3d6986bfa by Antoine Pitrou in branch '3.2': Issue #11235: Fix OverflowError when trying to import a source file whose modification time doesn't fit in a 32-bit timestamp. http://hg.python.org/cpython/rev/a2f3d6986bfa New changeset cb13d8cff050 by Antoine Pitrou in branch 'default': Issue #11235: Fix OverflowError when trying to import a source file whose modification time doesn't fit in a 32-bit timestamp. http://hg.python.org/cpython/rev/cb13d8cff050 New changeset 0499eed74126 by Antoine Pitrou in branch '2.7': Issue #11235: Fix OverflowError when trying to import a source file whose modification time doesn't fit in a 32-bit timestamp. http://hg.python.org/cpython/rev/0499eed74126 |
|
|
| msg151916 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2012-01-24 16:57 |
| Committed, thanks. |
|
|