Issue 10939: imaplib: Internaldate2tuple raises KeyError parsing month and does not work with negative TZ offset due to bytes/str issues (original) (raw)

Created on 2011-01-18 22:31 by lavajoe, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
imaplib_Internaldate2tuple_bytes_fixes_python32.patch lavajoe,2011-01-28 22:22
issue10939.diff belopolsky,2011-01-29 16:35
Messages (12)
msg126499 - (view) Author: Joe Peterson (lavajoe) Date: 2011-01-18 22:31
There are two issues with conversion to Python 3: 1. It raise "KeyError". This is because the Mon2num dictionary keys are strings (str), not bytes objects (note that many other strings in imaplib have been updated, but not Mon2num). 2. The sign character of the TZ offset (e.g. -0700) is compared to the string (str) '-', not bytes array b'-', so the compare is never true, causing a large error when the TZ offset is negative. Patch attached that also adds a unit test.
msg126572 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-19 22:35
Georg, This is an obvious artifact of an untested py3k port. Can this go in 3.2? The patch looks good except for a long line in the test which I can take care of.
msg127393 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-01-29 09:50
The new test is faulty; it appears to be specific to the timezone of the patch submitter. The library fix should go in nevertheless, if you could add a correct test, Alexander, it would be great.
msg127437 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-29 16:33
Attached patch fixes the test (hopefully we don't have to support systems with non-POSIX epoch) and cleans up whitespace.
msg127444 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-29 17:19
Committed in revision 88231.
msg127462 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-29 18:45
Merged to 3.1 in r88233.
msg127470 - (view) Author: Joe Peterson (lavajoe) Date: 2011-01-29 19:16
Good catch on the test. Note that for issue 10941, we'll want a new non-timezone-dependent test case that can catch the DST-related problem. I'll post a new patch to issue 10941 now and describe this some more there.
msg127471 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-29 19:20
It looks like my test is not robust enough: http://www.python.org/dev/buildbot/all/builders/x86%20OpenIndiana%203.1/builds/298
msg127485 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-29 20:37
Some systems don't like times too close to epoch. Fixed in revision 88239.
msg127489 - (view) Author: Joe Peterson (lavajoe) Date: 2011-01-29 21:38
Alexander, looks like you hit a weirdness in the Europe/London timezone for dates before 31-Oct-1971, as if DST was in effect even in winter. And the fail of the test is caused by the same bug that causes issue 10941: the use of mktime to interpret the values from an internal time string (which should not be dealing with timezones, but rather explicit offsets from UTC, before converting to localtime).
msg127494 - (view) Author: Joe Peterson (lavajoe) Date: 2011-01-29 22:47
More info on the Europe/London "near the epoch" thing (there is no weirdness in the tz stuff - it's just issue 10941 causing the test to fail)... I looked at the sources for the tz data, and here is the definition for Europe/London: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Europe/London -0:01:15 - LMT 1847 Dec 1 0:00s 0:00 GB-Eire %s 1968 Oct 27 1:00 - BST 1971 Oct 31 2:00u 0:00 GB-Eire %s 1996 0:00 EU GMT/BST I appears that London was always 1 hour ahead (BST time) from 1968 Oct 27 until 1971 Oct 31 2:00u, thus triggering issue 10941: Internaldate2tuple() actually returns the wrong local time (00:00 rather than [the correct] 01:00) in its tuple for the epoch, so mktime(), doing the right thing, returns -3600. The patch in issue 10941 fixes this, so the right local time in London (01:00) is returned for the epoch (internal date string "01-Jan-1970 00:00:00 +0000"). Note that this also exposes another problem with Time2Internaldate(), since it uses time.timezone/time.altzone, which are only valid for the current rules, not old rules as in the London case near the epoch.
msg127496 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-29 23:07
On Sat, Jan 29, 2011 at 5:48 PM, Joe Peterson <report@bugs.python.org> wrote: .. > Note that this also exposes another problem with Time2Internaldate(), since it uses > time.timezone/time.altzone, which are only valid for the current rules, not old rules as in the London case > near the epoch. This is a known issue. Hopefully this example will raise more interest for issue 9527. See also issue 1647654 which suggests a python-only work-around.
History
Date User Action Args
2022-04-11 14:57:11 admin set github: 55148
2011-01-29 23:07:11 belopolsky set messages: +
2011-01-29 22:47:59 lavajoe set messages: +
2011-01-29 21:38:06 lavajoe set messages: +
2011-01-29 20:43:16 belopolsky set status: open -> closed
2011-01-29 20:37:09 belopolsky set messages: +
2011-01-29 19:20:42 belopolsky set messages: +
2011-01-29 19:16:42 lavajoe set messages: +
2011-01-29 18:45:09 belopolsky set versions: + Python 3.1nosy: + benjamin.petersonmessages: + resolution: accepted -> fixedstage: commit review -> resolved
2011-01-29 17:19:36 belopolsky set nosy:georg.brandl, belopolsky, lavajoemessages: +
2011-01-29 16:35:35 belopolsky set files: + issue10939.diffnosy:georg.brandl, belopolsky, lavajoe
2011-01-29 16:34:35 belopolsky set files: - issue10939.diffnosy:georg.brandl, belopolsky, lavajoe
2011-01-29 16:33:55 belopolsky set files: + issue10939.diffnosy:georg.brandl, belopolsky, lavajoemessages: + resolution: acceptedstage: patch review -> commit review
2011-01-29 09:50:20 georg.brandl set priority: normal -> release blockernosy:georg.brandl, belopolsky, lavajoemessages: +
2011-01-28 23:53:20 belopolsky set assignee: belopolsky -> georg.brandlnosy:georg.brandl, belopolsky, lavajoe
2011-01-28 22:22:39 lavajoe set files: + imaplib_Internaldate2tuple_bytes_fixes_python32.patchnosy:georg.brandl, belopolsky, lavajoe
2011-01-28 22:20:11 lavajoe set files: - imaplib_Internaldate2tuple_bytes_fixes_python32.patchnosy:georg.brandl, belopolsky, lavajoe
2011-01-19 22:35:08 belopolsky set nosy: + georg.brandl, belopolskymessages: + assignee: belopolskystage: patch review
2011-01-18 22:37:58 lavajoe set title: imaplib: Internaldate2tuple fails to parse month and does not work with negative TZ offset due to bytes/str issues -> imaplib: Internaldate2tuple raises KeyError parsing month and does not work with negative TZ offset due to bytes/str issues
2011-01-18 22:31:34 lavajoe create