Issue 3611: invalid exception context (original) (raw)
Created on 2008-08-20 02:33 by vstinner, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Messages (41)
Author: STINNER Victor (vstinner) * 
Date: 2008-08-20 02:33
After few seconds (30 sec to 5 minutes), my program (Fusil) crashs at:
PyEval_EvalFrameEx (f=0x85b4324, throwflag=0) at Python/ceval.c:2459 Py_CLEAR(tstate->exc_traceback);
It crashs because tstate->exc_traceback points to a zombi object: {_ob_next = 0xdbdbdbdb, _ob_prev = 0xdbdbdbdb, ob_refcnt = -606348326, ob_type = 0xdbdbdbdb}
(refcnt is 0xdbdbdbdb-1)
I'm using py3k rev 65882 compiled with CFLAGS "-O0 -ggdb" and --with-pydebug. Sorry, I don't have more informations yet and I can't explain how to reproduce the bug :-/
Author: STINNER Victor (vstinner) *
Date: 2008-08-20 02:38
The crash is on ProcError exception raising (line 21):
20 except IOError as err:
21 raise ProcError("Unable to open %r: %s" % (filename, (gdb) where #0 0x0805bcc7 in PyObject_Hash (v=0x8691034) at Objects/object.c:796 #1 0x08062125 in set_contains_key (so=0x856ec6c, key=0x8691034) at Objects/setobject.c:682 #2 0x0806659f in PySet_Contains (anyset=0x856ec6c, key=0x8691034) at Objects/setobject.c:2263 #3 0x080def3a in print_exception_recursive (f=0xb7bea654, value=0x8656814, seen=0x856ec6c) at Python/pythonrun.c:1432 #4 0x080df0ea in PyErr_Display (exception=0x84542ac, value=0x8656814, tb=0x865bc34) at Python/pythonrun.c:1470 #5 0x080e68f6 in sys_excepthook (self=0xb7e013b4, args=0x865b334) at Python/sysmodule.c:119 #6 0x08161d29 in PyCFunction_Call (func=0xb7e016c4, arg=0x865b334, kw=0x0) at Objects/methodobject.c:81 #7 0x08118cc5 in PyObject_Call (func=0xb7e016c4, arg=0x865b334, kw=0x0) at Objects/abstract.c:2181 #8 0x080b2ea8 in PyEval_CallObjectWithKeywords (func=0xb7e016c4, arg=0x865b334, kw=0x0) at Python/ceval.c:3283 #9 0x080de463 in PyErr_PrintEx (set_sys_last_vars=1) at Python/pythonrun.c:1258 #10 0x080de06f in PyErr_Print () at Python/pythonrun.c:1150 #11 0x080e0007 in Py_FatalError (msg=0xbf9a1498 "Python/ceval.c:2459 object at 0x865b6b4 has negative ref count -606348326") at Python/pythonrun.c:1863 #12 0x0805a821 in _Py_NegativeRefcount (fname=0x8197098 "Python/ceval.c", lineno=2459, op=0x865b6b4) at Objects/object.c:194 #13 0x080b004f in PyEval_EvalFrameEx (f=0x85b4324, throwflag=0) at Python/ceval.c:2459 (...)
(gdb) frame 13
#13 0x080b004f in PyEval_EvalFrameEx (f=0x85c9794, throwflag=0) at
Python/ceval.c:2459
2459
Py_CLEAR(tstate->exc_traceback);
(gdb) pystack /home/haypo/ptrace/ptrace/linux_proc.py (21): openProc /home/haypo/ptrace/ptrace/linux_proc.py (24): readProc /home/haypo/ptrace/ptrace/linux_proc.py (31): readProcessProc /home/haypo/ptrace/ptrace/linux_proc.py (74): readProcessStat (...)
linux_proc.py code near line 21: (...) 7 class ProcError(Exception): 8 pass (...) 16 def openProc(path): 17 try: 18 filename = procFilename(path) 19 return open(filename) 20 except IOError as err: 21 raise ProcError("Unable to open %r: %s" % (filename, err)) (...) 29 def readProcessProc(pid, key): 30 try: 31 return readProc(path_join(str(pid), str(key))) 32 except ProcError as error: 33 raise ProcError("Process %s doesn't exist: %s" % ( 34 pid, error)) (...) 73 def readProcessStat(pid): 74 stat = readProcessProc(pid, 'stat') 75 return ProcessState(stat) 76
Author: STINNER Victor (vstinner) *
Date: 2008-08-20 02:45
I also noticed a crash in PyErr_SetObject(): in block <line 68..86>, tstate->exc_value value may changes and so it's possible that tstate->exc_value becomes NULL. I added a test to avoid this crash:
Index: Python/errors.c
--- Python/errors.c (révision 65899) +++ Python/errors.c (copie de travail) @@ -88,7 +88,7 @@ This is O(chain length) but context chains are usually very short. Sensitive readers may try to inline the call to PyException_GetContext. */
if (tstate->exc_value != value) {
if (tstate->exc_value != value && tstate->exc_value !=
NULL) { PyObject *o = tstate->exc_value, *context; while ((context = PyException_GetContext(o))) { Py_DECREF(context);
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-20 09:02
I also noticed a crash in PyErr_SetObject(): in block <line 68..86>, tstate->exc_value value may changes and so it's possible that tstate->exc_value becomes NULL.
How can tstate->exc_value become NULL at that point? Could you investigate?
Author: STINNER Victor (vstinner) *
Date: 2008-08-20 14:58
How can tstate->exc_value become NULL at that point? [error.c:86] Could you investigate?
PyEval_CallObject(exception, args) may calls PyErr_SetObject(). Since the same thread state is the same, tstate->exc_value also changes during this call.
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-20 15:24
Selon STINNER Victor <report@bugs.python.org>:
How can tstate->exc_value become NULL at that point? [error.c:86] Could you investigate?
PyEval_CallObject(exception, args) may calls PyErr_SetObject(). Since the same thread state is the same, tstate->exc_value also changes during this call.
Hmm, indeed. However, I don't see why your Python code triggers that. Does instantiating the ProcError object somehow fail, and for what reason? Or did you witness it in other circumstances?
As for the original problem ("tstate->exc_traceback points to a zombi object") it would be nice to have a small snippet of Python code to reproduce it. Is it possible for you to do that?
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-20 15:44
I retract what I said above. PyErr_SetObject() and friends only change tstate->curexc_* variables, not tstate->exc_*.
The former (tstate->curexc_*) contain the current, pending (uncaught) exception state of the thread. The latter (tstate->exc_*) contain the exception currently handled by an "except" statement in Python code.
So, theoretically, there is no way calling PyEval_CallObject() can change tstate->exc_value:
- if it raises an exception or calls code that raises an exception without catching it, the exception ends up in tstate->curexc_*, not in tstate->exc_*
- if it calls code that raises an exception /and catches it/, exception stacking means the exception should be discarded after the end of the "except" block and the last tstate->exc_* values restored.
Of course, there may be bugs in the implementation of the above. The best way to find out would be to have a small snippet of Python code reproducing the problem. :-)
Author: STINNER Victor (vstinner) *
Date: 2008-08-20 16:01
About the PyEval_CallObject() call in errors.c, here is an example: Call exception<0x81dcee0>(args<0x8751dc4>) with exception= object : <class 'AttributeError'> type : type refcount: 6 address : 0x81dcee0 and args= object : ("type object 'ProcError' has no attribute 'subclasscheck'",) type : tuple refcount: 1 address : 0x8751dc4
This exception may comes from PyObject_IsSubclass() which calls PyObject_GetAttr(cls, "subclasscheck").
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-20 16:49
[...]
This exception may comes from PyObject_IsSubclass() which calls PyObject_GetAttr(cls, "subclasscheck").
If you look at the code for PyObject_IsSubclass(), there is a pair of PyErr_Fetch / PyErr_Restore calls around PyObject_GetAttr(cls, "subclasscheck"). So this exception should vanish immediately rather than stay around.
Author: STINNER Victor (vstinner) *
Date: 2008-08-20 17:10
I'm unable to write a short Python script to reproduce the bug. So download the full program: http://neudorf.hachoir.org/tmp/fusil3000.tar.gz
Run in with: $ gdb python3.0 (gdb) run fusil-python --fast
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-20 19:50
The problem comes when PyErr_SetObject triggers garbage collection which runs python code (finalizers...).
I could reproduce the crash several times, and each time garbage collection was triggered by the normalization of the exception (the call to PyEval_CallObject(exception, args)).
An obvious fix is to save exc_value near the "/* Implicit exception chaining */" comment.
However, I fear that there are other places where exc_value is reset, and "exception chaining" may break in subtle ways.
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-20 20:03
The problem comes when PyErr_SetObject triggers garbage collection which runs python code (finalizers...).
Mmmh, normally this shouldn't change the value of tstate->exc_value once that Python code returns. That's what exception stacking is for.
Having a snippet deterministically reproducing the problem would really help in any case.
An obvious fix is to save exc_value near the "/* Implicit exception chaining */" comment.
Well, it may be a fix for the crash but I'm not sure it makes the semantics correct. If tstate->exc_value was really changed, there is a reason and I'm not sure it should be ignored.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-20 21:21
Having a snippet deterministically reproducing the problem would really help in any case. Here you are. Don't ask how I found this.
The attached script, when run, prints
(KeyError(), ValueError()) (KeyError(), None)
The current exception context (tstate->exc_value) is lost when a generator is deleted, if it was paused inside a "try" block.
This is the cause of the crash: the gc runs inside PyErr_SetObject and collects such a generator.
Author: STINNER Victor (vstinner) *
Date: 2008-08-20 21:21
It looks that the problem is that PyErr_SetObject() is not re-entrant. The crash occurs when PyErr_SetObject() is called (indirectly) by PyErr_SetObject(): tstate->exc_value value is changed (set to NULL).
As noticed by amaury.forgeotdarc, the problem is linked to be garbage collector: the crash occurs when the garbage collector calls a destructor which will reuse PyErr_SetObject().
Author: STINNER Victor (vstinner) *
Date: 2008-08-20 21:58
Great job amaury! So in ceval, here is the block responsible to clear the execution informations:
Index: Python/ceval.c
--- Python/ceval.c (révision 65915) +++ Python/ceval.c (copie de travail) @@ -2453,11 +2453,6 @@
if (b->b_type == EXCEPT_HANDLER) {
UNWIND_EXCEPT_HANDLER(b);if (why == WHY_EXCEPTION) {Py_CLEAR(tstate->exc_type);Py_CLEAR(tstate->exc_value);- Py_CLEAR(tstate->exc_traceback);
} continue; } UNWIND_BLOCK(b);
Without these 5 lines, the bug disappears and it looks (i tried few tests) that CPython is still ok.
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2008-08-20 22:54
Unfortunately, removing those lines causes a RuntimeError about exceeding the recursion limit for about 1/3 of the tests.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-20 22:59
Can you try with the following patch, "throwflag" is specific to a generator being deleted.
Index: Python/ceval.c
--- Python/ceval.c (revision 65919) +++ Python/ceval.c (working copy) @@ -2453,7 +2453,7 @@
if (b->b_type == EXCEPT_HANDLER) {
UNWIND_EXCEPT_HANDLER(b);if (why == WHY_EXCEPTION) {
if (why == WHY_EXCEPTION && !throwflag) { Py_CLEAR(tstate->exc_type); Py_CLEAR(tstate->exc_value); Py_CLEAR(tstate->exc_traceback);
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-20 23:09
And if it doesn't work, try the following instead:
if (why == WHY_EXCEPTION) { PyObject *tmp1, *tmp2, *tmp3; tmp1 = tstate->exc_type; tmp2 = tstate->exc_value; tmp3 = tstate->exc_traceback; tstate->exc_type = NULL; tstate->exc_value = NULL; tstate->exc_traceback = NULL; Py_XDECREF(tmp1); Py_XDECREF(tmp2); Py_XDECREF(tmp3); }
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-20 23:13
This last proposal does not correct the behaviour of lostcontext.py.
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2008-08-20 23:19
Here's Amaury's patch and a unittest based on lostcontext.py.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-20 23:24
Short patch review: context tests are mostly found in test_raise.py (class TestContext)
test_exceptions.py seems to only deal with the exception object.
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2008-08-20 23:24
Patch was applied in r65921.
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2008-08-20 23:25
Ok. I'll move the test after the betas.
Author: STINNER Victor (vstinner) *
Date: 2008-08-21 16:26
The bug is not closed :-/ With py3k trunk, I still get a crash. So I added a flag to detect inner calls. Here is an example of inner call backtrace:
(gdb) where (...) #2 0xb7df4201 in abort () from /lib/tls/i686/cmov/libc.so.6 #3 0x080a23a9 in PyErr_SetObject (exception=0x818b760, value=0xb75a3694) at Python/errors.c:61 #4 0x08094048 in do_raise (exc=0xb75a3694, cause=0x0) at Python/ceval.c:2928 #5 0x0808fa70 in PyEval_EvalFrameEx (f=0x83557d4, throwflag=0) at Python/ceval.c:1510 (...) #23 0x08094bd8 in PyEval_CallObjectWithKeywords (func=0xb7a6c96c, arg=0xb7c6802c, kw=0x0) at Python/ceval.c:3283 #24 0x0806befd in slot_tp_del (self=0xb759d44c) at Objects/typeobject.c:5268 (...) #26 0x0811f05d in PyDict_Clear (op=0xb7595e84) at Objects/dictobject.c:846 #27 0x08121112 in dict_tp_clear (op=0xb7595e84) at Objects/dictobject.c:1841 #28 0x080c81e9 in delete_garbage (collectable=0xbfcc75c4, old=0x8179434) at Modules/gcmodule.c:684 (...) #31 0x080c9000 in _PyObject_GC_Malloc (basicsize=28) at Modules/gcmodule.c:1333 #32 0x08060aee in PyType_GenericAlloc (type=0x818b0a0, nitems=0) at Objects/typeobject.c:667 (...) #37 0x080a24cc in PyErr_SetObject (exception=0x818b0a0, value=0xb75acea0) at Python/errors.c:87 (...) #44 0x080912f7 in PyEval_EvalFrameEx (f=0x83660e4, throwflag=0) at Python/ceval.c:1940 (...)
First Python stack:
(gdb) pystack /home/haypo/prog/py3k/Lib/io.py (1074): _flush_unlocked /home/haypo/prog/py3k/Lib/io.py (1070): flush /home/haypo/prog/py3k/Lib/io.py (1454): flush /home/haypo/prog/py3k/Lib/io.py (1459): close /home/haypo/prog/py3k/Lib/io.py (390): del /home/haypo/fusil3000/fusil/process/cpu_probe.py (30): live /home/haypo/fusil3000/fusil/mas/univers.py (15): executeAgent (...)
Second Python stack:
(gdb) pystack /home/haypo/fusil3000/fusil/process/cpu_probe.py (30): live /home/haypo/fusil3000/fusil/mas/univers.py (15): executeAgent /home/haypo/fusil3000/fusil/mas/univers.py (24): execute /home/haypo/fusil3000/fusil/application.py (196): executeProject (...)
So the real fix is to make PyErr_SetObject() re-entrant.
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-21 17:14
On a Windows box, I manage to make the following script reliably hang on a non-debug build of beta3. This can be a good base for further diagnosing.
def f(): class Bug: def del(self): 1/0
import gc
trash = [Bug()]
trash.append(trash)
try:
gc.collect()
gc.set_threshold(1, 1, 1)
del trash
len()
except TypeError:
raiseif name == "main": f()
Author: STINNER Victor (vstinner) *
Date: 2008-08-21 17:16
Here is a new snippet with strange exception handling: --------------------- 8< ------------------------- from gc import collect import _weakref
class FuzzingUserClass: pass
obj = _weakref.ref(FuzzingUserClass)
Exception not raised??
obj.init( 0, 0, 0, )
Exception catched here??
collect() --------------------- 8< -------------------------
Result: Exception TypeError: 'init expected at most 2 arguments, got 3' in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Abandon (core dumped)
The exception is raised in Objects/weakrefobject.c: weakref___init__() => parse_weakref_init_args() => PyArg_UnpackTuple() here
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-21 17:19
Haypo, this is a separate bug I think. Please open a new ticket :)
Author: STINNER Victor (vstinner) *
Date: 2008-08-21 17:25
@pitrou: Ok, done (issue #3634). We were right, it's a different bug.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-21 17:43
Antoine, your script hangs at the end due to the io.py deadlock (see #3618)
- at the end of the script, flush_io() is called
- this enters code in io.py
- here, garbage collection occurs (thanks to low thresholds)
- the Bug() instance is collected
- the exception is handled by a PyErr_WriteUnraisable
- which tries to print
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-21 18:18
I found another way to loose the current exception context; see lostcontext2.py
Completely removing the block starting with if (why == WHY_EXCEPTION && !throwflag) { corrects the script;
but according to Benjamin: """removing those lines causes a RuntimeError about exceeding the recursion limit for about 1/3 of the tests."""
So a better fix must be found. At least we have the unit test :-).
Author: STINNER Victor (vstinner) *
Date: 2008-08-21 23:26
It's now known that PyErr_SetObject() have to be re-entrant because of the garbage collector interaction. As I wrote in my comments, tstate may be changed during PyEval_CallObject() call. The problem is to known which values have to be protected during PyErr_SetObject() re-entrant call... I tried to save/restore tstate->exc_value if it's value changes, but I'm not sure that it's enough.
Author: STINNER Victor (vstinner) *
Date: 2008-08-21 23:54
Ok, it was not enough: exc_{type,value,traceback} have to be saved/restored. So here is a new patch. I doesn't change Python behaviour for previous pitrou snippet (): it doesn't crash here and display two errors.
I think that this issue mix differents bugs: (1) lostcontext.py fixed by « if (why == WHY_EXCEPTION && !throwflag) » (2) pitrou snippet (3) PyErr_SetObject() is no re-entrant: not fixed yet (please review my new patch attached to this comment ;-))
Author: STINNER Victor (vstinner) *
Date: 2008-08-22 00:53
With my last patch (pyerr_setobject_reentrant-v2.patch) my program works fine! I known that PyErr_SetObject() makes re-entrant calls every ~30 seconds, so the code is tested ;-)
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-22 10:07
Victor, your patch addresses the symptom of the problem, not the cause.
The cause is that in some cases, the exception context chain is lost. Of course what could be a minor annoyance becomes dramatic when this precisely happens in code that makes use of this context.
IMO, PyErr_SetObject should not restore the previous exc_* members; it's not its job to correct others' mistakes. I join a patch that only protects it from crashing in this case: it just saves exc_value in a local variable.
The bug shown by lostcontext2.py is still unresolved, but at least it won't crash the interpreter.
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-22 10:22
Agreed with Amaury, it's not PyErr_SetObject's job to try to save/restore the tstate->exc_* variables. We'll probably have to live with the small context-losing glitches in 3.0.
For 3.1, a radical solution would be to drop the "exception normalization" misfeature (which is a source of complications and potential bugs) and always instantiate exception objects as soon as they are raised rather than lazily. We need to make sure the performance loss is reasonable though.
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-22 10:31
Hmm, answering to myself, I don't think dropping exception normalization would solve the problem, it would just let it occur in a different place...
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-22 23:24
Benjamin,
I am testing the patch proposed earlier by Victor in http://bugs.python.org/msg71579 (who might be the correct one, after all), and I don't see the problems you mentioned. I run the test suite without problem, on winXP & Linux Debian x86.
Can you please try again?
Author: Benjamin Peterson (benjamin.peterson) *
Date: 2008-08-22 23:29
I can't reproduce it either. I suspect that I removed the wrong lines initially and caused other problems. :) Sorry for the false alarm.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-27 20:52
Some progress: The lines suppressed by the patch at http://bugs.python.org/msg71579 either do nothing (because e.g exc_type is already NULL or None), or happen to be in a case similar to the script "lostcontext2.py" (Most of the time, the flush() function in io.py).
So again, I think these lines should be suppressed. The test suite still pass, and all the "-R::" I ran did return identical result. Can this patch go in?
Author: Antoine Pitrou (pitrou) *
Date: 2008-08-27 21:36
Can this patch go in?
I'm ok for it.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) *
Date: 2008-08-29 07:14
Committed as r66056.
History
Date
User
Action
Args
2022-04-11 14:56:37
admin
set
github: 47861
2008-08-29 07:14:24
amaury.forgeotdarc
set
status: open -> closed
resolution: fixed
messages: +
2008-08-27 21:36:16
pitrou
set
messages: +
2008-08-27 20:52:37
amaury.forgeotdarc
set
messages: +
2008-08-22 23:29:11
benjamin.peterson
set
messages: +
2008-08-22 23:24:14
amaury.forgeotdarc
set
messages: +
2008-08-22 10:31:47
pitrou
set
messages: +
2008-08-22 10:22:09
pitrou
set
messages: +
2008-08-22 10:07:55
amaury.forgeotdarc
set
files: + pyerr_seterror_protect.py
messages: +
2008-08-22 00:57:00
benjamin.peterson
set
keywords: + needs review
2008-08-22 00:53:13
vstinner
set
messages: +
2008-08-21 23:54:22
vstinner
set
files: + pyerr_setobject_reentrant-v2.patch
messages: +
2008-08-21 23:26:20
vstinner
set
files: + pyerr_setobject_reentrant.patch
messages: +
2008-08-21 18🔞12
amaury.forgeotdarc
set
files: + lostcontext2.py
messages: +
2008-08-21 17:43:47
amaury.forgeotdarc
set
messages: +
2008-08-21 17:25:50
vstinner
set
messages: +
2008-08-21 17:19:32
pitrou
set
messages: +
2008-08-21 17:16:29
vstinner
set
messages: +
2008-08-21 17:14:31
pitrou
set
messages: +
2008-08-21 16:28:28
benjamin.peterson
set
status: closed -> open
resolution: accepted -> (no value)
2008-08-21 16:26:59
vstinner
set
messages: +
2008-08-21 00:58:37
barry
set
status: open -> closed
2008-08-20 23:25:46
benjamin.peterson
set
messages: +
2008-08-20 23:24:52
benjamin.peterson
set
messages: +
2008-08-20 23:24:45
amaury.forgeotdarc
set
messages: +
2008-08-20 23:21:53
barry
set
resolution: accepted
2008-08-20 23:19:56
benjamin.peterson
set
files: + 3611.patch
keywords: + patch
messages: +
2008-08-20 23:13:10
amaury.forgeotdarc
set
messages: +
2008-08-20 23:09:08
pitrou
set
messages: +
2008-08-20 22:59:49
amaury.forgeotdarc
set
messages: +
2008-08-20 22:54:38
benjamin.peterson
set
nosy: + benjamin.peterson
messages: +
2008-08-20 22:25:23
barry
set
priority: release blocker
2008-08-20 21:58:03
vstinner
set
messages: +
2008-08-20 21:21:30
vstinner
set
messages: +
2008-08-20 21:21:25
amaury.forgeotdarc
set
files: + lostcontext.py
messages: +
2008-08-20 20:03:59
pitrou
set
messages: +
2008-08-20 19:50:55
amaury.forgeotdarc
set
nosy: + amaury.forgeotdarc
messages: +
2008-08-20 17:10:03
vstinner
set
messages: +
2008-08-20 16:49:36
pitrou
set
messages: +
2008-08-20 16:01:12
vstinner
set
messages: +
2008-08-20 15:44:53
pitrou
set
messages: +
2008-08-20 15:24:24
pitrou
set
messages: +
2008-08-20 14:58:19
vstinner
set
messages: +
2008-08-20 09:02:42
pitrou
set
nosy: + pitrou
messages: +
2008-08-20 02:45:51
vstinner
set
messages: +
2008-08-20 02:38:20
vstinner
set
messages: +
2008-08-20 02:33:18
vstinner
create