Issue 13183: pdb skips frames after hitting a breakpoint and running step (original) (raw)

Created on 2011-10-14 18:56 by xdegaye, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (24)
msg145557 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2011-10-14 18:56
Pdb skips frames after hitting a breakpoint and running step commands that walk back the frame stack. Run the following two tests with the two files named foo.py and bar.py: === foo.py ============================================ from bar import bar def foo(): bar() def nope(): pass def foobar(): foo() nope() foobar() === bar.py ============================================ def bar(): print('1') === test_1 ============================================ $ python3 --version Python 3.2 $ python3 -m pdb foo.py > /path/to/foo.py(1)() -> from bar import bar (Pdb) from bar import bar (Pdb) break bar Breakpoint 1 at /path/to/bar.py:1 (Pdb) continue > /path/to/bar.py(2)bar() -> print('1') (Pdb) step 1 --Return-- > /path/to/bar.py(2)bar()->None -> print('1') (Pdb) step --Call-- > /path/to/foo.py(6)nope() -> def nope(): (Pdb) === test_2 ============================================ $ python3 -m pdb foo.py > /path/to/foo.py(1)() -> from bar import bar (Pdb) break nope Breakpoint 1 at /path/to/foo.py:6 (Pdb) from bar import bar (Pdb) break bar Breakpoint 2 at /path/to/bar.py:1 (Pdb) continue > /path/to/bar.py(2)bar() -> print('1') (Pdb) step 1 --Return-- > /path/to/bar.py(2)bar()->None -> print('1') (Pdb) step --Return-- > /path/to/foo.py(4)foo()->None -> bar() (Pdb) =========================================================== Note: stop_here, break_anywhere and dispatch_call are methods of the Bdb class. test_1 fails to stop in foo() after the second 'step' command because the trace function is not set for all the frames being created in the foo module, since stop_here() and break_anywhere() are both False whenever dispatch_call() is invoked in this module. So after the second 'step' command, trace_dispatch is not invoked by the interpreter until a new frame is created, which happens when nope() is called. test_2 succeeds and stops in foo() after the second 'step' command. After setting the dummy breakpoint 1 in the foo module in test_2, break_anywhere() becomes True in the foo module and the trace function is set for all the frames created in this module (with an associated performance penalty). The problem exists in all python versions. The attached patch fixes this problem by restoring the trace function on returning from a frame when the command is 'step'. The patch includes a test case.
msg145633 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2011-10-16 19:23
Uploaded restore_trace_2.patch that improves the test case.
msg145634 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2011-10-16 19:24
Uploaded restore_trace.py27.patch with a fix and test case for python 2.7.
msg154889 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2012-03-04 12:13
This message is just a reminder that this 4 months old issue raises the point that the step command in pdb is broken. A patch and test case have been proposed. No comment so far. As the author of pyclewn, a Vim front end to pdb and gdb, I would be grateful for any progress on this problem.
msg154933 - (view)	Author: Senthil Kumaran (orsenthil) *	Date: 2012-03-05 09:19
It's an interesting issue. Thanks for the report and patch, Xavier. I am setting patch to needing review.
msg159706 - (view)	Author: Senthil Kumaran (orsenthil) *	Date: 2012-04-30 16:53
Hello Xavier, This issue required some tracing through the calls and I see the problem that you have mentioned and patch fixes the problem. One comment on the patch, for the tests in the module, this line - self.frame_returning = None does not seem to have a coverage. Is there a specific significance in setting this to None and would a test help? def dispatch_return(self, frame, arg): if self.stop_here(frame) or frame == self.returnframe: + self.frame_returning = frame self.user_return(frame, arg) + self.frame_returning = None if self.quitting: raise BdbQuit return self.trace_dispatch Sorry for the delay, I shall commit the fix in 3.3.
msg159734 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2012-04-30 20:00
Hi Senthil, Thanks for your help with this issue. self.frame_returning is both a flag to indicate that we are returning from the current frame and a value (the current frame). We need both as set_step() (the method invoked when the user runs the step command) does not know the current frame and wether we are returning from the current frame. Here is a raw sketch of the call chain in the case where the user types the step command on returning from the current frame (Pdb subclasses both bdb.Bdb and cmd.Cmd): Bdb::dispatch_return Pdb::user_return (Bdb overriden method) Pdb::interaction Cmd::cmdloop Cmd::onecmd Pdb::do_step Bdb::set_step So self.frame_returning must be set to None after the call to self.user_return() so that its value is not used in another later step command, where we are not returning from this frame. Actually it is more explicit and more robust to use a try-finally clause, such as: def dispatch_return(self, frame, arg): if self.stop_here(frame) or frame == self.returnframe: try: self.frame_returning = frame self.user_return(frame, arg) finally: self.frame_returning = None if self.quitting: raise BdbQuit return self.trace_dispatch Xavier
msg159741 - (view)	Author: Roundup Robot (python-dev)	Date: 2012-05-01 02:37
New changeset 96cb47f8142e by Senthil Kumaran in branch '3.2': - Fix pdb skipping frames after hitting a breakpoint and running step. Patch by Xavier de Gaye http://hg.python.org/cpython/rev/96cb47f8142e New changeset 5ea23739e9ba by Senthil Kumaran in branch '2.7': - Fix pdb skipping frames after hitting a breakpoint and running step. Patch by Xavier de Gaye http://hg.python.org/cpython/rev/5ea23739e9ba New changeset ab63e874265e by Senthil Kumaran in branch 'default': - Fix pdb skipping frames after hitting a breakpoint and running step. Patch by Xavier de Gaye http://hg.python.org/cpython/rev/ab63e874265e
msg159742 - (view)	Author: Senthil Kumaran (orsenthil) *	Date: 2012-05-01 02:49
Hello Xavier, Thanks for the explanation. Understood the reason for setting the frame_returning to None. The patch is committed in all python versions and the issue is fixed now. Thanks for the patch and prodding through this. -- Senthil
msg159761 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2012-05-01 18:02
The test fails on Windows. Whereas on Unix, the two step commands produce this output: -> print('1') (Pdb) step 1 --Return-- > /net/pao/export/home/staff/loewis/work/33/bar.py(2)bar()->None -> print('1') (Pdb) step --Return-- > /net/pao/export/home/staff/loewis/work/33/main.py(5)foo()->None -> bar() (Pdb) quit on Windows, they produce this output: -> print('1') (Pdb) step --Call-- > c:\users\martin\33\python\lib\encodings\cp850.py(18)encode() -> def encode(self, input, final=False): (Pdb) step > c:\users\martin\33\python\lib\encodings\cp850.py(19)encode() -> return codecs.charmap_encode(input,self.errors,encoding_map)[0] (Pdb) quit I.e. the stepping enters the print, and breaks in the codec. Reopening the issue.
msg159763 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2012-05-01 18:47
My fault :( The call to print is useless for the test, so I suggest to replace it with a plain 'pass' statement.
msg159765 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2012-05-01 19:06
That indeed makes the test pass.
msg159769 - (view)	Author: Roundup Robot (python-dev)	Date: 2012-05-02 00:01
New changeset 3b2aa777b725 by Senthil Kumaran in branch '2.7': fix windows test failure - http://hg.python.org/cpython/rev/3b2aa777b725 New changeset d17ecee3f752 by Senthil Kumaran in branch '3.2': fix windows test failure - http://hg.python.org/cpython/rev/d17ecee3f752 New changeset 0269c592c8b1 by Senthil Kumaran in branch 'default': fix closes - windows test failure http://hg.python.org/cpython/rev/0269c592c8b1
msg159771 - (view)	Author: Senthil Kumaran (orsenthil) *	Date: 2012-05-02 02:03
Fixed again with replacing print with pass. But it is strange behavior that "stepping through" enters print in Windows and does not so in Unix. What's the difference in windows that could cause this? Not sure if this was expected behavior.
msg159774 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2012-05-02 05:38
> But it is strange behavior that "stepping through" enters print in > Windows and does not so in Unix. What's the difference in windows > that could cause this? Not sure if this was expected behavior. On Unix, the codec most likely is UTF-8, which is directly written in C, i.e. doesn't support Python single-stepping.
msg159775 - (view)	Author: Roundup Robot (python-dev)	Date: 2012-05-02 05:41
New changeset 6c9ce7e34511 by Martin v. Löwis in branch 'default': Issue #13183: Revert 0b53b70a40a0 (reenable test on windows) http://hg.python.org/cpython/rev/6c9ce7e34511
msg159813 - (view)	Author: Charles-François Natali (neologix) *	Date: 2012-05-02 18:42
All 3.2 and 2.7 buildbots are still broken: http://python.org/dev/buildbot/all/builders/x86 OpenIndiana 3.2/builds/1080 """ ====================================================================== FAIL: test_issue13183 (test.test_pdb.PdbTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/export/home/buildbot/32bits/3.2.cea-indiana-x86/build/Lib/test/test_pdb.py", line 662, in test_issue13183 'Fail to step into the caller after a return') AssertionError: 'main.py(5)foo()->None' not found in '-> bar()' : Fail to step into the caller after a return ---------------------------------------------------------------------- Ran 2 tests in 0.558s """
msg159818 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2012-05-02 20:05
The test has been changed in the default branch by changeset 1b174a117e19. This change replaces the assertIn by a less restrictive assertTrue. These changes should also probably be made in 3.2 and 2.7 and hopefully this will fix the problem in 3.2 and 2.7. The changeset 1b174a117e19 in the default branch is: diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -604,6 +604,7 @@ filename = 'main.py' with open(filename, 'w') as f: f.write(textwrap.dedent(script)) + self.addCleanup(support.unlink, filename) cmd = [sys.executable, '-m', 'pdb', filename] stdout = stderr = None with subprocess.Popen(cmd, stdout=subprocess.PIPE, @@ -660,9 +661,11 @@ """ with open('bar.py', 'w') as f: f.write(textwrap.dedent(bar)) + self.addCleanup(support.unlink, 'bar.py') stdout, stderr = self.run_pdb(script, commands) - self.assertIn('main.py(5)foo()->None', stdout.split('\n')[-3], - 'Fail to step into the caller after a return') + self.assertTrue( + any('main.py(5)foo()->None' in l for l in stdout.splitlines()), + 'Fail to step into the caller after a return')
msg160065 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2012-05-06 09:32
If the failures don't get fixed, the offending commit should be reverted.
msg160067 - (view)	Author: Roundup Robot (python-dev)	Date: 2012-05-06 09:49
New changeset e275a9f7daa9 by Georg Brandl in branch '3.2': #13183: backport fixes to test_pdb to 3.2 branch http://hg.python.org/cpython/rev/e275a9f7daa9
msg160068 - (view)	Author: Roundup Robot (python-dev)	Date: 2012-05-06 09:54
New changeset 2644e4ea02d3 by Georg Brandl in branch '2.7': #13183: backport fixes to test_pdb to 2.7 branch http://hg.python.org/cpython/rev/2644e4ea02d3
msg160069 - (view)	Author: Georg Brandl (georg.brandl) *	Date: 2012-05-06 09:54
Should be fixed now.
msg160168 - (view)	Author: Xavier de Gaye (xdegaye) *	Date: 2012-05-07 19:53
Note that now that this issue is fixed, issue 14743 has become more visible.
msg160301 - (view)	Author: Senthil Kumaran (orsenthil) *	Date: 2012-05-09 15:44
> Georg Brandl <georg@python.org> added the comment: > > Should be fixed now. Thanks for the commit fix, Georg. The comment on buildbot failures had escaped my attention. Sorry for that.