Issue 4197: Doctest module does not work with zipped packages (original) (raw)

Created on 2008-10-24 21:59 by belopolsky, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
testmodule.zip belopolsky,2008-10-24 21:59 Zip file with a module containing doctest inside
doctest.patch belopolsky,2008-10-24 22:07
doctest-1.patch belopolsky,2008-10-24 22:12
Messages (14)
msg75186 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 21:59
With attached teestmodule.zip and svn revision 67006, $ ./python.exe testmodule.zip Traceback (most recent call last): File "/Users/sasha/Work/python-svn/trunk/Lib/runpy.py", line 121, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/Users/sasha/Work/python-svn/trunk/Lib/runpy.py", line 34, in _run_code exec code in run_globals File "testmodule.zip/__main__.py", line 7, in File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 1817, in testmod for test in finder.find(m, name, globs=globs, extraglobs=extraglobs): File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 842, in find self._find(tests, obj, name, module, source_lines, globs, {}) File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 884, in _find test = self._get_test(obj, name, module, globs, source_lines) File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 965, in _get_test if filename[-4:] in (".pyc", ".pyo"): TypeError: 'NoneType' object is unsubscriptable
msg75188 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 22:07
Attached patch addresses the crash, but two issues remain: $ ./python.exe testmodule.zip ********************************************************************** File "__main__", line ?, in __main__.c Failed example: 'line 2' Expected nothing Got: 'line 2' ********************************************************************** 1 items had failures: 1 of 1 in __main__.c ***Test Failed*** 1 failures. The line number for the failed example cannot be found and file name is reported as __main__.c instead of __main__.py
msg75189 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 22:12
The next patch, doctest-1.patch addresses the line number issue, but the file name is still wrong. $ ./python.exe testmodule.zip ********************************************************************** File "__main__", line 2, in __main__.c Failed example: 'line 2' Expected nothing Got: 'line 2' ********************************************************************** 1 items had failures: 1 of 1 in __main__.c ***Test Failed*** 1 failures.
msg75191 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 22:22
I don't think there is an easy way to fix source file name reporting without injecting __file__ = '/' into the module namespace is zipimporter, but I don't know if any code relies on __file__ set to None. Setting __file__ to (in the test case) testmodule.zip/__main__.py will eliminate the need for the first patch and the second patch will only need to pass globs to linecache.getlines() to fix the line number issue.
msg76837 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2008-12-03 20:49
runpy needs a non-standard PEP 302 extension to set __file__ correctly in the modules it runs. The pkgutil stuff it uses to find pure Python modules in the filesystem supports that extension, but zipimport doesn't (yet).
msg76860 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-12-04 00:44
I created to implement get_filename extension in zipimport. Will post a patch shortly.
msg76872 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-12-04 05:54
While the patch at fixes the crash, it does not fix the line number issue. I would prefer to start with doctest-1.patch like solution because changing the PEP 302 loader protocol by adding get_filename needs additional discussion and certainly is not a candidate for 2.5.x or 2.6.x bug fix releases.
msg77511 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-12-10 08:55
The precise solution still seems to be debated, so this is out of scope for 2.5.3.
msg77783 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2008-12-14 10:56
Fixed for 2.7 in r67750. Will be ported to 2.6, 3.0 and 3.1.
msg77791 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2008-12-14 12:11
Final revisions for fix: 2.7 = r67751 (there was a new test file missing from the initial checkin) 2.6 = r67752 3.1 = r67753 3.0 = r67754
msg77805 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-12-14 16:32
Nick, Your commit does not fix the line number reporting from doctest failures: $ ./python.exe testmodule.zip ********************************************************************** File "testmodule.zip/__main__.py", line ?, in __main__.c ... Note the '?' in the output. Did you miss the doctest.py changes in doctest-1.patch, or should I open a separate issue for the line number bug? There is also a problem of __main__.c which my patch does not address, but it is less important than the line number. On Sun, Dec 14, 2008 at 7:11 AM, Nick Coghlan <report@bugs.python.org> wrote: > > Nick Coghlan <ncoghlan@gmail.com> added the comment: > > Final revisions for fix: > 2.7 = r67751 (there was a new test file missing from the initial checkin) > 2.6 = r67752 > 3.1 = r67753 > 3.0 = r67754 > > ---------- > resolution: -> fixed > status: open -> closed > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue4197> > _______________________________________ >
msg77857 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2008-12-15 10:29
Re-opening, pending development of a fully passing test specifically for the __main__.py in zipfile case.
msg77858 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2008-12-15 11:05
I have a test case locally that demonstrates both the missing line number in the __main__.py from a zipfile case, as well as the display of main.c as the filename (the latter is not specific to the zipfile case - it also happens for a normal main module). It still seems to me that there is something going wrong at a deeper level for doctest to be getting the "in main.c" part wrong. I want to get to the bottom of that before working around the problem at the line number lookup stage.
msg77863 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2008-12-15 12:09
It turns out you were pretty close to pinpointing the problem in doctest, but didn't quite manage to identify which step was going wrong. The problem was actually that even after __file__ was being set correctly, the call to linecache.getlines wasn't being given the __loader__ access it needed in order to retrieve the source code from inside the zipfile. The __main__.c thing turned out just to be a red herring caused by the choice of class name in the test code (I changed it to be __main__.Test for the checked in unit test). Checkin versions: 2.7 = r67790 2.6 = r67791 3.1 = r67792 3.0 = r67793
History
Date User Action Args
2022-04-11 14:56:40 admin set github: 48447
2008-12-15 12:09:35 ncoghlan set status: open -> closedresolution: fixedmessages: +
2008-12-15 11:05:52 ncoghlan set messages: +
2008-12-15 10:29:12 ncoghlan set status: closed -> openresolution: fixed -> (no value)messages: +
2008-12-14 16:32:45 belopolsky set messages: +
2008-12-14 12:11:16 ncoghlan set status: open -> closedresolution: fixedmessages: +
2008-12-14 10:56:41 ncoghlan set messages: +
2008-12-10 08:55:55 loewis set nosy: + loewismessages: + versions: - Python 2.5.3
2008-12-04 05:54:53 belopolsky set messages: +
2008-12-04 00:44:56 belopolsky set messages: +
2008-12-03 20:49:29 ncoghlan set messages: +
2008-12-03 20:29:01 ncoghlan set assignee: ncoghlannosy: + ncoghlan
2008-10-25 01:53:49 belopolsky set versions: + Python 2.6, Python 2.5.3
2008-10-24 22:22:14 belopolsky set messages: +
2008-10-24 22:12:44 belopolsky set files: + doctest-1.patchmessages: +
2008-10-24 22:07:44 belopolsky set files: + doctest.patchkeywords: + patch
2008-10-24 22:07:07 belopolsky set messages: +
2008-10-24 21:59:36 belopolsky create