| msg120626 - (view) |
Author: Alexander Belopolsky (belopolsky) *  |
Date: 2010-11-06 15:26 |
| Please run attached zip archive as a python script. Note that the problem is not specific for using __main__.py - any module that comes from a zip archive or loaded by a custom loader would show the same bug. $ unzip -l test.zip Archive: test.zip Length Date Time Name -------- ---- ---- ---- 79 10-24-08 18:26 __main__.py -------- ------- 79 1 file $ $ cat __main__.py from trace import Trace def traced(flag): if flag: return 1 else: return 2 tracer = Trace() tracer.runfunc(traced, False) results = tracer.results() results.write_results(coverdir='.') $ python testtrace.zip --- modulename: __main__, funcname: traced Not printing coverage data for 'testtrace.zip/__main__.py': [Errno 20] Not a directory: 'testtrace.zip/__main__.py' __main__.py(4): __main__.py(7): $ |
|
|
| msg120691 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2010-11-07 15:56 |
| I commited Alexander's fix for #10329: r86303. The trace module now uses the input Python script encoding, instead of the locale encoding. |
|
|
| msg120701 - (view) |
Author: Alexander Belopolsky (belopolsky) * |
Date: 2010-11-08 01:32 |
| I am attaching a "proof of concept" patch. The trace code is in a dire need of restructuring to eliminate repeated reading of source files. |
|
|
| msg120703 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2010-11-08 01:47 |
| + try: + with open(filename, 'rb') as fp: + encoding, _ = tokenize.detect_encoding(fp.readline) + except IOError: + encoding = None You should use 'utf-8' instead of None (which will fall back to the locale encoding) here. |
|
|
| msg120704 - (view) |
Author: Alexander Belopolsky (belopolsky) * |
Date: 2010-11-08 01:57 |
| On Sun, Nov 7, 2010 at 8:47 PM, STINNER Victor <report@bugs.python.org> wrote: > > STINNER Victor <victor.stinner@haypocalc.com> added the comment: > > + try: > + with open(filename, 'rb') as fp: > + encoding, _ = tokenize.detect_encoding(fp.readline) > + except IOError: > + encoding = None > > You should use 'utf-8' instead of None (which will fall back to the locale encoding) here. > I know. I was too lazy to look up the correct spelling for the proof of concept. I am really posting this patch to show how this bug can be fixed in theory and discuss how much of refactoring is acceptable. For example, do we need to preserve trace.find_strings() function? What is the best way to pass around source code? - file-like objects, line iterators, readline-like function? Also compile(prog, filename, "exec") to find module's bytecode is a hack. There must be a standard way to achieve that which would use a .pyc file if available. |
|
|
| msg120707 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2010-11-08 01:59 |
| The handling of __loader__ looks fine to me, although I don't quite see the point of the get_source call as it isn't returned or used. |
|
|
| msg120708 - (view) |
Author: Alexander Belopolsky (belopolsky) * |
Date: 2010-11-08 02:03 |
| On Sun, Nov 7, 2010 at 8:59 PM, Brett Cannon <report@bugs.python.org> wrote: > .. I don't quite see the point of the get_source call as it isn't returned or used. It is passed to find_docstrings() to produce the "strs" dictionary. |
|
|
| msg121051 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2010-11-12 17:54 |
| >What is the best way to pass around source code? > - file-like objects, line iterators, readline-like function? Line iterator (list of lines) as returned by open().readlines. Memory should not be an issue. Read disk once and close. with open('file', ...) as f: src = f.readlines() |
|
|
| msg121056 - (view) |
Author: Alexander Belopolsky (belopolsky) * |
Date: 2010-11-12 19:09 |
| On Fri, Nov 12, 2010 at 12:54 PM, Terry J. Reedy <report@bugs.python.org> wrote: .. >>What is the best way to pass around source code? >> - file-like objects, line iterators, readline-like function? > > Line iterator (list of lines) as returned by open().readlines. > Memory should not be an issue. Read disk once and close. > > with open('file', ...) as f: > src = f.readlines() Line iterator does not carry the information about source encoding which may be important for annotating the source code. |
|
|
| msg121067 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2010-11-12 20:07 |
| >Line iterator does not carry the information about source encoding which may be important for annotating the source code. I would pass around both encoding and lines, possibly as a tuple. A person heavily into OO might define a _Source class and turn the private functions into methods. But that would not play well with the deprecation process until it is done with. |
|
|
| msg221928 - (view) |
Author: Alexander Belopolsky (belopolsky) * |
Date: 2014-06-30 01:01 |
| I updated the patch for 3.5. [Brett] > I don't quite see the point of the get_source call as it isn't returned or used. It *is* used: it is passed to _find_strings_stream: + source = loader.get_source(modulename) + strs = _find_strings_stream(io.StringIO(source)) |
|
|
| msg221944 - (view) |
Author: PCManticore (Claudiu.Popa) *  |
Date: 2014-06-30 06:55 |
| Hi, I left a couple of comments on Rietveld. |
|
|