[Python-Dev] Treating tokenize.Untokenizer as private (original) (raw)

Eric Snow ericsnowcurrently at gmail.com
Mon Mar 31 20:30:36 CEST 2014

On Tue, Feb 18, 2014 at 2:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:

I am working through the multiple bugs afflicting tokenize.untokenize, which is described in the tokenize doc and has an even longer docstring. While the function could be implemented as one 70-line function, it happens to be implemented as a 4-line wrapper for a completely undocumented (Untokenizer class with 4 methods. (It is unmentioned in the doc and there are currently no docstrings.)

I view the class as a private implementation detail and would like to treat it as such, and perhaps even rename it Untokenizer to make that clear. The issue arises in #9974. It appears that a fix may require the addition of an instance attribute or .addwhitespace parameter. If there is objection to treating the whole class as private, I would at least like to treat addwhitespace as the private helper that it is. There is no reason to call it directly except for testing. Otherwise, it could just as well have been left inline at the one call site.

Is this still an open question, Terry?

-eric

More information about the Python-Dev mailing list