[Python-Dev] More optimisation ideas (original) (raw)

Terry Reedy tjreedy at udel.edu
Thu Feb 4 19:58:30 EST 2016

On 2/4/2016 12:18 PM, Sven R. Kunze wrote:

On 04.02.2016 14:09, Nick Coghlan wrote:

On 2 February 2016 at 06:39, Andrew Barnert via Python-Dev <python-dev at python.org> wrote:

On Feb 1, 2016, at 09:59,mike.romberg at comcast.net wrote:

If the stdlib were to use implicit namespace packages (https://www.python.org/dev/peps/pep-0420/ ) and the various loaders/importers as well, then python could do what I've done with an embedded python application for years. Freeze the stdlib (or put it in a zipfile or whatever is fast). Then arrange PYTHONPATH to first look on the filesystem and then look in the frozen/ziped storage. This is a great solution for experienced developers, but I think it would be pretty bad for novices or transplants from other languages (maybe even including Python 2).

There are already multiple duplicate questions every month on StackOverflow from people asking "how do I find the source to stdlib module X". The canonical answer starts off by explaining how to import the module and use its file, which everyone is able to handle. If we have to instead explain how to work out the .py name from the qualified module name, how to work out the stdlib path from sys.path, and then how to find the source from those two things, with the caveat that it may not be installed at all on some platforms, and how to make sure what they're asking about really is a stdlib module, and how to make sure they aren't shadowing it with a module elsewhere on sys.path, that's a lot more complicated. Especially when you consider that some people on Windows and Mac are writing Py thon scripts without ever learning how to use the terminal or find their Python packages via Explorer/Finder. For folks that do know how to use the terminal: $ python3 -m inspect --details inspect Target: inspect Origin: /usr/lib64/python3.4/inspect.py Cached: /usr/lib64/python3.4/pycache/inspect.cpython-34.pyc Loader: <frozenimportlib.SourceFileLoader object at 0x7f0d8d23d9b0> (And if they just want to read the source code, then leaving out "--details" prints the full module source, and would work even if the standard library were in a zip archive)

This is completely inadequate as a replacement for loading source into an editor, even if just for reading.

First, on Windows, the console defaults to 300 lines. Print more and only the last 300 lines remain. The max is buffer size is 9999. But setting the buffer to that is obnoxious because the buffer is then padded with blank lines to make 9999 lines. The little rectangle that one grabs in the scrollbar is then scaled down to almost nothing, becoming hard to grab.

Second is navigation. No Find, Find-next, or Find-all. Because of padding, moving to the unpadded 'bottom of file' is difficult.

Third, for a repository version, I would have to type, without error, instead of 'python3', some version of, for instance, some suffix of 'F:/python/dev/35/PcBuild//python_d.exe'. "" depends, I believe, on the build options.

I want to see and debug also core Python in PyCharm and this is not acceptable.

If you want to make it opt-in, fine. But opt-out is a no-go. I have a side-by-side comparison as we use Java and Python in production. It's the ease of access that makes Python great compared to Java. @Andrew Even for experienced developers it just sucks and there are more important things to do.

I agree that removing stdlib python source files by default is an poor idea. The disk space saved is trivial. So, for me, would be nearly all of the time saving.

Over recent versions, more and more source files have been linked to in the docs. Guido recently approved of linking the rest. Removing source contradicts this trend.

Easily loading modules, including stdlib modules, into an IDLE Editor Window is a documented feature that goes back to the original commit in Aug 2000. We not not usually break stdlib features without acknowledgement, some decussion, and a positive decision to do so.

Someone has already mentioned the degredation of tracebacks.

So why not just leave the source files alone in /Lib. As far as I can see, they would not hurt anything At least on Windows, zip files are treated as directories and python35.zip comes before /Lib on sys.path.

The Windows installer currently has an option, selected by default I believe, to run compileall. Add to compileall an option to compile all to python35.zip rather than __pycache and and use that in that installer. Even if the zip is including in the installer, compileall-zip + source files would let adventurous people patch their stdlib files.

Editing a stdlib file, to see if a confirmed bug disappeared (it did), was how I made my first code contribution. If I had had to download and setup svn and maybe visual c to try a one line change, I would not have done it.

-- Terry Jan Reedy

More information about the Python-Dev mailing list