[Python-Dev] More optimisation ideas (original) (raw)

Brett Cannon brett at python.org
Mon Feb 1 12:23:55 EST 2016

On Mon, 1 Feb 2016 at 08:48 R. David Murray <rdmurray at bitdance.com> wrote:

On Mon, 01 Feb 2016 14:12:27 +1100, Steven D'Aprano <steve at pearwood.info> wrote: > On Sun, Jan 31, 2016 at 08:23:00PM +0000, Brett Cannon wrote: > > So freezing the stdlib helps on UNIX and not on OS X (if my old testing is > > still accurate). I guess the next question is what it does on Windows and > > if we would want to ever consider freezing the stdlib as part of the build > > process (and if we would want to change the order of importers on > > sys.metapath so frozen modules came after file-based ones). > > I find that being able to easily open stdlib .py files in a text editor > to read the source is extremely valuable. I've learned much more from > reading the source than from (e.g.) StackOverflow. Likewise, it's often > handy to do a grep over the stdlib. When you talk about freezing the > stdlib, what exactly does that mean? > > - will the source files still be there?

Well, Brett said it would be optional, though perhaps the above paragraph is asking about doing it in our Windows build.

Nope, it would probably need to be across all OSs to have consistent semantics.

But the linux distros might make also use the option if it exists, so the question is very meaningful. However, you'd have to ask the distro if the source would be shipped in the linux case, and I'd guess not in most cases.

I don't know about anyone else, but on my own development systems it is not that unusual for me to edit the stdlib files (to add debug prints) while debugging my own programs. Freeze would definitely interfere with that. I could, of course, install a separate source build on my dev system, but I thought it worth mentioning as a factor.

This is what would need to be discussed in terms of how to handle this. For instance, we already do stuff in (I believe) site.py when we detect the build is in a checkout, so we could in that instance make sure the stdlib file directory takes precedence over any frozen code (hence why I wondered if the frozen importer on sys.meta_path should come after the sys.path importer). If we did that then we could make installing the stdlib files optional but still take precedence.

It's all workable, it's just a question of if we want to. This is why I think we should get concrete benchmark numbers on Windows, Linux, and OS X to see if this is even worth considering as something we provide in our own binaries.

On the other hand, if the distros go the way Nick has (I think) been advocating, and have a separate 'system python for system scripts' that is independent of the one installed for user use, having the system-only python be frozen and sourceless would actually make sense on a couple of levels.

It at least wouldn't hurt anything. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20160201/ee500827/attachment.html>

More information about the Python-Dev mailing list