[Python-Dev] Python startup optimization: script vs. service (original) (raw)

Nick Coghlan ncoghlan at gmail.com
Wed Oct 4 00:24:10 EDT 2017

On 3 October 2017 at 03:02, Christian Heimes <christian at python.org> wrote:

On 2017-10-02 16:59, Barry Warsaw wrote:

On Oct 2, 2017, at 10:48, Christian Heimes <christian at python.org> wrote:

That approach could work, but I think that it is the wrong approach. I'd rather keep Python optimized for long-running processes and introduce a new mode / option to optimize for short-running scripts. What would that look like, how would it be invoked, and how would that change the behavior of the interpreter? I haven't given it much thought yet. Here are just some wild ideas: - add '-l' command line option (l for lazy) - in lazy mode, delay some slow operations (re compile, enum, ...) - delay some imports in lazy mode, e.g. with a deferred import proxy

I don't think is the right way to structure the defaults, since the web services world is in the middle of moving back closer to the CLI/CGI model, where a platform like AWS Lambda will take care of spinning up language interpreter instances on demand, using them to process a single request, and then discarding them.

It's also somewhat unreliable to pass command line options as part of shebang lines, and packaging tools need to be able to generate shebang lines that are compatible with a wide variety of Python implementations

By contrast, long running Python services will typically be using some form of WSGI server (whether that's mod_wsgi, uWSGI, gunicorn, Twisted, tornado, or something else) that can choose to adjust their defaults to force the underlying language runtime into an "eager state initialisation" mode, even if the default setting is to respect requests for lazy initialisation.

Cheers, Nick.

-- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia

More information about the Python-Dev mailing list