[Python-Dev] Python startup time (original) (raw)

Neil Schemenauer nas-python at arctrix.com
Mon May 7 12:28:46 EDT 2018

On 2018-05-03, Lukasz Langa wrote:

> On May 2, 2018, at 8:57 PM, INADA Naoki <songofacandy at gmail.com> wrote: > * Add lazy compiling API or flag in re module. The pattern is compiled > when first used.

How about go the other way and allow compiling at Python compile-time? That would actually make things faster instead of just moving the time spent around.

Lisp has a special form 'eval-when'. It can be used to cause evaluation of the body expression at compile time.

In Carl's "A fast startup patch" post, he talks about getting rid of the unmarshal step and storing objects in the heap segment of the executable. Those would be the objects necessary to evaluate code. The marshal module has a limited number of types that it handle. I believe they are: bool, bytes, code objects, complex, Ellipsis float, frozenset, int, None, tuple and str.

If the same mechanism could handle more types, rather than storing the code to be evaluated, we could store the objects created after evaluation of the top-level module body. Or, have a mechanism to mark which code should be evaluated at compile time (much like the eval-when form).

For the re.compile example, the compiled regex could be what is stored after compiling the Python module (i.e. the re.compile gets run at compile time). The objects created by re.compile (e.g. SRE_Pattern) would have to be something that the heap dumper could handle.

Traditionally, Python has had the model "there is only runtime". So, starting to do things at compile time complicates that model.

Regards,

Neil

More information about the Python-Dev mailing list