Pickling — Serialisation of running tasklets — Stackless-Python 3.9.0a0 documentation (original) (raw)

One of the most impressive features of Stackless-Python, is the ability to pickle tasklets. This allows you to take a tasklet mid-execution, serialise it to a chunk of data and then unserialise that data at a later point, creating a new tasklet from it that resumes where the last left off.

What makes this particularly impressive is the fact that the Python® pickle structure is platform independent. Code can for instance initially be run on a x86 Windows machine, then interrupted, pickled and sent over the network to be resumed on an ARM Linux machine.

Example - pickling a tasklet:

def func(): ... busy_count = 0 ... while 1: ... busy_count += 1 ... if busy_count % 10 == 0: ... print(busy_count) ... stackless.tasklet(func)() <stackless.tasklet object at 0x01BD16B0> t1 = stackless.run(100) 10 20 s = pickle.dumps(t1) t1.kill() t2 = pickle.loads(s) t2.insert() stackless.run(100) 30 40 50

In the above example, a tasklet is created that increments the counter_busy_count_ and outputs the value when it is a multiple of 10.

Run the tasklet for a while:

t1 = stackless.run(100) 10 20

The tasklet has been interrupted at some point in its execution. If it were to be resumed, we would expect its output to be the values following those previously displayed.

Serialise the tasklet:

As any other object is pickled, so are tasklets. In this case, the serialised representation of the tasklet is a string, stored in s.

Destroy the tasklet:

We want to show that the old code cannot be resumed, and in order to do so, we destroy the tasklet it was running within.

Unserialise the stored representation:

As any other object is unpickled, so are tasklets. We take the string and by unpickling it, get a new tasklet object back.

Schedule the new tasklet:

Now the newly recreated tasklet is inserted into the scheduler, so that when the scheduler is next run, the tasklet is resumed.

Run the scheduler:

stackless.run(100) 30 40 50 <stackless.tasklet object at 0x01BD1D30>

When the scheduler is run, the values displayed are indeed the ones that follow those displayed by the original tasklet. The value returned bystackless.run() is not stored in a variable this time, so the interpreter displays the recreated tasklet. You can see that it has a different address than t1, which was displayed earlier.

Note

It should be possible to pickle any tasklets that you might want to. However, not all tasklets can be unpickled. One of the cases in which this is true, is where not all the functions called by the code within the tasklet are Python® functions. The Stackless-Python pickling mechanism has no ability to deal with C functions that may have been called.

Note

It is not possible to unpickle running tasklets which were pickled by a different minor version of Stackless-Python. A running tasklet contains frame objects and frame objects contain code objects. And code objects are usually incompatible between different minor versions of standard Python®.

Pickling other objects

In order to be able to pickle tasklets Stackless-Python needs to be able to pickle several other objects, which can’t be pickled by standard Python®. If the modulestackless gets imported for the first time, _Stackless-Python_uses copyreg.pickle() to register “reduction” functions for the following types:AsyncGeneratorType,CodeType,CoroutineType,FunctionType,GeneratorType,ModuleType,TracebackType,Cell Objects, C-types PyAsyncGenASend and PyAsyncGenAThrow (see PEP 525) as well as all kinds of Dictionary view objects.

Code

Stackless-Python can pickle CodeType objects.

Changed in version 3.8: The pickled representation of a code object contains the bytecode version number (MAGIC_NUMBER). If a program tries to unpickle a code object with a wrong bytecode version number, then Stackless-Python

Frames

Stackless-Python can pickle frames, but only as part of a tasklet, a traceback-object, a generator, a coroutine or an asynchronous generator. Stackless-Python does not register a “reduction” function forFrameType. This way Stackless-Python stays compatible with application code that registers its own “reduction” function for FrameType.

It is not possible to execute an unpickled frame, if the tasklet the original frame belonged to was not restorable. In this case the frame is marked as invalid and any attempt to execute it raises

Changed in version 3.8: If a program tries to unpickle a frame using a code object whose first bytecode instruction is invalid, then _Stackless-Python_marks the frame as invalid. Any attempt to execute the frame raises RuntimeError.

Functions

Stackless-Python can pickle functions including lambda-objects objects by value.

Changed in version 3.8: If a program tries to unpickle a function using a code object whose first bytecode instruction is invalid, then _Stackless-Python_emits a RuntimeWarning('Unpickling function with invalid code object: %V'). Any attempt to execute the function raises SystemError.

Asynchronous Generators

New in version 3.7.

At C-level asynchronous generators have an attribute ag_finalizer and a flag, if ag_finalizer has been initialised. The value of ag_finalizer is a callable_Python®_-object, which has been set by sys.set_asyncgen_hooks(). You can use stackless.pickle_flags() to control how Stackless-Python pickles and unpickles an asynchronous generator.

Pickling

By default (no flags set) Stackless-Python does not pickle ag_finalizer but a marker, if a ag_finalizer has been set. If PICKLEFLAGS_PRESERVE_AG_FINALIZER has been set,Stackless-Python pickles ag_finalizer by value. Otherwise, if PICKLEFLAGS_RESET_AG_FINALIZER has been set, Stackless-Python pickles ag_finalizer as uninitialised.

Unpickling

By default Stackless-Python initialises the generator upon unpickling using thefirstiter and finalizer values set by sys.set_asyncgen_hooks(), if ag_finalizer of the original asynchronous generator was initialised. If PICKLEFLAGS_PRESERVE_AG_FINALIZER has been set and ifag_finalizer has been pickled by value, Stackless-Python unpicklesag_finalizer by value. Otherwise, if PICKLEFLAGS_RESET_AG_FINALIZER has been set, Stackless-Python unpickles ag_finalizer as uninitialised.