[Python-Dev] PEP 435: pickling enums created with the functional API (original) (raw)

Eli Bendersky eliben at gmail.com
Tue May 7 15:34:50 CEST 2013

One of the contended issues with PEP 435 on which Guido pronounced was the functional API, that allows created enumerations dynamically in a manner similar to namedtuple:

Color = Enum('Color', 'red blue green')

The biggest complaint reported against this API is interaction with pickle. As promised, I want to discuss here how we're going to address this concern.

At this point, the pickle docs say that module-top-level classes can be pickled. This obviously works for the normal Enum classes, but is a problem with the functional API because the class is created dynamically and has no module.

To solve this, the reference implementation is used the same approach as namedtuple (*). In the metaclass's new (this is an excerpt, the real code has some safeguards):

module_name = sys._getframe(1).f_globals['name'] enum_class.module = module_name

According to an earlier discussion, this is works on CPython, PyPy and Jython, but not on IronPython. The alternative that works everywhere is to define the Enum like this:

Color = Enum('the_module.Color', 'red blue green')

The reference implementation supports this as well.

Some points for discussion:

  1. We can say that using the functional API when pickling can happen is not recommended, but maybe a better way would be to just explain the way things are and let users decide?

  2. namedtuple should also support the fully qualified name syntax. If this is agreed upon, I can create an issue.

  3. Antoine mentioned that work is being done in 3.4 to enable pickling of nested classes (http://www.python.org/dev/peps/pep-3154/). If that gets implemented, I don't see a reason why Enum and namedtuple can't be adjusted to find the qualname of the class they're internal to. Am I missing something?

  4. Using _getframe(N) here seems like an overkill to me. What we really need is just the module in which the current execution currently is (i.e. the metaclass's new in our case). Would it make sense to add a new function somewhere in the stdlib of 3.4 (in sys or inspect or ...) that just provides the current module name? It seems that all Pythons should be able to easily provide it, it's certainly a very small subset of the functionality provided by walking the callframe stack. This function can then be used for build fully qualified names for pickling of Enum and namedtuple. Moreover, it can be general even more widely - dynamic class building is quite common in Python code, and as Nick mentioned somewhere earlier, the extra power of metaclasses in the recent 3.x's will probably make it even more common.

Eli

(*) namedtuple uses an explicit function to build the resulting class, not a metaclass (ther's no class syntax for namedtuple). -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20130507/7e26c729/attachment.html>

More information about the Python-Dev mailing list