| msg249082 - (view) |
Author: Samuel Isaacson (Samuel Isaacson) |
Date: 2015-08-24 23:49 |
| When inheriting from namedtuples, _asdict and __dict__ return empty dictionaries: from collections import namedtuple class Point(namedtuple('_Point', ['x', 'y'])): pass a = Point(3, 4) print(a._asdict() == {}) gives False; it is True on Python 2.7.6 |
|
|
| msg249093 - (view) |
Author: Samuel Isaacson (Samuel Isaacson) |
Date: 2015-08-25 00:56 |
| Sorry; it returns True on Python 3.4, False on Python 2.7.6. |
|
|
| msg249100 - (view) |
Author: Raymond Hettinger (rhettinger) *  |
Date: 2015-08-25 02:54 |
| For __dict__, I'm not sure what the right behavior should by for subclasses that don't define __slots__. In Python 3, the __dict__ is returning the dict for the subclass. This might be the correct and most desirable behavior: >>> class Point(namedtuple('_Point', ['x', 'y'])): pass >>> a = Point(3, 4) >>> a.w = 5 >>> a.__dict__ {'w': 5} If we leave the __dict__ behavior as is in Py3, then we still need to get _asdict() back to its documented behavior. For that, we would need to disconnect it from __dict__ by restoring the Py2.7 code for _asdict(): def _asdict(self): 'Return a new OrderedDict which maps field names to their values' return OrderedDict(zip(self._fields, self)) All this needs to be thought-out carefully. Putting in __dict__ support originally looked like a bugfix to get vars() working correctly, but it caused problems with pickling which then led to the addition of __getnewargs__. It seems that defining __dict__ leads to problems no matter how you do it. My inclination is to remove __dict__ and __getewargs__ from the namedtuple definition entirely and return to a simpler state of affairs that is easier to reason about and less likely to lead to unexpected behaviors like the one in this bug report. One note: using the Py2.7 namedtuple code in Python3 still doesn't restore the old behavior. Something else in the language appears to have changed (causing the subclasses' __dict__ to take precedence over the inherited __dict__ property). |
|
|
| msg249358 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2015-08-30 16:17 |
| New changeset fa3ac31cfa44 by Raymond Hettinger in branch '3.4': Issue #24931: Resolve __dict__ conflict in namedtuple subclasses. https://hg.python.org/cpython/rev/fa3ac31cfa44 |
|
|
| msg249403 - (view) |
Author: Eric Snow (eric.snow) * |
Date: 2015-08-31 15:09 |
| Doesn't the fix mean that `vars(MyNamedtuple)` will no longer work? While I personally don't mind (though I prefer that spelling) and appreciate the benefit of simpler code, isn't there a backward-compatibility issue here? I do concede that fixing this bug without a compatibility break is complicated and probably not worth it. However I want to make sure that the point is at least discussed briefly. |
|
|
| msg252796 - (view) |
Author: Matthias Klose (doko) * |
Date: 2015-10-11 10:11 |
| reopening. the incompatible behavior is report in a Debian report as well. """ collections.namedtuple appears to not to create namedtuples which have the __dict__ attribute any more. Given that the stdlib docs stated that using vars() (and hence the existence of __dict__) was preferred, it seems odd that it has disappeared. """ https://bugs.debian.org/800370 |
|
|
| msg252802 - (view) |
Author: Larry Hastings (larry) * |
Date: 2015-10-11 12:41 |
| Why is this marked as a release blocker? It doesn't strike me as all that major. |
|
|
| msg252804 - (view) |
Author: Matthias Klose (doko) * |
Date: 2015-10-11 12:45 |
| now marked as regression, and lowered the priority. but how should regressions on release branches be marked else? |
|
|
| msg252869 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2015-10-12 16:18 |
| > it seems odd that it has disappeared. It disappeared because it was fundamentally broken in Python 3, so it had to be removed. Providing __dict__ broke subclassing and produced odd behaviors. |
|
|
| msg256129 - (view) |
Author: Nicholas Chammas (nchammas) * |
Date: 2015-12-08 21:51 |
| Should this change be called out in the 3.5.1 release docs? It makes some code that works on 3.5.0 break in 3.5.1. See: http://stackoverflow.com/q/34166469/877069 |
|
|
| msg256130 - (view) |
Author: Larry Hastings (larry) * |
Date: 2015-12-08 22:04 |
| You're a little late; 3.5.1 was released two days ago. |
|
|
| msg256131 - (view) |
Author: Nicholas Chammas (nchammas) * |
Date: 2015-12-08 22:37 |
| I know. I came across this issue after upgrading to the 3.5.1 release and seeing that vars(namedtuple) didn't work anymore. I looked through the changelog [0] for an explanation of why that might be and couldn't find one, so I posted that question on Stack Overflow. I'm guessing others will go through the same flow after they upgrade to 3.5.1 and wonder why their vars(namedtuple) code broke, so I posted here asking if we should amend the changelog to call this change out. But I gather from your comment that the changelog cannot be updated after the release, so I guess there is nothing to do here. (Sorry about the distraction. I'm new to the Python dev community.) [0] https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-1-final |
|
|
| msg256995 - (view) |
Author: Vedran Čačić (veky) * |
Date: 2015-12-25 15:56 |
| I agree that namedtuples having __dict__ is probably more trouble than benefit. But in my view, that's no reason for _asdict to not work correctly. The whole point of separate function (even going through the pain of underscore-starting public API, since everything else is even bigger pain) is that we sidestep the question of vars() and other standard Python hooks, and provide our way of extracting a namedtuple's namespace, for ones who need it. Of course, the fix/workaround is trivial, as Raymond says: just def _asdict(self): return collections.OrderedDict(zip(self._fields, self)) That way, it doesn't matter whether self has a `__dict__` or not. If you think it shouldn't be done, the documentation really needs to be changed. Current wording is very misleading, showing that `_asdict` works on a class having `__slots__ = ()`, but containing no explanation of that attribute being _necessary_ for `_asdict` to work. (It just says "This helps keep memory requirements low by preventing the creation of instance dictionaries.") |
|
|
| msg257065 - (view) |
Author: Jens Troeger (_savage) * |
Date: 2015-12-27 04:29 |
| With my update from Python 3.4.3 to Python 3.4.4 (default, Dec 25 2015, 06:14:41) I started experiencing crashes of my applications and I suspect this change is the culprit. I have a class that inherits from namedtuple, and code calls vars() (i.e. retrieve __dict__) to iterate over an instance's attributes. Much like Raymond points out in http://bugs.python.org/msg249100 For example with 3.4.3: >>> from collections import namedtuple >>> Point = namedtuple('Point', ['x', 'y']) >>> p = Point(1,2) >>> p Point(x=1, y=2) >>> p.__dict__ OrderedDict([('x', 1), ('y', 2)]) >>> vars(p) OrderedDict([('x', 1), ('y', 2)]) After the most recent update this breaks with 3.4.4: >>> from collections import namedtuple >>> Point = namedtuple('Point', ['x', 'y']) >>> p = Point(1,2) >>> p Point(x=1, y=2) >>> p.__dict__ Traceback (most recent call last): File "", line 1, in AttributeError: 'Point' object has no attribute '__dict__' >>> vars(p) Traceback (most recent call last): File "", line 1, in TypeError: vars() argument must have __dict__ attribute I am not sure about the fix on my side. Should I use _asdict() instead of vars() although I would argue that vars() should remain functional across this change. Calling _asdict() seems messy to me, but it works: >>> p._asdict() OrderedDict([('x', 1), ('y', 2)]) Why not keep the __dict__ property in tact? @property def __dict__(self): return self._asdict() Thanks! |
|
|