Issue 4331: Add functools.partialmethod - Python tracker (original) (raw)
Created on 2008-11-16 06:32 by ssadler, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Messages (32)
Author: scott sadler (ssadler)
Date: 2008-11-16 06:32
Calling a function created by _functools.partial as a method raises an exception:
"TypeError: method_new() takes exactly n non-keyword arguments (0 given)"
Where method_new is the function passed into partial() and n is the number of arguments it expects.
This does not happen when using a python version of partial().
Strangely, in the circumstance that I originally encountered the bug, there was one instance that I was doing this and it DID WORK. The function being passed into partial() was the same as in the place where it was failing. The only significant difference that I could see was that the input function to partial() was being imported, rather than being defined in the same namespace as it was used I was unable to reproduce it in my test case (attatched).
Tested on 2.6 and 2.5.2
Author: scott sadler (ssadler)
Date: 2008-11-16 20:24
A short update, I believe that the reason that it was working in one instance was because of some abstractions by a base class (Django model, get_absolute_url).
Author: Calvin Spealman (ironfroggy)
Date: 2008-11-16 22:44
I don't think this is any kind of bug, it is simply a product of only function objects being decorated automatically as methods. Your python version works because it is, in fact, a function. _functools.partial objects are not functions, but simply callable objects.
Author: Raymond Hettinger (rhettinger) * 
Date: 2008-11-16 23:03
Reclassifying as a feature request. A descriptor could be added to partial() so that it too would have automatic method binding just like pure python functions.
Author: Christophe Simonis (Christophe Simonis)
Date: 2010-01-09 20:59
I followed the advice of Raymond and implement a descriptor on partial.
Author: Alexander Belopolsky (Alexander.Belopolsky)
Date: 2010-01-29 00:25
Christophe,
It looks like your patch goes out of its way to avoid creating nested partials. This is a worthwhile goal and I think it should be done in partial_new so that partial(partial(f, x), y) returns partial(f, x, y).
If fact, I was surprised to learn that current partial implementation does not behave this way:
partial(partial(f, 1), 2).func <functools.partial object at 0x100435af8>
Does anyone know the reason for the current behavior? It is possible that I am missing some subtlety related to keyword arguments.
Author: Alexander Belopolsky (Alexander.Belopolsky)
Date: 2010-02-01 19:09
Please see for a related patch.
Author: Jack Diederich (jackdied) *
Date: 2010-02-22 16:33
I'm having some trouble wrapping my head around this one. It isn't obvious to me that my_method(*args): print(args) class A(): meth = partial(my_method, 'argA') ob = A() ob.meth('argB')
should print (<A object at 0x1234>, 'argA', 'argB') and not ('argA', <A object at 0x1234>, 'argB')
The patch seems to prefer the first form but if you are using a partial shouldn't you expect 'argA' to always be the first argument to the partial-ized function?
Author: R. David Murray (r.david.murray) *
Date: 2010-02-22 19:11
I would expect the second and would view the first as a bug.
Author: Jack Diederich (jackdied) *
Date: 2010-02-23 21:35
We talked about it at sprints and the semantics are ambiguous and there are alternatives.
Ambiguous: def show_funcs(*args): print(args) class A(): run = partial(1) ob = A() ob.run(2,3) Should this print (self, 1, 2, 3) or (1, self, 2, 3)? And what about partial(ob.run, 2)(3)
Alternatives: partial is a convenience function not an optimization (it doesn't offer a speedup. So you can write a lambda or named function that has the exact semantics you want without suffering a speed penalty.
So unless there are a lot of good use cases with obvious behavior, we should refuse the temptation to guess and leave partial as-is.
Author: Jack Diederich (jackdied) *
Date: 2010-02-23 21:37
correction: run = partial(1) should have been run = partial(show_funcs, 1)
Author: Matt Joiner (anacrolix)
Date: 2012-03-24 18:23
I've attached a patch that implements the descriptor protocol for functools.partial with minimum changes.
Author: Ulrich Eckhardt (eckhardt)
Date: 2013-02-07 08:36
Just for the record, the behaviour is documented, unfortunately in the very last line of the functools documentation: "Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up."
Concerning how exactly they should behave during that lookup, I'd use the least surprising variant, namely that they are not treated differently from other functions: The first parameter is implicitly "self".
Author: Matt Joiner (anacrolix)
Date: 2013-02-08 04:02
What's preventing this from being committed and closed?
Author: Ulrich Eckhardt (eckhardt)
Date: 2013-02-25 14:19
There is at least one thing that is missing in the patch, it lacks the necessary tests. The partialbug.py demonstrates the issue, it could be used as a base. However, even then, there is still one thing that is problematic: The fact that partial() returns something that behaves like a static method is documented and changing that is not backward compatible.
I still think that something like this should become part of Python though. Jack Diederich argues that you can use lambda to achieve the same, but that is not always true. If you want to bind an argument to the current value of a variable instead of a constant, lambda fails. You need the closure created by a function call to bind those variables inside a local function. Having a dedicated function for that is IMHO preferable to people copying the Python-only equivalent of partial() to achieve the same effect or even inventing their own.
Author: R. David Murray (r.david.murray) *
Date: 2013-02-25 15:02
See also issue 11470.
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-06-05 04:48
I don't believe it is reasonable to change the behaviour of partial at this late stage of the game. It's documented as behaving like staticmethod (albeit by not implementing the descriptor protocol at all), so that's no longer something we can change. If issue 11470 is added, then we'll just implement the staticmethod-like behaviour explicitly rather than leaving it as implicit.
More importantly, the acceptance of PEP 443's functools.singledispatch makes it more desirable than ever to support partial binding for method implementations even when the descriptor protocol is not involved.
Accordingly, I suggest converting this proposal to a separate functools.partialmethod API that:
- When called directly, passes the first positional argument as the first positional argument of the underlying function (providing call time binding of self, just like a normal function)
- When retrieved from a class, returns itself
- When retrieved from an instance, returns an appropriate bound method object (providing method lookup time binding of self, just like a normal function)
functools.partial will then continue to behave as it always has (thus posing no backwards compatibility risks), while the new partialmethod implementation should work with both class descriptor protocol based dispatch (through point 3) and the new functools.singledispatch mechanism (through point 1).
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-06-05 07:42
Any associated tests may also want check that wrapping classmethod around a partialmethod generates a well behaved class method, and ditto for property.
If singledispath, classmethod, partialmethod and class and instance attribute access all work correctly, then we can be absolutely certain the results is behaving just like an ordinary function does :)
Author: Matt Joiner (anacrolix)
Date: 2013-06-05 10:00
This sounds excellent Nick.
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-10-24 10:33
To clarify the current state of this:
- I'm still in favour of adding this feature for Python 3.4
- a suitable patch is still needed, as the currently attached patches modify the existing functools.partial object, rather than adding a separate "partialmethod" API
- a Python implementation would be fine
The following prototype should work as a starting point to be elaborated into a full patch with docs and tests:
class partialmethod(functools.partial): def get(self, obj, cls): if obj is None: return self return functools.partial(self.func, *((obj,) + self.args), **self.keywords) def call(*args, **kwds): self, *args = args call_kwds = {} call_kwds.update(self.keywords) call_kwds.update(kwds) return self.func(self, *(self.args + args), **call_kwds)
class C: def example(self, *args, **kwds): print(self, args, kwds) fails = functools.partial(example, 1, 2, 3, x=1) works = partialmethod(example, 1, 2, 3, x=1)
C().fails() 1 (2, 3) {'x': 1} C().works() <__main__.C object at 0x7f91cefeea90> (1, 2, 3) {'x': 1} C().fails(4, 5, 6) 1 (2, 3, 4, 5, 6) {'x': 1} C().works(4, 5, 6) <__main__.C object at 0x7f91cefeea10> (1, 2, 3, 4, 5, 6) {'x': 1}
Author: alon horev (alonho) *
Date: 2013-10-25 19:27
I just want to make sure I understand the semantics concerning class methods, the following example demonstrates a usage similar to regular methods as much as possible:
class A(object): def add(self, x, y): print(self) return x + y add10 = partialmethod(add, 10) add10class = classmethod(partialmethod(add, 10))
assert A().add10(5) == 15 # prints <__main__.A object at 0x1097e1390> assert A.add10class(5) == 15 # prints <class '__main__.A'>
Another option would be to return a class-bound partial from the get method. It's not as consistent as the first example but perhaps nicer:
class A(object): def add(self, x, y): print(self) return x + y add10 = partialmethod(add, 10)
assert A().add10(5) == 15 # prints <__main__.A object at 0x1097e1390> assert A.add10(5) == 15 # prints <class '__main__.A'>
Is the first option what you had in mind?
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-10-26 01:57
On 26 Oct 2013 05:28, "alon horev" <report@bugs.python.org> wrote:
Is the first option what you had in mind?
That's actually an interesting question. I was going to say yes, but then I realised it would be better to just "do the right thing" when the underlying object was a classmethod descriptor, rather than composing them the other way around.
That view means we should be delegating get to the underlying descriptor and responding appropriately to the result. And for call we then can't play games at all, since what my sketch does would be wrong when wrapping staticmethod.
We also need to make sure the descriptor does the right thing when @abstractmethod is involved.
Author: alon horev (alonho) *
Date: 2013-10-26 14:59
Here's another attempt at a consistent api with regular methods. I'm contemplating whether partialmethod should support call. Given the fact partial is used to bind positional arguments, it will do the 'wrong' thing when calling the partialmethod directly and will shift all positional arguments (example at the last line of the snippet).
I also think staticmethod in this context is useless but agree consistency is a thing (you could just use partial instead).
If consistency hadn't held us back, the first proposal of partialmethod, working both for instances and classes, would have been most elegant.
from functools import partial
class partialmethod(object):
def __init__(self, func, *args, **keywords):
self.func = func
self.args = args
self.keywords = keywords
def __call__(self, *args, **keywords):
call_keywords = {}
call_keywords.update(self.keywords)
call_keywords.update(keywords)
return self.func(*(self.args + args), **call_keywords)
def __get__(self, obj, cls):
return partial(self.func.__get__(obj, cls), *self.args, **self.keywords)class A(object): def add(self, x, y): print(self) return x + y add10 = partialmethod(add, 10) add10class = partialmethod(classmethod(add), 10) add10static = partialmethod(staticmethod(add), 'self', 10)
assert A().add10(5) == 15 # prints <__main__.A object at 0x1097e1390> assert A.add10class(5) == 15 # prints <class '__main__.A'> assert A.add10static(5) == 15 # prints self assert A.add10(2, 3) == 5 # prints 10 because the first positional argument is self..
Once we approve of the API I'll provide a full fledged patch.
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-10-26 17:18
I like your suggestion of not providing call(), as I don't see a way to make it work with arbitrary underlying descriptors, and neither classmethod nor staticmethod is callable.
In terms of usage, I think this approach will be OK, as in practice I expect @classmethod, etc, will be applied to the initial method definition as appropriate, so they won't need to be inline except in test cases like these.
Additional test cases needed:
A().add10class(5)
A().add10static(5)
A.add10(A(), 5)All three should produce 15 as the result, and print the same thing as the following:
A.add10class(5)
A.add10static(5)
A().add10(5)A test method that uses a partial instance as the callable would also be helpful in assuring the edge cases are covered.
I believe it may be necessary to have a get method something like:
def _make_unbound_method(self, cls):
def _unbound_method(*args, **kwds):
call_keywords = self.keywords.copy()
call_keywords.update(keywords)
return self.func(*(self.args + args), **call_keywords)
_unbound_method.__objclass__ = cls
return _unbound_method
def __get__(self, obj, cls):
get = getattr(self.func, "__get__")
if get is None:
if obj is None:
return self._make_unbound_method(cls)
callable = self.func
else:
callable = get(obj, cls)
if callable is self.func:
return self._make_unbound_method(cls)
return partial(callable, *self.args, **self.keywords)Author: alon horev (alonho) *
Date: 2013-10-27 12:17
I think the following test demonstrates the API we're looking for.
- Am I missing any functionality?
- How does partialmethod relate to @absolutemethod?
from functools import partial
class partialmethod(object):
def __init__(self, func, *args, **keywords):
# func could be a descriptor like classmethod which isn't callable,
# so we can't inherit from partial (it verifies func is callable)
if isinstance(func, partialmethod):
# flattening is mandatory in order to place cls/self before all other arguments
# it's also more efficient since only one function will be called
self.func = func.func
self.args = func.args + args
self.keywords = {}
self.keywords.update(func.keywords)
self.keywords.update(keywords)
else:
self.func = func
self.args = args
self.keywords = keywords
def _make_unbound_method(self):
def _method(*args, **keywords):
keywords.update(self.keywords)
cls_or_self, *rest = args
call_args = (cls_or_self,) + self.args + tuple(rest)
return self.func(*call_args, **keywords)
return _method
def __get__(self, obj, cls):
get = getattr(self.func, "__get__", None)
if get is None:
if obj is None:
return self._make_unbound_method()
return partial(self._make_unbound_method(), obj) # returns a bound method
else:
callable = get(obj, cls)
if callable is self.func:
return self._make_unbound_method()
return partial(callable, *self.args, **self.keywords)class A(object): def add(self, x, y): return self, x + y
add10 = partialmethod(add, 10)
add10class = partialmethod(classmethod(add), 10)
add10static = partialmethod(staticmethod(add), 'self', 10)
return15 = partialmethod(add10, 5) # nested partialmethod
add2partial = partial(add, x=2)
return12 = partialmethod(add2partial, y=10) # partialmethod over partialdef test(): cls = A instance = cls()
assert instance.add10class(5) == (cls, 15)
assert cls.add10class(5) == (cls, 15)
assert instance.add10static(5) == ('self', 15)
assert cls.add10static(5) == ('self', 15)
assert instance.add10(5) == (instance, 15)
assert cls.add10(instance, 5) == (instance, 15)
assert instance.return15() == (instance, 15)
assert cls.return15(instance) == (instance, 15)
assert instance.return12() == (instance, 12)
assert cls.return12(instance) == (instance, 12)test()
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-10-27 12:32
On 27 Oct 2013 22:17, "alon horev" <report@bugs.python.org> wrote:
alon horev added the comment:
I think the following test demonstrates the API we're looking for.
- Am I missing any functionality?
The keyword arg handling is backwards for unbound methods (the call time kwds should override the preset ones).
Otherwise, looks good to me.
- How does partialmethod relate to @absolutemethod?
Do you mean @abstractmethod?
We need a isabstractmethod property implementation that delegates the question to the underlying descriptor.
See http://docs.python.org/dev/library/abc#abc.abstractmethod for details.
Author: alon horev (alonho) *
Date: 2013-10-29 00:10
Adding a patch with tests and documentation. Please feel free to comment on anything: my English, coding/testing style.
I'm pretty sure the documentation can be better but it turns out I speak better Python than English.
Two decisions I've made and unsure of:
- I didn't use @wraps or copied attributes from the wrapped function (doc, dict) to the partialmethod object. This is intentionally so partial and partialmethod would have similar semantics.
- I've implemented a repr although in all cases get returns a partial object or bound method. I consider it nice for debugging when looking at an object's dict.
Author: alon horev (alonho) *
Date: 2013-10-29 13:11
I've changed the test according to the code review. Thanks
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-10-31 13:54
Updated patch based on Alon's last patch.
The major functional change is to ensure self is set appropriately on any bound methods returned by the descriptor.
I also updated the docs and docstring, and added a What's New entry (as well as rewording the existing entry for functools.singledispatch)
There were a few other cosmetic changes, with the most noticeable being moving the partialmethod implementation and tests adjacent to the existing ones for functools.partial.
Assuming nobody pokes any significant holes in this idea and implementation in the meantime, I'll commit this before beta 1.
Author: Roundup Robot (python-dev) 
Date: 2013-11-03 06:43
New changeset 46d3c5539981 by Nick Coghlan in branch 'default': Issue #4331: Added functools.partialmethod http://hg.python.org/cpython/rev/46d3c5539981
Author: Vajrasky Kok (vajrasky) *
Date: 2013-11-04 04:27
Should we add partialmethod to all for consistency?
Author: Alyssa Coghlan (ncoghlan) *
Date: 2013-11-04 13:34
Indeed, added to all in http://hg.python.org/cpython/rev/ac1685661b07
History
Date
User
Action
Args
2022-04-11 14:56:41
admin
set
github: 48581
2013-11-04 13:34:43
ncoghlan
set
messages: +
2013-11-04 04:27:13
vajrasky
set
nosy: + vajrasky
messages: +
2013-11-03 06:43:39
ncoghlan
set
status: open -> closed
resolution: fixed
stage: commit review -> resolved
2013-11-03 06:43:01
python-dev
set
nosy: + python-dev
messages: +
2013-10-31 13:54:40
ncoghlan
set
files: + issue4331_partialmethod.diff
messages: +
2013-10-31 12:06:08
anacrolix
set
nosy: - anacrolix
2013-10-31 12:01:27
ncoghlan
set
stage: needs patch -> commit review
2013-10-29 13:11:29
alonho
set
files: + 4331.v2.patch
messages: +
2013-10-29 10:55:15
ncoghlan
set
assignee: ncoghlan
2013-10-29 00:10:27
alonho
set
files: + 4331.v1.patch
messages: +
2013-10-27 12:32:22
ncoghlan
set
messages: +
2013-10-27 12:17:03
alonho
set
messages: +
2013-10-26 17🔞38
ncoghlan
set
messages: +
2013-10-26 14:59:35
alonho
set
messages: +
2013-10-26 01:57:43
ncoghlan
set
messages: +
2013-10-25 19:27:55
alonho
set
nosy: + alonho
messages: +
2013-10-24 10:33:01
ncoghlan
set
messages: +
stage: needs patch
2013-06-05 10:00:50
anacrolix
set
messages: +
2013-06-05 07:42:14
ncoghlan
set
messages: +
2013-06-05 04:48:41
ncoghlan
set
title: Can't use _functools.partial() created function as method -> Add functools.partialmethod
nosy: + ncoghlan
messages: +
versions: - Python 2.7, Python 3.3
components: + Library (Lib)
2013-02-25 15:02:11
r.david.murray
set
messages: +
2013-02-25 14:19:14
eckhardt
set
messages: +
2013-02-08 09:20:10
Ramchandra Apte
set
versions: + Python 3.3, Python 3.4, - Python 3.0
2013-02-08 04:02:49
anacrolix
set
messages: +
2013-02-07 08:36:05
eckhardt
set
nosy: + eckhardt
messages: +
2012-03-24 18:23:40
anacrolix
set
files: + functools.partial-descrget.patch
messages: +
2012-03-24 16:14:44
anacrolix
set
nosy: + anacrolix
2011-12-11 01:28:07
jcea
set
nosy: + jcea
2010-02-23 21:37:15
jackdied
set
messages: +
2010-02-23 21:35:44
jackdied
set
messages: +
2010-02-22 19:11:42
r.david.murray
set
nosy: + r.david.murray
messages: +
2010-02-22 16:33:51
jackdied
set
messages: +
2010-02-18 21:09:22
jackdied
set
nosy: + jackdied
2010-02-01 19:09:23
Alexander.Belopolsky
set
messages: +
2010-01-29 00:25:25
Alexander.Belopolsky
set
nosy: + Alexander.Belopolsky
messages: +
2010-01-09 20:59:31
Christophe Simonis
set
files: + issue4331.patch
keywords: + patch
messages: +
2009-03-05 20:23:48
Christophe Simonis
set
nosy: + Christophe Simonis
2008-12-03 05:03:56
belopolsky
set
nosy: + belopolsky
2008-11-16 23:03:10
rhettinger
set
type: behavior -> enhancement
messages: +
nosy: + rhettinger
versions: + Python 3.0, Python 2.7, - Python 2.6, Python 2.5
2008-11-16 22:44:28
ironfroggy
set
nosy: + ironfroggy
messages: +
2008-11-16 20:24:31
ssadler
set
messages: +
2008-11-16 06:32:58
ssadler
create