Issue 13804: Python library structure creates hard to read code when using higher order functions (original) (raw)
Created on 2012-01-17 09:48 by dwt, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Messages (9)
Author: Martin Häcker (dwt)
Date: 2012-01-17 09:48
Code that uses higher order methods is often the clearest description of what you want to do. However since the higher order methods in python (filter, map, reduce) are free functions and aren't available on collection classes as methods, using them creates really hard to read code.
For example, if I want to get an attribute out of an array of objects and concatenate the results it's two steps, get the attributes out of the object, then concatenate the results.
Without higher order methods its like this. Uses three lines, intermediate variable is quite clear, but hard to compose and hard to abstract out parts of it.
self.questions = [] for topic in self.topics: self.questions.append(topic.questions)
if I want to do it with higher order functions there's really two ways, do it in one step with reduce: self.questions = reduce(lambda memo, topic: memo + topic.questions, self.topics, [])
Or use two steps with the operator module self.questions = reduce(operator.add, map(operator.attrgetter('questions'), self.topics), [])
Of these thee first still couples two steps into one lambda, while the second one decoples everything nicely but is a total train wreck to read, as you need to constantly jump back and forth in the line to understand what it does.
Having map and reduce defined on collections would not only make it drastically shorter but also allows you to read it from front to back in one go. (Ok, there is still the caveat that the assignment is in the front instead of at the end, but hey)
self.questions = self.topics.map(attrgetter('questions')).reduce(add, [])
That would be nicely separated into individual steps that exactly describe what each step is actually doing, is easy to abstract over and actually very succinct.
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * 
Date: 2012-01-17 14:01
Did you consider list comprehension?
self.questions = sum((topic.questions for topic in self.topics), [])
Author: Martin Häcker (dwt)
Date: 2012-01-17 14:40
Yes - however it has the same problem as the higher order version in the python libraries that to read it you need to constantly jump back and forth in the line.
Author: Éric Araujo (eric.araujo) *
Date: 2012-01-17 16:20
I think this discussion would be more useful on the python-ideas mailing list. The request (adding map to sequences) will probably be rejected*, but you have a good chance to get an explanation for this design choice by Guido van Rossum or one of the old-timer core devs.
*I think so because of str.join: people have said that they would prefer a list operation, but it’s always been denied because it would put the burden on any sequency type to implement the method, whereas with a str method then all iterables and iterators are supported for free.
Author: Daniel Stutzbach (stutzbach)
Date: 2012-01-17 19:42
If I'm understanding Martin Häcker's code correctly, the list comprehension equivalent is:
self.questions = [topic.questions for topic in self.topics]
The reduce() variants are not only much harder to read, but they will take O(n**2) operations because they create an O(n) list O(n) times.
Author: Martin Häcker (dwt)
Date: 2012-01-18 09:31
@stutzbach: I believe you got me wrong, as the example topic.questions is meant to return a list of questions that need concatenating - thus you can't save the second step.
Author: Daniel Stutzbach (stutzbach)
Date: 2012-01-18 17:54
Ah - in your first example (the one with 3 lines) did you mean to use .extend instead of .append?
Author: Martin Häcker (dwt)
Date: 2012-01-19 08:45
Jup - oh the joys of writing code in a bugtracker :)
Author: Terry J. Reedy (terry.reedy) *
Date: 2012-01-21 03:10
I am closing this because map has even less chance of being made a collection method than join. Unlike join, map takes any positive number of iterables as args, not just one. 'Iterables' includes iterators, which intentionally need have no methods other than iter and next.
If map were an attribute, it should be an attribute of 'function' (except that there is no one function class).
To abstract attributes, use get/setattr. Untested example:
def atcat(self, src, src_at, dst): res = [] for col in getattr(self, src): res += getattr(col, src_at) setattr(self, dst, res)
History
Date
User
Action
Args
2022-04-11 14:57:25
admin
set
github: 58012
2012-01-21 03:10:55
terry.reedy
set
status: open -> closed
versions: + Python 3.3
nosy: + terry.reedy
messages: +
resolution: rejected
2012-01-19 08:45:07
dwt
set
messages: +
2012-01-18 17:54:32
stutzbach
set
messages: +
2012-01-18 09:31:51
dwt
set
messages: +
2012-01-17 19:42:34
stutzbach
set
nosy: + stutzbach
messages: +
2012-01-17 16:20:07
eric.araujo
set
nosy: + eric.araujo
messages: +
2012-01-17 14:40:38
dwt
set
messages: +
2012-01-17 14:01:38
amaury.forgeotdarc
set
nosy: + amaury.forgeotdarc
messages: +
2012-01-17 09:48:36
dwt
create