parallelStream() methods (original) (raw)

Doug Lea dl at cs.oswego.edu
Sat Feb 9 08:49:17 PST 2013

On 02/09/13 11:41, Brian Goetz wrote:

You can't drop a bomb like that and walk away! You have to explain why you don't like it, because I suspect most people's first guess about why will be wrong.

I'll take my best stab at explaining why:

Yes, thanks. Stateful Stream methods are clearly problematic. Most people like them anyway because they are convenient. And in any case, whenever they show up, many API discussions follow.

because it (like the stateful methods (sort, distinct, limit)) which you also don't like, move us incrementally farther from being able to express stream pipelines in terms of traditional data-parallel constructs, which further constrains our ability to to map them directly to tomorrow's computing substrate, whether that be vector processors, FPGAs, GPUs, or whatever we cook up.

Filter-map-reduce map very cleanly to all sorts of parallel computing substrates; filter-parallel-map-sequential-sorted-limit-parallel-map-uniq-reduce does not. So the whole API design here embodies many tensions between making it easy to express things the user is likely to want to express, and doing is in a manner that we can predictably make fast with transparent cost models.

More information about the lambda-libs-spec-observers mailing list