[10] RFR 8193856 takeWhile produces incorrect result with elements produced by flatMap (original) (raw)

Paul Sandoz paul.sandoz at oracle.com
Wed Dec 20 23:03:51 UTC 2017

Hi Jonathan,

On 20 Dec 2017, at 14:47, Jonathan Bluett-Duncan <jbluettduncan at gmail.com> wrote:

Hi Paul, It seems that some clever Googler managed to find a workaround for aggressive flatMap operations in the form of a so-called MoreStreams.flatten operation, implemented in a side project called google/mug.

Thanks. MoreStreams.flatten “lowers” a stream of streams to a spliterator of spliterators which in turn is input as a new stream source, enabling a cancellation check + tryAdvance for any following short-circuit or terminal ops.

I have a solution in the works for flatMap based on sorted operation (which has to buffer all elements, sort ‘em, then push ‘em all downstream).

Paul.

I'm sharing the javadoc <https://google.github.io/mug/apidocs/com/google/mu/util/stream/MoreStreams.html#flatten-java.util.stream.Stream-> and GitHub project homepage <https://github.com/google/mug> with you and the rest of the mailing list in the hope that they prove to be useful.

Cheers, Jonathan On 20 December 2017 at 21:28, Paul Sandoz <paul.sandoz at oracle.com <mailto:paul.sandoz at oracle.com>> wrote: Hi, Please review this fix for a bug in the stream takeWhile operation: http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/> <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/>> The flatMap operation is currently aggressive and does not detect if a downstream operation may or has cancelled processing, and will push all of it’s elements downstream. Short-circuiting operations should be guarded against such behaviour but unfortunately takeWhile was not guarded. — Separately i plan to figure out a way to ensure flatMap operations become less aggressive if there are short-circuiting downstream operations. This may increase efficiency and also allow support for flat map results that are infinite. Paul.

More information about the core-libs-dev mailing list