Comparator in collector in stream causes issues with type inference?

Rick :

I have the following simplified example that groups a List of Strings into categories, in the form of a TreeMap from Integer to List

public static void main(String[] args)
{
    List<String> list = Arrays.asList("A", "B", "C", "D", "E");

    TreeMap<Integer, List<String>> res = list.stream()
        .collect(Collectors.groupingBy(
            s -> s.charAt(0) % 3,
            () -> new TreeMap<>(Comparator.<Integer>reverseOrder()), // Type required
            Collectors.toList()
        ));

    System.out.println(res);
}

If I don't specify the type of the Comparator.reverseOrder() the code will fail to compile (see bottom of the post for the error).

If I explicitly specify the type of the TreeMap instead of the type of Comparator.reverseOrder() the code works fine.

() -> new TreeMap<Integer, List<String>>(Comparator.reverseOrder()), // Type required

So:

The compiler is able to infer the type of the TreeMap
The compiler is able to infer the type of the Comparator if it knows the type of the TreeMap
But the compiler is unable to figure out the type of the Comparator if it has to infer the type of the TreeMap.

I don't understand why the compiler can't infer both types. I've tested this with both JDK 1.8.0_191 from Oracle and JDK 11.0.1_13 from AdoptOpenJDK, with the same results.

Is this some limitation I'm not aware of?

Error:(22, 32) java: no suitable method found for groupingBy((s)->s.cha[...]) % 3,()->new Tr[...]er()),java.util.stream.Collector<java.lang.Object,capture#1 of ?,java.util.List<java.lang.Object>>)
    method java.util.stream.Collectors.<T,K>groupingBy(java.util.function.Function<? super T,? extends K>) is not applicable
      (cannot infer type-variable(s) T,K
        (actual and formal argument lists differ in length))
    method java.util.stream.Collectors.<T,K,A,D>groupingBy(java.util.function.Function<? super T,? extends K>,java.util.stream.Collector<? super T,A,D>) is not applicable
      (cannot infer type-variable(s) T,K,A,D
        (actual and formal argument lists differ in length))
    method java.util.stream.Collectors.<T,K,D,A,M>groupingBy(java.util.function.Function<? super T,? extends K>,java.util.function.Supplier<M>,java.util.stream.Collector<? super T,A,D>) is not applicable
      (inferred type does not conform to upper bound(s)
        inferred: java.lang.Object
        upper bound(s): java.lang.Comparable<? super T>,T,java.lang.Object)

Holger :

Unfortunately, the type inference has a really complex specification, which makes it very hard to decide whether a particular odd behavior is conforming to the specification or just a compiler bug.

There are two well-known deliberate limitations to the type inference.

First, the target type of an expression is not used for receiver expressions, i.e. in a chain of method invocations. So when you have a statement of the form

TargetType x = first.second(…).third(…);

the TargetType will be use to infer the generic type of the third() invocation and its argument expressions, but not for second(…) invocation. So the type inference for second(…) can only use the stand-alone type of first and the argument expressions.

This is not an issue here. Since the stand-alone type of list is well defined as List<String>, there is no problem in inferring the result type Stream<String> for the stream() call and the problematic collect call is the last method invocation of the chain, which can use the target type TreeMap<Integer, List<String>> to infer the type arguments.

The second limitation is about overload resolution. The language designers made a deliberate cut when it comes to a circular dependency between incomplete types of argument expressions which need to know the actual target method and its type, before they could help determine the right method to invoke.

This also doesn’t apply here. While groupingBy is overloaded, these methods differ in the number of parameters, which allows to select the only appropriate method without knowing the argument types. It can also be shown that the compiler’s behavior doesn’t change when we replace groupingBy with a different method which has the intended signature but no overloads.

Your issue can be solved by using, e.g.

TreeMap<Integer, List<String>> res = list.stream()
    .collect(Collectors.groupingBy(
        (String s) -> s.charAt(0) % 3,
        () -> new TreeMap<>(Comparator.reverseOrder()),
        Collectors.toList()
    ));

This uses an explicitly typed lambda expression for the grouping function, which, while not actually contributes to the types of the map’s key, causes the compiler to find the actual types.

While the use of explicitly typed lambda expressions instead of implicitly typed ones can make a difference on method overload resolution, as said above, it shouldn’t apply here, as this specific scenario is not an issue of overloaded methods.

Weirdly enough, even the following change makes the compiler error go away:

static <X> X dummy(X x) { return x; }
…

TreeMap<Integer, List<String>> res = list.stream()
    .collect(Collectors.groupingBy(
        s -> s.charAt(0) % 3,
        dummy(() -> new TreeMap<>(Comparator.reverseOrder())),
        Collectors.toList()
    ));

Here, we’re not helping with any additional explicit type and also not changing the formal nature of the lambda expressions, but still, the compiler suddenly infers all types correctly.

The behavior seems to be connected to the fact that zero parameter lambda expressions are always explicitly typed. Since we can’t change the nature of a zero parameter lambda expression, I created the followed alternative collector method for verification:

public static <T, K, D, A, M extends Map<K, D>>
Collector<T, ?, M> groupingBy(Function<? super T, ? extends K> classifier,
                              Function<Void,M> mapFactory,                                 
                              Collector<? super T, A, D> downstream) {
    return Collectors.groupingBy(classifier, () -> mapFactory.apply(null), downstream);
}

Then, using an implicitly typed lambda expression as map factory compiles without problems:

TreeMap<Integer, List<String>> res = list.stream()
    .collect(groupingBy(
        s -> s.charAt(0) % 3,
        x -> new TreeMap<>(Comparator.reverseOrder()),
        Collectors.toList()
    ));

whereas using an explicitly typed lambda expression causes a compiler error:

TreeMap<Integer, List<String>> res = list.stream()
    .collect(groupingBy(                           // compiler error
        s -> s.charAt(0) % 3,
        (Void x) -> new TreeMap<>(Comparator.reverseOrder()),
        Collectors.toList()
    ));

In my opinion, even if the specification backs up this behavior, it should get corrected, as the implication of providing explicit types should never be that the type inference gets worse than without. That’s especially true for zero argument lambda expressions which we can’t turn into implicitly typed ones.

It also doesn’t explain why turning all arguments into explicitly typed lambda expressions will also eliminate the compiler error.

Comparator in collector in stream causes issues with type inference?

Guess you like