I'm doing some multi-threading work in java and I have a question about the Future get method.
I have an ExecutorSevice which submits n Callable. The submit returns a Future which is added to a list (later on some code). The Callable's call method loops through a list, and for every item of the list someMethod is called and returns a CompletableFuture<T>. Then, the Completable future is added to a map.
At the end of the loop call returns the map containing a bunch of CompletableFuture.
So, the initial list (uglyList) contains n maps of <String, CompletableFuture<String>>.
My question is, when I call get on a element of uglyList the execution is blocked until the entire list (passed as parameter to the Callable) is visited and all CompletableFutures have been inserted in the map, right?
Because, I have a doubt, is it possible that the get in this example, also waits the completion of the CompletableFutures in the map? I don't know how to test this
public class A {
private class CallableC implements Callable<Map<String, CompletableFuture<String>>> {
private List<String> listString;
Map<String, CompletableFuture<String>> futureMap;
public CallableC(List<String> listString) throws Exception {
this.listString = listString;
this.futureMap = new HashMap<>();
}
@Override
public Map<String, CompletableFuture<String>> call() throws Exception {
for (int j = 0; j < listString.size(); ++j) {
CompletableFuture<String> future = someMethod();
futureMap.put(listString.get(i), future);
}
return futureMap;
}
}
public void multiThreadMethod(List<String> items) throws Exception {
List<Future<Map<String, CompletableFuture<String>>>> uglyList = new ArrayList<>();
int coresNumber = (Runtime.getRuntime().availableProcessors());
// This method split a List in n subList
List<List<String>> splittedList = split(items, coresNumber);
ExecutorService executor = Executors.newFixedThreadPool(coresNumber);
try {
for (int i = 0; i < coresNumber; ++i) {
threads.add(executor.submit(new CallableC(splittedList.get(i),)));
}
for (int i = 0; i < coresNumber; ++i) {
uglyList.get(i).get(); //here
}
} catch (Exception e) {
// YAY errors
}
finally {
executor.shutdown();
}
}
}
when i call get on a element of uglyList the execution is blocked until the entire list (passed as parameter to the Callable) is visited and all CompletableFutures have been inserted in the map, right?
Yes that is correct. The get() will wait for the completion of the Callable, which in your case means the end of the CallableC#call() method, which will be reached at the end of the for loop, when the CompletableFutures are all created and inserted into futureMap, regardless of the status / completion of those CompletableFutures.
it is possible that the get in this example, also waits the completion of the CompletableFutures in the map?
It is possible to achieve this, but not without additionnal code.
The minimal way to do this in your code would be to add a join call on the CompletableFutures inside the tasks.
@Override
public Map<String, CompletableFuture<String>> call() throws Exception {
for (int j = 0; j < listString.size(); ++j) {
CompletableFuture<String> future = someMethod();
futureMap.put(listString.get(j), future);
}
// This will wait for all the futures inside the map to terminate
try {
CompletableFuture.allOf(futureMap.values().toArray(new CompletableFuture[0])).join();
} catch (CompletionException e) {
// ignored - the status will be checked later on
}
return futureMap;
}
Another way would be to wait at the very end (I skip exception handling for brevity) :
for (int i = 0; i < coresNumber; ++i) {
threads.add(executor.submit(new CallableC(splittedList.get(i),)));
}
// This will wait for all the futures to terminate (thanks to f.get())
// And then, all the completable futures will be added to an array of futures that will also be waited upon
CompletableFuture.allOf(
uglyList.stream().flatMap(f -> f.get().values().stream()).toArray(CompletableFuture[]::new)
).join();
But if possible for you, I'd simplify the whole code by using only completable futures, unless you have a very good reason to break tasks the way you do (using a list of list, a dedicated executor, etc...). This could look like this (maybe a few typos)
List<String> items = ...
ConcurrentMap<String, String> resultsByItem = new ConcurrentHashMap<>();
Future<Void> allWork = CompletableFuture.allOf(
items.stream()
.map(item -> someWork(item) // Get a completable future for this item
.andThen(result -> resultsByItem.put(item, result)) // Append the result to the map
).toArray(CompletableFuture[]::new)
);
allWork.join();
// Your futureMap is completed.
This would replace your whole code in the question.