AsyncCompleter encapsulates the use of a CompletionService such that a group of jobs can be submitted for asynchronous execution and then the results obtained in the order they arrive.

Usage

Executor executor = getExecutor(…);
AsyncCompleter completer = new AsyncCompleter.Builder(executor)
  .ignoreExceptions()
  .limitParallelExecutionTo(8)
  .build();

Iterable<Callable<T>> jobs = getJobs(…);
Iterable<T> results = completer.invokeAll(jobs);
… // do some other work
for(T result : results) { // this blocks and waits for the first available result
  … // do something with the result
}

There are options to throttle the submission so you don't flood (in this example only 8 concurrent submissions will be made to the executor at once) and to ignore or throw exceptions (ignored ones will be filtered out so it is possible to see less results than jobs).

Internally, on each submission, an AsyncCompletionFunction is created that contains a new CompletionService, this acts to transform the Jobs to a result supplier. We submit the jobs to it and copy the copying the result which enacts the submission. We add a further transform to check for exceptions according to the specified policy and filter out null results. The resulting Iterable<T> is then ready to start delivering your results in the order they arrive.

src/main/java/com/atlassian/util/concurrent/AsyncCompleter.java