CompletableFuture.exceptionally() method takes a lambda, but there is no flavor of the method that takes a custom Executor, or even an "...Async" flavor of it.
Which executor does exceptionally lambda run on? Would it be the same executor which ran the original CompletableFuture which threw the exception? Or (I would be surprised if this is the case) is it the commonPool ?
Form JDK bug discussion CompletableFuture.exceptionally may execute on main thread :
CompletableFuture.exceptionally does not take an Executor argument
since it is not designed to execute the exceptionally task
asynchronously.
If the dependent task is not yet completed then the exceptionally task
will complete on the same thread that dependent tasks completes on.
If the dependent task is completed then the exceptionally task will
complete on the thread that executed the call to exceptionally.
This is the same behaviour that will occur for any non-asynchronous
execution, such as thenAccept.
To guarantee that the exceptionally task is executed on a thread
within the executor thread pool then it is necessary to use
whenCompleteAsync or handleAsync and passing in the executor.
Note that as of JDK 12, there is CompletionStage.exceptionallyAsync (and exceptionallyAsync which takes an Executor).
Looks like it runs in the same executor as its CompletionStage
public static void main(String[] args) throws Exception {
//ExecutorService executorService = Executors.newFixedThreadPool(3);
ExecutorService executorService = Executors.newWorkStealingPool();
while (true) {
int i = Instant.now().getNano();
CompletableFuture<?> completableFuture = CompletableFuture.supplyAsync(
()-> {
System.err.printf("async thread at %d -> %s\n", i, Thread.currentThread().getName());
try {
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
e.printStackTrace();
}
throw new RuntimeException();
}, executorService);
completableFuture.exceptionally(
(err)-> {
System.err.printf("error thread for %d -> %s\n", i, Thread.currentThread().getName());
return null;
});
TimeUnit.SECONDS.sleep(5);
}
}
For fixed size:
async thread at 418000000 -> pool-1-thread-1
error thread for 418000000 -> pool-1-thread-1
async thread at 646000000 -> pool-1-thread-2
error thread for 646000000 -> pool-1-thread-2
async thread at 646000000 -> pool-1-thread-3
error thread for 646000000 -> pool-1-thread-3
async thread at 646000000 -> pool-1-thread-1
error thread for 646000000 -> pool-1-thread-1
async thread at 647000000 -> pool-1-thread-2
error thread for 647000000 -> pool-1-thread-2
For stealing pool(4 cores):
async thread at 96000000 -> ForkJoinPool-1-worker-1
error thread for 96000000 -> ForkJoinPool-1-worker-1
async thread at 196000000 -> ForkJoinPool-1-worker-1
error thread for 196000000 -> ForkJoinPool-1-worker-1
async thread at 197000000 -> ForkJoinPool-1-worker-1
error thread for 197000000 -> ForkJoinPool-1-worker-1
async thread at 197000000 -> ForkJoinPool-1-worker-1
error thread for 197000000 -> ForkJoinPool-1-worker-1
async thread at 197000000 -> ForkJoinPool-1-worker-1
error thread for 197000000 -> ForkJoinPool-1-worker-1
async thread at 197000000 -> ForkJoinPool-1-worker-1
with no executor:
async thread at 848000000 -> ForkJoinPool.commonPool-worker-1
error thread for 848000000 -> ForkJoinPool.commonPool-worker-1
async thread at 944000000 -> ForkJoinPool.commonPool-worker-1
error thread for 944000000 -> ForkJoinPool.commonPool-worker-1
async thread at 944000000 -> ForkJoinPool.commonPool-worker-1
error thread for 944000000 -> ForkJoinPool.commonPool-worker-1
async thread at 944000000 -> ForkJoinPool.commonPool-worker-1
error thread for 944000000 -> ForkJoinPool.commonPool-worker-1
async thread at 944000000 -> ForkJoinPool.commonPool-worker-1
error thread for 944000000 -> ForkJoinPool.commonPool-worker-1
I did some experiments. Looks like the executor for the exceptionally handler is not chosen deterministically.
If I do not place any breakpoints, the exceptionally handler is occasionally executed in the same executor as the CompletableFuture that has the exceptionally handler. Furthermore, I could notice that it ran in the same ForkJoinTask. Once in two or three runs, it was executed on the main thread.
If I placed breakpoint at runtime prior to attaching the exceptionally handler and waited there for a moment, the exceptionally lambda was consistently executed on the main (calling) thread!
Here is the experiment code:
ForkJoinPool ex = new ForkJoinPool(2,
ForkJoinPool.defaultForkJoinWorkerThreadFactory, null, false);
// AsyncThreadLocal.requestId.set((int)(Math.random()*1000));
CompletableFuture<Integer> f1 = CompletableFuture.supplyAsync(() -> {
System.out.println(Thread.currentThread().getName());
throw new RuntimeException();
// return 3;
}, ex);
CompletableFuture<Integer> f8 = f1.exceptionally(t -> {
System.out.println(Thread.currentThread().getName());
return 5;
});
Thread.sleep(10000);
Output of program: Sometimes it is
ForkJoinPool-1-worker-1
main
Other times it is
ForkJoinPool-1-worker-1
ForkJoinPool-1-worker-1
Related
I found this example compile successfully, but isn't Rc<> thread unsafe? What if tikio start multi thread to execute test fn, won't it cause Rc to run on different threads?
use std::rc::Rc;
use tokio::join;
use std::time;
use async_std::task::{sleep};
async fn test(t:Rc<String>){
let k = t;
println!("1. test,{:?}", k);
sleep(time::Duration::from_secs(1)).await;
println!("2. test,{:?}", k);
}
#[tokio::main]
async fn main() {
let r = Rc::new("abc".to_string());
let f1 = test(r.clone());
let f2 = test(r.clone());
join!(f1,f2);
}
This is a bit nuanced, but #[tokio::main] will rewrite your async main() into something like this:
fn main() {
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.unwrap()
.block_on(async {
// your code from `async main()`
})
}
The last step that actually runs everything, .block_on(), does not require the given task to be thread-safe (a.k.a. implement Send) because it will not be ran on a separate thread, even when using the multi-threaded executor:
This runs the given future on the current thread, blocking until it is complete, and yielding its resolved result.
Other tokio utilities and any spawned tasks may be ran on separate threads, and even move between them, but not the initial task.
I am using async programming in one of my project. In one of the method where the return type is CompletionStage, for testing I am simulating an exception throw which does not get caught in the exceptionally(). Below are some Test code i created based on the use case.
public class TestMain {
TestService service = new TestService();
public static void main(String[] args) throws InterruptedException {
TestMain t = new TestMain();
System.out.println("STARTED: THREAD " + Thread.currentThread().getName());
t.execute("test").toCompletableFuture().join();
System.out.println("DONE: THREAD " + Thread.currentThread().getName());
}
public CompletionStage<Void> execute(String s) {
System.out.println("EXECUTE: THREAD " + Thread.currentThread().getName());
return CompletableFuture.runAsync(() -> {
System.out.println("EXECUTE-ASYNC STARTED: THREAD " + Thread.currentThread().getName());
process(s);}).exceptionally(ex->{
System.out.println("EXECUTE-ASYNC EXCEPTIONAL: THREAD " + Thread.currentThread().getName());
System.err.println("EXECUTE-ASYNC EXCEPTIONAL-EXCEPTION " + ex.toString());
if (ex.getCause()!=null)
System.err.println("EXECUTE-ASYNC EXCEPTIONAL-EXCEPTION CAUSE " + ex.getCause().toString());
throw ex instanceof CompletionException ? (CompletionException) ex : new CompletionException(ex);
});
}
private CompletionStage<Void> process(String s) {
System.out.println("EXECUTE-PROCESS: THREAD " + Thread.currentThread().getName());
// throw new RuntimeException();
return service.service(s);
}
}
public class TestService {
public CompletionStage<Void> service(String s) {
System.out.println("SERVICE: THREAD " + Thread.currentThread().getName());
// throw new RuntimeException();
return preProcess(s).thenCompose(v -> {
System.out.println("SERVICE-PRE PROCESS -> COMPOSE: THREAD " + Thread.currentThread().getName());
return process(s);
});
}
private CompletionStage<Void> preProcess(String s) {
System.out.println("SERVICE-PRE PROCESS: THREAD " + Thread.currentThread().getName());
// throw new RuntimeException();
return CompletableFuture.completedFuture(null);
}
private CompletionStage<Void> process(String s) {
System.out.println("SERVICE-PROCESS: THREAD " + Thread.currentThread().getName());
throw new RuntimeException();
// return CompletableFuture.completedFuture(null);
}
}
When i run the code given above the output shows that the exceptionally() in TestMain.execute() is not called.
Actual Output (when exception thrown from TestService.process()):
STARTED: THREAD main
EXECUTE: THREAD main
EXECUTE-ASYNC STARTED: THREAD ForkJoinPool.commonPool-worker-1
EXECUTE-PROCESS: THREAD ForkJoinPool.commonPool-worker-1
SERVICE: THREAD ForkJoinPool.commonPool-worker-1
SERVICE-PRE PROCESS: THREAD ForkJoinPool.commonPool-worker-1
SERVICE-PRE PROCESS -> COMPOSE: THREAD ForkJoinPool.commonPool-worker-1
SERVICE-PROCESS: THREAD ForkJoinPool.commonPool-worker-1
DONE: THREAD main
However if I throw the exception at any other place, which are marked by the commented code, the exceptionally() is called. Below is an example of one such instance.
Actual Output (when exception thrown from TestService.preProcess()):
Was expecting above output to be similar to this.
STARTED: THREAD main
EXECUTE: THREAD main
EXECUTE-ASYNC STARTED: THREAD ForkJoinPool.commonPool-worker-1
EXECUTE-PROCESS: THREAD ForkJoinPool.commonPool-worker-1
SERVICE: THREAD ForkJoinPool.commonPool-worker-1
SERVICE-PRE PROCESS: THREAD ForkJoinPool.commonPool-worker-1
EXECUTE-ASYNC EXCEPTIONAL: THREAD main
EXECUTE-ASYNC EXCEPTIONAL-EXCEPTION java.util.concurrent.CompletionException: java.lang.RuntimeException
EXECUTE-ASYNC EXCEPTIONAL-EXCEPTION CAUSE java.lang.RuntimeException
Exception in thread "main" java.util.concurrent.CompletionException: java.lang.RuntimeException
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1643)
at java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1632)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:172)
Caused by: java.lang.RuntimeException
at in.TestService.preProcess(TestService.java:19)
at in.TestService.service(TestService.java:11)
at in.TestMain.process(TestMain.java:34)
at in.TestMain.lambda$0(TestMain.java:22)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
... 5 more
Can somebody help out as to why this is happening? I need the exception thrown to be caught in the TestMain.execute().exceptionally() always to implement a retry logic that I want to use here.
Thanks in advance.
You are invoking a method which returns a CompletionStage but ignoring the returned value:
CompletableFuture.runAsync(() -> {
System.out.println("EXECUTE-ASYNC STARTED: THREAD "+Thread.currentThread().getName());
process(s); // return value ignored
})
The stage created by CompletableFuture.runAsync(…) would be completed exceptionally if the function/Runnable threw an exception. Which is what happens when you place throw new RuntimeException(); inside the process method. But in the other scenario it doesn’t throw an exception but returns a failed stage which is ignored by the caller.
You could change the code to
CompletableFuture.runAsync(() -> {
System.out.println("EXECUTE-ASYNC STARTED: THREAD "+Thread.currentThread().getName());
process(s).toCompletableFuture().join();
})
to not ignore the returned stage or, preferably, use the pattern you’ve already used at another place:
CompletableFuture.completedFuture(null).thenComposeAsync(v -> {
System.out.println("EXECUTE-ASYNC STARTED: THREAD "+Thread.currentThread().getName());
return process(s);
})
The code sample below
public static void Test()
{
TaskCompletionSource<bool> inner = new TaskCompletionSource<bool>();
TaskCompletionSource<bool> outer = new TaskCompletionSource<bool>();
TaskScheduler ctScheduler = new CurrentThreadTaskScheduler();
outer.Task.ContinueWith(
completedTask =>
{
Console.WriteLine("Continuation of the outer in the thread #{0}", Thread.CurrentThread.ManagedThreadId);
inner.SetResult(true);
},
CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.Default);
Task t = Task.Run(
async () =>
{
await inner.Task;
Console.WriteLine("Awaiter continuation in the thread #{0}", Thread.CurrentThread.ManagedThreadId);
});
Thread.Sleep(1000);
Console.WriteLine("Setting the outer to completed in the thread #{0}", Thread.CurrentThread.ManagedThreadId);
outer.SetResult(true);
}
produces the following output
Setting the outer to completed in the thread #1
Continuation of the outer in the thread #4
Awaiter continuation in the thread #4
As expected, the continuation after await inner.Task was executed synchronously on the thread which completed the task, namely the thread #4.
When I'm trying to run all the continuations synchronously on the same current thread
outer.Task.ContinueWith(
completedTask =>
{
Console.WriteLine("Continuation of the outer in the thread #{0}", Thread.CurrentThread.ManagedThreadId);
inner.SetResult(true);
},
CancellationToken.None,
TaskContinuationOptions.None,
ctScheduler);
using a simple custom task scheduler, implemented as follows
public sealed class CurrentThreadTaskScheduler : TaskScheduler
{
protected override IEnumerable<Task> GetScheduledTasks()
{
return Enumerable.Empty<Task>();
}
protected override void QueueTask(Task task)
{
this.TryExecuteTask(task);
}
protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
return this.TryExecuteTask(task);
}
}
the 'awaiter' continuation runs asynchronously, as seen in the output down here
Setting the outer to completed in the thread #1
Continuation of the outer in the thread #1
Awaiter continuation in the thread #4
Why the continuation in question is running asynchronously and how should I implement the scheduler to guarantee the expected behaviour?
We say that, if an awaited expression is not complete then an async method
pauses and returns to the caller.
Once the awaited expression is complete, it resumes it execution.
On which context it resumes is dictated by ConfigureAwait.
But what happens in between pausing and returning to the caller and resuming after the awaited expression is completed.
Where does the awaited expression execute mean while ?
On a Thread Pool thread or UI's thread.
private async void Button_Click(object sender, RoutedEventArgs e)
{
// will resume on UI's context.
button.Content = await GetString().ConfigureAwait(true);
}
private async Task<string> GetString()
{
// where does this section of code runs ?
return await Task.Delay(3000).ContinueWith(x => "Async Content");
}
Neither thread pool nor UI thread. It does not execute at all. When Task.Delay is called, the Delay method creates a Timer and both GetString and Button_Click return.
After 3000 milliseconds the Timer callback gets executed on some thread pool thread. It completes the task and schedules task continuations, and the rest of code in GetString and Button_Click is executed on respective threads.
I'm trying to get a responsive JavaFX graphical interface while executing a cmd command.
The command I'm executing is the following.
youtube-dl.exe --audio-format mp3 --extract-audio https://www.youtube.com/watch?v=l2vy6pJSo9c
As you see this is a youtube-downloader that converts a youtube link to an mp3-file.
I want this to be executed in a second thread and not in the main FX thread.
I've solved this by implementing interface Callable in the class StartDownloadingThread.
#Override
public Process call() throws Exception {
Process p = null;
p = ExecuteCommand(localCPara1, localCPara2, localDirectory).start();
try {
Thread.sleep(30);
}catch (InterruptedException e){}
return p;
}
The method ExecuteCommand just returns a ProcessBuilder object.
I try to use Thread.sleep to make the program return to the main thread and thus making the application responsive. Unfortunately the program still freezes.
This is how the method call is called.
ExecutorService pool = Executors.newFixedThreadPool(2);
StartDownloadingThread callable = new StartDownloadingThread(parameter1, parameter2, directory);
Future future = pool.submit(callable);
Process p = (Process) future.get();
p.waitFor();
How do I make my GUI responsive using the interface Callable?
Using a executor to run a task just for you to use the get method of the Future that is returned when submitting the task does not actually free the original thread to continue with other tasks. Later you even use the waitFor method on the original thread, which is likely to take even more time than anything you do in your Callable.
For this purpose the Task class may be better suited, since it allows you to handle success/failure on the application thread using event handlers.
Also please make sure an ExecutorService is shut down after you're done submitting tasks.
Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
Process p = null;
p = ExecuteCommand(localCPara1, localCPara2, localDirectory).start();
// why are you even doing this?
try {
Thread.sleep(30);
}catch (InterruptedException e){}
// do the rest of the long running things
p.waitFor();
return null;
}
};
task.setOnSucceeded(event -> {
// modify ui to show success
});
task.setOnFailed(event -> {
// modify ui to show failure
});
ExecutorService pool = Executors.newFixedThreadPool(2);
pool.submit(task);
// add more tasks...
// shutdown the pool not keep the jvm alive because of the pool
pool.shutdown();