Related
I want to run a computation that might take too long. If it doesn't finish in a specific time, abort the computation and return a different value. For this problem I found Future.timeout, it would almost do the same thing that I want, except it doesn't work for this code.
Future<String> timeoutTest() async
{
return await longComputation().timeout(
Duration(milliseconds: 10),
onTimeout: () => "Took too long"
);
}
Future<String> longComputation() async
{
int startTime = DateTime.now().millisecondsSinceEpoch;
Rational n = Rational.one;
for(int i = 1; i < 2000; i++)
{
n *= Rational.fromInt(i);
}
String result = n.toDecimalString();
print("Time took: ${DateTime.now().millisecondsSinceEpoch - startTime} ms");
return result;
}
When I call print(await timeoutTest()) I either expect a string of digits that took maximum 10ms to calculate OR the "Took too long" string if it took more than 10ms. But I get the string of digits, and the message in the console: "Time took: 877 ms". So the timeout didn't work.
If I fake the computation with Future.delay, the timeout works as expected. It returns a different value, because the longComputation took at least 100ms. (I still get the message in the console: "Time took: 103ms", but this is not the main problem.)
Future<String> longComputation() async
{
int startTime = DateTime.now().millisecondsSinceEpoch;
String result = await Future.delayed(
Duration(milliseconds: 100),
() => "Fake computation result"
);
print("Time took: ${DateTime.now().millisecondsSinceEpoch - startTime} ms");
return result;
}
I'm assuming I messed up something in the longComputation, but what? There were no un-awaited Futures.
This behavior can be confusing but it is important to remember that your Dart code are only executed in a single thread (unless you are using isolates).
The problem is that the logic behind .timeout needs to run in the same single thread and Dart can't just stop the execution of your own code. So if you are running a CPU intensive calculation without any pauses you are stopping the Dart VM from running any other events on the event queue.
What the implementation of .timeout actually does, is creating an internal Timer which are going to be triggered in the future unless you get a result before the timeout value. This Timer event are going on top on the event queue like any other event in the Dart VM.
But in your first example, we are actually never going to execute any other event on the event queue before you are giving the result. So from the Future's point of view, you are returning a result before the deadline.
This is going to look like .timeout is kind of pointless but what it really are for is when you are making some IO operations like API requests where the Dart VM are actually waiting for some answer.
If you are going to use it for heavy calculations, you can either spawn an Isolate so your main isolate instance can wait on the other isolate. Alternative, you can insert some pauses in your calculation which makes space for the Dart VM to execute other events on the event queue. An example could be inserting await Future<void>(() => null); which are going to spawn a new event on top of the event queue. When we wait for all events on the queue to be executed before our own empty calculation.
It would then also make sense to add some logic so your own code can see if the timeout value has been reached so you can stop the calculation if that is the case.
You can't actually cancel Futures. At best you could have your .timeout callback set a flag that longComputation periodically checks to become a no-op as soon as possible.
Also see Future.timeout documention is misleading.
I have an async method that uses tokio::fs to explore a directory:
use failure::Error;
use futures::Future;
use std::path::PathBuf;
use tokio::prelude::*;
fn visit_async(path: PathBuf) -> Box<Future<Item = (), Error = Error> + Send> {
let task = tokio::fs::read_dir(path)
.flatten_stream()
.for_each(move |entry| {
let path = entry.path();
if path.is_dir() {
let task = visit_async(entry.path());
tokio::spawn(task.map_err(drop));
} else {
println!("File: {:?}", path);
}
future::ok(())
})
.map_err(Error::from);
Box::new(task)
}
I need to execute another future after all the the future returned by this method ends as well as all the tasks spawned by it. Is there a better way that just starting another runtime?
let t = visit_async(PathBuf::from(".")).map_err(drop);
tokio::run(t);
tokio::run(future::ok(()));
I'd strive to avoid using tokio::spawn() here, and try to wrap it all into a single future (in general, I think you only do tokio::spawn when you don't care about the result or execution, which we do here). That should make it easy to wait for completion. I haven't tested this, but something along these lines might do the trick:
let task = tokio::fs::read_dir(path)
.flatten_stream()
.for_each(move |entry| {
let path = entry.path();
if path.is_dir() {
let task = visit_async(entry.path());
future::Either::A(task)
} else {
println!("File: {:?}", path);
future::Either::B(future::ok(()))
}
})
.map_err(Error::from)
.and_then(|_| {
// Do some work once all tasks complete
});
Box::new(task)
This will cause the asynchronous tasks to execute in sequence. You could use and_then instead of for_each to execute them in parallel and then into_future().and_then(|_| { ... }) to tuck on some action to execute afterwards.
There's another issue with parallel descent in the FS: you may run out of file descriptors.
There is a way to solve both issues by creating a tokio::sync::Semaphore to limit the concurrent number of these tasks. After you are done spawning all of them, you can use Semaphore::acquire_many with the same value you used at creation, to block until all other tasks are finished.
For correctness, you should acquire the semaphore before spawning the task, and then pass the SemaphorePermit to the task (and make sure it doesn't get dropped before you are done). If you acquire the semaphore inside the tasks, there is a risk the main task might acquire all the permits before the first sub-task has a chance to run.
Since you can only move a SemaphorePermit<'static> inside the task, you will need to have a &'static Semaphore, for instance using lazy_static! or Box::leak.
In the kotlinx.coroutines library you can start new coroutine using either launch (with join) or async (with await). What is the difference between them?
launch is used to fire and forget coroutine. It is like starting a new thread. If the code inside the launch terminates with exception, then it is treated like uncaught exception in a thread -- usually printed to stderr in backend JVM applications and crashes Android applications. join is used to wait for completion of the launched coroutine and it does not propagate its exception. However, a crashed child coroutine cancels its parent with the corresponding exception, too.
async is used to start a coroutine that computes some result. The result is represented by an instance of Deferred and you must use await on it. An uncaught exception inside the async code is stored inside the resulting Deferred and is not delivered anywhere else, it will get silently dropped unless processed. You MUST NOT forget about the coroutine you’ve started with async.
I find this guide to be useful. I will quote the essential parts.
🦄 Coroutines
Essentially, coroutines are light-weight threads.
So you can think of a coroutine as something that manages thread in a very efficient way.
🐤 launch
fun main(args: Array<String>) {
launch { // launch new coroutine in background and continue
delay(1000L) // non-blocking delay for 1 second (default time unit is ms)
println("World!") // print after delay
}
println("Hello,") // main thread continues while coroutine is delayed
Thread.sleep(2000L) // block main thread for 2 seconds to keep JVM alive
}
So launch starts a coroutine, does something, and returns a token immediately as Job. You can call join on this Job to block until this launch coroutine completes.
fun main(args: Array<String>) = runBlocking<Unit> {
val job = launch { // launch new coroutine and keep a reference to its Job
delay(1000L)
println("World!")
}
println("Hello,")
job.join() // wait until child coroutine completes
}
🦆 async
Conceptually, async is just like launch. It starts a separate coroutine which is a light-weight thread that works concurrently with all the other coroutines. The difference is that launch returns a Job and does not carry any resulting value, while async returns a Deferred -- a light-weight non-blocking future that represents a promise to provide a result later.
So async starts a background thread, does something, and returns a token immediately as Deferred.
fun main(args: Array<String>) = runBlocking<Unit> {
val time = measureTimeMillis {
val one = async { doSomethingUsefulOne() }
val two = async { doSomethingUsefulTwo() }
println("The answer is ${one.await() + two.await()}")
}
println("Completed in $time ms")
}
You can use .await() on a deferred value to get its eventual result, but Deferred is also a Job, so you can cancel it if needed.
So Deferred is actually a Job. Read this for more details.
interface Deferred<out T> : Job (source)
🦋 async is eager by default
There is a laziness option to async using an optional start parameter with a value of CoroutineStart.LAZY. It starts coroutine only when its result is needed by some await or if a start function is invoked.
launch and async are used to start new coroutines. But, they execute them in different manner.
I would like to show very basic example which will help you understand difference very easily
launch
class MainActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
btnCount.setOnClickListener {
pgBar.visibility = View.VISIBLE
CoroutineScope(Dispatchers.Main).launch {
val currentMillis = System.currentTimeMillis()
val retVal1 = downloadTask1()
val retVal2 = downloadTask2()
val retVal3 = downloadTask3()
Toast.makeText(applicationContext, "All tasks downloaded! ${retVal1}, ${retVal2}, ${retVal3} in ${(System.currentTimeMillis() - currentMillis)/1000} seconds", Toast.LENGTH_LONG).show();
pgBar.visibility = View.GONE
}
}
// Task 1 will take 5 seconds to complete download
private suspend fun downloadTask1() : String {
kotlinx.coroutines.delay(5000);
return "Complete";
}
// Task 1 will take 8 seconds to complete download
private suspend fun downloadTask2() : Int {
kotlinx.coroutines.delay(8000);
return 100;
}
// Task 1 will take 5 seconds to complete download
private suspend fun downloadTask3() : Float {
kotlinx.coroutines.delay(5000);
return 4.0f;
}
}
In this example, my code is downloading 3 data on click of btnCount button and showing pgBar progress bar until all download gets completed. There are 3 suspend functions downloadTask1(), downloadTask2() and downloadTask3() which downloads data. To simulate it, I've used delay() in these functions. These functions waits for 5 seconds, 8 seconds and 5 seconds respectively.
As we've used launch for starting these suspend functions, launch will execute them sequentially (one-by-one). This means that, downloadTask2() would start after downloadTask1() gets completed and downloadTask3() would start only after downloadTask2() gets completed.
As in output screenshot Toast, total execution time to complete all 3 downloads would lead to 5 seconds + 8 seconds + 5 seconds = 18 seconds with launch
async
As we saw that launch makes execution sequentially for all 3 tasks. The time to complete all tasks was 18 seconds.
If those tasks are independent and if they do not need other task's computation result, we can make them run concurrently. They would start at same time and run concurrently in background. This can be done with async.
async returns an instance of Deffered<T> type, where T is type of data our suspend function returns. For example,
downloadTask1() would return Deferred<String> as String is return type of function
downloadTask2() would return Deferred<Int> as Int is return type of function
downloadTask3() would return Deferred<Float> as Float is return type of function
We can use the return object from async of type Deferred<T> to get the returned value in T type. That can be done with await() call. Check below code for example
btnCount.setOnClickListener {
pgBar.visibility = View.VISIBLE
CoroutineScope(Dispatchers.Main).launch {
val currentMillis = System.currentTimeMillis()
val retVal1 = async(Dispatchers.IO) { downloadTask1() }
val retVal2 = async(Dispatchers.IO) { downloadTask2() }
val retVal3 = async(Dispatchers.IO) { downloadTask3() }
Toast.makeText(applicationContext, "All tasks downloaded! ${retVal1.await()}, ${retVal2.await()}, ${retVal3.await()} in ${(System.currentTimeMillis() - currentMillis)/1000} seconds", Toast.LENGTH_LONG).show();
pgBar.visibility = View.GONE
}
This way, we've launched all 3 tasks concurrently. So, my total execution time to complete would be only 8 seconds which is time for downloadTask2() as it is largest of all of 3 tasks. You can see this in following screenshot in Toast message
both coroutine builders namely launch and async are basically lambdas with receiver of type CoroutineScope which means their inner block is compiled as a suspend function, hence they both run in an asynchronous mode AND they both will execute their block sequentially.
The difference between launch and async is that they enable two different possibilities. The launch builder returns a Job however the async function will return a Deferred object. You can use launch to execute a block that you don't expect any returned value from it i.e writing to a database or saving a file or processing something basically just called for its side effect. On the other hand async which return a Deferred as I stated previously returns a useful value from the execution of its block, an object that wraps your data, so you can use it for mainly its result but possibly for its side effect as well. N.B: you can strip the deferred and get its value using the function await, which will block the execution of your statements until a value is returned or an exceptions is thrown! You could achieve the same thing with launch by using the function join()
both coroutine builder (launch and async) are cancelable.
anything more?: yep with launch if an exception is thrown within its block, the coroutine is automatically canceled and the exceptions is delivered. On the other hand, if that happens with async the exception is not propagated further and should be caught/handled within the returned Deferred object.
more on coroutines https://kotlinlang.org/docs/tutorials/coroutines/coroutines-basic-jvm.html
https://www.codementor.io/blog/kotlin-coroutines-6n53p8cbn1
Async and Launch, both are used to create coroutines that run in the background. In almost every situation one can use either of them.
tl;dr version:
When you dont care about the task's return value, and just want to run it, you may use Launch. If you need the return type from the task/coroutine you should use async.
Alternate:
However, I feel the above difference/approach is a consequence of thinking in terms of Java/one thread per request model. Coroutines are so inexpensive, that if you want to do something from the return value of some task/coroutine(lets say a service call) you are better off creating a new coroutine from that one. If you want a coroutine to wait for another coroutine to transfer some data, I would recommend using channels and not the return value from Deferred object. Using channels and creating as much number of coroutines as required, is the better way IMO
Detailed answer:
The only difference is in the return type and what functionality it provides.
Launch returns Job while Async returns Deferred. Interestingly enough, Deferred extends Job. Which implies it must be providing additional functionality on top of Job. Deferred is type parameterised over where T is the return type. Thus, Deferred object can return some response from the block of code executed by async method.
p.s. I only wrote this answer because I saw some factually incorrect answers on this question and wanted to clarify the concept for everyone. Also, while working on a pet project myself I faced similar problem because of previous Java background.
launch returns a job
async returns a result (deferred job)
launch with join is used to wait until the job gets finished. It simply suspends the coroutine calling join(), leaving the current thread free to do other work (like executing another coroutine) in the meantime.
async is used to compute some results. It creates a coroutine and returns its future result as an implementation of Deferred. The running coroutine is cancelled when the resulting deferred is cancelled.
Consider an async method that returns a string value. If the async method is used without await it will return a Deferred string but if await is used you will get a string as the result
The key difference between async and launch:
Deferred returns a particular value of type T after your Coroutine finishes executing, whereas Job doesn’t.
launch / async no result
Use when don't need the result,
Don't block the code where is called,
Run in sequential
async for result
When you need to wait for the result and can run in parallel for
efficiency,
Block the code where is called,
Run in concurrent
Alongside the other great answers, for the people familiar with Rx and getting into coroutines, async returns a Deferred which is akin to Single while launch returns a Job that is more akin to Completable. You can .await() to block and get the value of the first one, and .join() to block until the Job is completed.
I have a collection of requests in POSTMAN. I wanted to add a pause between 2 requests but I couldn't find a way to do so from reading their docs. Any idea?
UPDATE I only wanted to put a pause after one request instead of after every request in the collection.
In case someone is still looking for this - You can add delay after/before 1 of many test in a collection you can use:
setTimeout(function(){}, [number]);
where 'number' is the milliseconds. If it's added at 'Tests' it will be executed after request is send. If it's added at Pre-request Scripts it will be executed before request is send.
Using javascript's busy wait is a good hack but it makes your CPU hot and the app unresponsive. I figured out this solution using postman-echo.
Assuming you want to add a long delay between Request_A and Request_B.
First, in Request_A's test script set up an env var to mark the start.
environment.delayTimerStart = new Date();
Then, create a GET request in the creation (here called 'Delay 10s'). It makes a GET on https://postman-echo.com/delay/10 (It returns after 10s)
In its test script, add
var curDate = new Date();
if (curDate - environment.delayTimerStart < delay_time_in_sec*1000) {
postman.setNextRequest('Delay 10s');
} else {
postman.setNextRequest("Request_B");
}
In this way you can add a delay of any length.
Note: 10 sec is the maxium value that postman-echo accepts. If you just need a short delay, simply GET https://postman-echo.com/delay/[1~10].
Try something like this-
if(jsonBody.responseCode=="SUCCESS"){
setTimeout(function(){
console.log("Sleeping for 3 seconds before next request.");
}, 3000);
postman.setNextRequest("nextAPI");
}
I know two possible ways to do this
Method I
Run your request as a collection. (https://www.getpostman.com/docs/collections)
Use Newman (Postman's collection runner from command line) to run your collection with --delay flag. The delay input value is in milliseconds.
Method II
This is a nice hack which I found here https://github.com/postmanlabs/postman-app-support/issues/1038. You can add a delay function to your test script in Postman.
Just simple example, I'm sure you will be understand.
setTimeout(() => {}, 15000);
15000 it's a value in miliseconds
looking at the current documentation if you are using Postman Runner
Delay
This is the interval (in ms) between each request in your collection run.
https://www.getpostman.com/docs/postman/collection_runs/starting_a_collection_run
and if you are using newman
--delay-request [number] Specify a delay (in ms) between requests [number]
https://www.getpostman.com/docs/postman/collection_runs/command_line_integration_with_newman
If you have the standalone Postman App (supports ES7) and you are intending to test only on Postman, not on newman(which doesn't support ES7), then you can have something like this in the Pre-Request script of the Request you want to delay:
function foo() {
return (new Promise((resolve, reject) => {
setTimeout(() => {
resolve("done!"); // passing argument is optional, can just use resolve()
}, 10000) // specify the delay time in ms here..
}))
}
async function waitForMe() {
await foo().then((val) => {
console.log(val); // not required, you can just do an await without then
})
}
waitForMe();
I prefer to use an online service Postman Echo's delay endpoint (docs are here). Simply create a request that uses this service and call it between the two other requests you wish to add a delay between.
If you want to check the status of something before continuing, you can use postman.setNextRequest() in the Tests of a request to loop until something has been completed. Simply do the following:
1) Create a collection structured as:
Delay For 10 Seconds
Status Check
Continue Processing
2) In the Status Check request Tests:
if(responseBody.has("Success")) //or any other success condition
{
postman.setNextRequest('Continue Processing');
tests["Success found"] = true;
}
else
{
postman.setNextRequest('Delay For 10 Seconds');
tests["No success found"] = true;
}
You can use JavaScript setTimeout method. This will make sure that your method will be executed after given delay.
setTimeout(checkStatusCode, 500);
function checkStatusCode() {
pm.sendRequest('https://postman-echo.com/get', function (err, res) {
tests['status code should be 200']= res.code ===200;
});
}
My actual program is more sophisticated than this, but I've tried to simplify things.
So let's say I'm reading a file with a list of URLs. I want to download the HTML from each URL and process it. The processing may be a bit complex, so I'd like it to be done on a separate thread.
The basic problem is how to tell when all the processing is done. For example, if the user tries to close the program before all URLs are processed, I want to give him a message and not exit the program. Alternatively, I want to terminate the program (perhaps with a MsgBox("Done") message) as soon as all the URLs are processed.
I'd like my code to look something as follows (assuming I've got an outer loop reading the URLs and calling this routine)...
List<Task> TaskList = new List<Task>();
async void ProcessSingleUrl(string url) {
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
Task t = Task.Run(() => ProcessWebPage(WebPageContents);
TaskList.Add(t);
}
The above code should run very quickly (Async methods run pretty well instantly) and return to the caller almost immediately.
But at that point, I may well have no entries whatsoever in TaskList, since a task isn't defined until the GetStringAsync is completed, and none (or maybe just a few) may have finished by then. So
Task.WaitAll(TaskList.ToArray());
doesn't work the way I need it to.
If absolutely necessary, I could first read in all the URLs and know how many Tasks to expect, but I'm hoping for a more elegant solution.
I suppose I could increment a counter just before the await, but that feels a bit kludgy.
I assume I'm structuring things incorrectly, but I'm not sure how to reorganize things.
Note: I'm not wedded to Task.Run. Good ol' QueueWorkItem is a possibility, but I think it has pretty well the same problems.
I assume I'm structuring things incorrectly, but I'm not sure how to reorganize things.
I think that is true. Here is a possible solution: Store the whole computation as a Task in your list, not just the second part:
async Task ProcessSingleUrlInner(string url) {
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
Task t = Task.Run(() => ProcessWebPage(WebPageContents);
await t;
}
void ProcessSingleUrl(string url) {
var t = ProcessSingleUrlInner(url);
TaskList.Add(t);
}
Waiting on all tasks of this list will guarantee that everything is done. Probably, you need to adapt this idea to your exact needs.
I'm assuming that you're getting the list of urls as an IEnumerable<string> or some such.
You can use LINQ to convert each url into a Task, and then await them all to complete:
async Task ProcessUrls(IEnumerable<string> urls)
{
var tasks = urls.Select(async url =>
{
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
await Task.Run(() => ProcessWebPage(WebPageContents);
});
await Task.WhenAll(tasks);
}
Note that if you use this solution and there are multiple different urls that have errors, then Task.WhenAll will only report one of those errors.