Parallel decomposition of independent async tasks with kotlin coroutines - asynchronous

I am trying to run multiple tasks in parallel, but the tasks are independent so if one of children coroutine fails, I do not want its siblings or parent to fail too. In the code below I have used coroutineScope to create a new scope in which these tasks run and launched 5 async tasks each on sending its id and the delay time it should wait. The second coroutine throws an exception. In this case the code does what i want it to do, it calculates the sum of the jobs that successfully finished and the ones that failed return 0.
However, I am reading that there is also supervisorScope in the kotlinx library which should be preferred instead of coroutineScope (which cancels parent/siblings if exception is not handled) for tasks are not dependent on others. I am not sure why I should be changing to use supervisorScope as I am getting the result I want with coroutineScope.
Q1: If i were to change to supervisorScope should something change in my async blocks?
Q2: Is it accepted to catch any exception inside the async block and dont let anything be propagated to its parent? I know you can also catch exceptions during .await() phase but is that the way it should be done?
runBlocking {
coroutineScope {
val job1 = async<Int> {
try {
request(1, 1000)
} catch (e: Exception) {
println("Job 1 failed with $e")
0
}
}
val job2 = async<Int> {
try {
request(2, 2000)
throw Exception("cancelling Job 2")
} catch (e: Exception) {
println("Job 2 failed: $e")
0
}
}
val job3 = async {
try {
request(3, 3000)
} catch (e: Exception) {
println("Job 3 failed with $e")
0
}
}
val job4 = async {
try {
request(4, 4000)
} catch (e: Exception) {
println("Job 4 failed with $e")
0
}
}
val job5 = async {
try {
request(5, 5000)
} catch (e: Exception) {
println("Job 5 failed with $e")
0
}
}
val result = job1.await() + job2.await() + job3.await() + job4.await() + job5.await()
println(result.toString())
}
println("Finished")
}
suspend fun request(id: Int, time: Long): Int {
println("Job $id started")
delay(time)
println("Job $id finished")
return id
}

The reason all coroutines run to completion is that you catch the exception thrown by job 2 within job 2 itself, so it never propagates up the hierarchy of Jobs, so nothing happens.
However, if you remove that catch clause in job2, job[1-5] will always be canceled, independent of if you use coroutineScope or supervisorScope.
This is because job2.await() will throw the exception instead. Since this happens within the parent job of job[1-5] (i.e. within the top coroutineScope/supervisorScope), and since a failed/cancelled parent job always cancel child jobs, job[1-5] will also be cancelled.
A1: Use none of coroutineScope or supervisorScope, remove coroutineScope and put things directly under runBlocking.
A2: It certainly is allowed to catch exceptions within async { } to make sure it doesn't happen in .await() if it fits your use case.

Related

firestore, coroutine and flow

firebase method is working on worker thread automatically. but I have used coroutine and callbackflow to implement firebase listener code synchronously or get return from the listener.
below is my code that I explained
coroutine await with firebase for one shot
override suspend fun checkNickName(nickName: String): Results<Int> {
lateinit var result : Results<Int>
fireStore.collection("database")
.document("user")
.get()
.addOnCompleteListener { document ->
if (document.isSuccessful) {
val list = document.result.data?.get("nickNameList") as List<String>
if (list.contains(nickName))
result = Results.Exist(1)
else
result = Results.No(0)
//document.getResult().get("nickNameList")
}
else {
}
}.await()
return result
}
callbackflow with firebase listener
override fun getOwnUser(): Flow<UserEntity> = callbackFlow{
val document = fireStore.collection("database/user/userList/")
.document("test!!!!!")
val subscription = document.addSnapshotListener { snapshot,_ ->
if (snapshot!!.exists()) {
val ownUser = snapshot.toObject<UserEntity>()
if (ownUser != null) {
trySend(ownUser)
}
}
}
awaitClose { subscription.remove() }
}
so I really wonder these way is good or bad practice and its reason
Do not combine addOnCompleteListener with coroutines await(). There is no guarantee that the listener gets called before or after await(), so it is possible the code in the listener won't be called until after the whole suspend function returns. Also, one of the major reasons to use coroutines in the first place is to avoid using callbacks. So your first function should look like:
override suspend fun checkNickName(nickName: String): Results<Int> {
try {
val userList = fireStore.collection("database")
.document("user")
.get()
.await()
.get("nickNameList") as List<String>
return if (userList.contains(nickName)) Results.Exist(1) else Results.No(0)
} catch (e: Exception) {
// return a failure result here
}
}
Your use of callbackFlow looks fine, except you should add a buffer() call to the flow you're returning so you can specify how to handle backpressure. However, it's possible you will want to handle that downstream instead.
override fun getOwnUser(): Flow<UserEntity> = callbackFlow {
//...
}.buffer(/* Customize backpressure behavior here */)

How to run a Firebase Transaction using Kotlin Coroutines?

I'm trying to run a Firebase Transaction under a suspended function in Kotlin and i see no documentation about it.
I'm using
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-play-services:1.5.2'
for coroutines with firebase (eg: setValue(*).await() ) but there seems to be no await function for runTransaction(*)
override suspend fun modifyProductStock(
product: ProductModel,
valueToModify: Long,
replace: Boolean
) {
CoroutineScope(Dispatchers.Main).launch {
val restaurantId = authRepository.restaurantId.value ?: throw Exception("No restaurant!")
val productId = product.id ?: throw Exception("No Product ID!")
val reference = FirebaseDatabase.getInstance().getReference("database/$restaurantId").child("products")
if (replace) {
reference.child(productId).child("stock").setValue(valueToModify).await()
} else {
reference.child(productId).child("stock")
.runTransaction(object : Transaction.Handler {
override fun doTransaction(p0: MutableData): Transaction.Result {
//any operation
return Transaction.success(p0)
}
override fun onComplete(p0: DatabaseError?, p1: Boolean, p2: DataSnapshot?) {
}
})
}
}
}
You could wrap it in suspendCoroutine:
val result: DataSnapshot? = suspendCoroutine { c ->
reference.child(productId).child("stock")
.runTransaction(object : Transaction.Handler {
override fun doTransaction(p0: MutableData): Transaction.Result {
//any operation
return Transaction.success(p0)
}
override fun onComplete(error: DatabaseError?, p1: Boolean, snapshot: DataSnapshot?) {
c.resume(snapshot)
}
})
}
suspendCoroutine
Obtains the current continuation instance inside suspend functions and suspends the currently running coroutine.
In this function both Continuation.resume and Continuation.resumeWithException can be used either synchronously in the same stack-frame where the suspension function is run or asynchronously later in the same thread or from a different thread of execution.
Given that the Kotlin example in the Firebase documentation on transactions uses the same callback style that you have, it seems indeed that there is no specific support for co-routines there.
It might be worth posting an issue on the Android SDK repo to get it added, or hear why it wasn't added.

asynchronous programming in dart (Futures, await)

what changes should I do to the following code to get the following output:
Task 1 complete
Task 4 complete
Task 2 complete
Task 3 complete with task 2 data
I currently getting the outputs given below:
Task 1 complete
Task 2 complete
Task 3 complete with task 2 data
Task 4 complete
import 'dart:async';
void main() {
performTasks();
}
void performTasks() async {
task1();
String task2Result = await task2();
task3(task2Result);
task4();
}
void task1() {
String result = 'task 1 data';
print('Task 1 complete');
}
Future<String> task2() async {
Duration threeSeconds = Duration(seconds: 3);
String result;
await Future.delayed(threeSeconds, () {
result = 'task 2 data';
print('Task 2 complete');
});
return result;
}
void task3(String task2Data) {
String result = 'task 3 data';
print('Task 3 complete with $task2Data');
}
void task4() {
String result = 'task 4 data';
print('Task 4 complete');
}
Don't call task4 after waiting for task2.
So:
void performTasks() async {
task1();
task4();
String task2Result = await task2();
task3(task2Result);
}
That looks pretty obvious, so I'm assuming your real problem is more complicated, and you can't move test4() around like that.
In that case, you should not use await. The await ensures that everything written after the await also executes after the awaited future has completed.
Instead you can fall back on the Future API:
void performTasks() { // no `async`
task1();
task2().then(task3); // no `await`
task4();
}
This sets up task2() to run, and when that is done, it calls task3 with the result. It doesn't wait for anything, though, and it executes task4 immediately after setting this up.
The then method on futures takes a callback, and eventually calls that with the result of the future. Here task3 takes one argument, so it can directly be used as that callback.
That assumes that task2's result is directly usable as an argument to task3. If not, and you have to capture the result and manipulate it first, you'd do it as:
void performTasks() { // no `async`
task1();
task2().then((result2) { // still no await here!
var argument3 = manipulate(result2);
// ... and whatever else you want to do
// between task2 completing and task3 starting.
task3(argument3);
});
task4();
}
in performTasks(), you should move task4(); right after task1();.

Unity3d with Firebase: Can't Start Coroutine

Greeting,
I'm currently facing a problem that my coroutine can't start. This is the first time I facing this issues and I can't find a proper solution online. Much appreciated if anyone can point me to the right direction to solve this issue.
Here are the code.
path_reference.GetDownloadUrlAsync().ContinueWith((Task<Uri> task) => {
if (!task.IsFaulted && !task.IsCanceled)
{
Debug.Log("Download URL: " + task.Result);
StartCoroutine(DownloadStuff(task.Result));
}
else
{
Debug.Log(task.Exception.ToString());
}
});
}
IEnumerator DownloadStuff(Uri uri)
{
Debug.Log("Start Download");
using (var www = UnityWebRequestTexture.GetTexture(uri))
{
yield return www.SendWebRequest();
if (www.isNetworkError || www.isHttpError)
{
Debug.Log(www.error);
}
else
{
var texture = DownloadHandlerTexture.GetContent(www);
//Texture2D texture = new Texture2D(1, 1);
//if you need sprite for SpriteRenderer or Image
Sprite sprite = Sprite.Create(texture, new Rect(0.0f, 0.0f, texture.width,
texture.height), new Vector2(0.5f, 0.5f), 100.0f);
Debug.Log("Finished downloading!");
}
Debug.Log(www.downloadProgress);
}
}'
The task returned by Firebase probably finishes execution on a thread other than the main thread, and Unity coroutines can only run on the main thread.
Unity's support of multithreading and async is pretty spotty, including "eating" some errors if the continuations of those errors would execute on another thread other than the main thread.
To fix this, you need to change the function that starts your coroutine:
try {
// Important: ConfigureAwait(true) ensures the code after this will run in the
// same thread as the code before (which is the main thread!)
var url = await path_reference.GetDownloadUrlAsync().ConfigureAwait(true);
StartCoroutine(DownloadStuff(url));
} catch (Exception ex) {
// Tip: when logging errors, use LogException and pass the whole exception,
// that way you will get pretty links to the error line in the whole stack trace.
Debug.LogException(ex);
}
As an aside, I usually have a few extension methods on all my projects to deal with that while staying in async-world instead of coroutine-world (because at least with async I can catch errors and not just "halt and catch fire" like Unity's coroutines)
The main ones are:
public static Task ToTask(this YieldInstruction self, MonoBehaviour owner) {
var source = new TaskCompletionSource<object>();
IEnumerable Routine() {
yield return self;
source.SetResult(null);
}
return source.Task;
}
private static Task SendAsync(this UnityWebRequest self, MonoBehaviour owner) {
var source = new TaskCompletionSource<object>();
await self.SendWebRequest().ToTask(owner);
if (
self.isHttpError
|| self.isNetworkError
) {
source.SetException(new Exception(request.error));
yield break;
}
source.SetResult(null);
}
Which you can use like this, inside a MonoBehaviour:
await new WaitForSeconds(0.2f).ToTask(this);
UnityWebRequest request = /* etc */;
await request.SendAsync(this);
var texture = DownloadHandlerTexture.GetContent(request);
Note that these methods do not require ConfigureAwait, since their SetResult/SetException invocations are ran from Unity-provided coroutine continuations.

what is the best practice of Vert.x handler for checking check existence?

I am implementing a method using Vertx to check the existence of certain value in the database and use Handler with AsyncResult.
I would like to know which one is the best practice:
Option 1: When nothing found, Handler is with succeededFuture but with result as FALSE:
public void checkExistence (..., String itemToFind, Handler<AsyncResult<Boolean>> resultHandler) {
// ....
doQuery(..., queryHandler -> {
if (queryHandler.succeeded()) {
List<JsonObject> results = queryHandler.result();
boolean foundIt = false;
for (JsonObject json: results) {
if (json.getString("someKey").equals(itemToFind)) {
foundIt = true;
break;
}
}
resultHandler.handle(Future.succeededFuture(foundIt));
} else {
resultHandler.handle(Future.failedFuture(queryHandler.cause().toString()));
}
});
}
Option 2: When nothing found, Handler is with failedFuture:
public void checkExistence (..., String itemToFind, Handler<AsyncResult<Void>> resultHandler) {
// ....
doQuery(..., queryHandler -> {
if (queryHandler.succeeded()) {
List<JsonObject> results = queryHandler.result();
boolean foundIt = false;
for (JsonObject json: results) {
if (json.getString("someKey").equals(itemToFind)) {
foundIt = true;
break;
}
}
// HERE IS THE DIFFERENCE!!!
if (foundIt) {
resultHandler.handle(Future.succeededFuture());
} else {
resultHandler.handle(Future.failedFuture("Item " + itemToFind + " not found!"));
}
} else {
resultHandler.handle(Future.failedFuture(queryHandler.cause().toString()));
}
});
}
UPDATE:
Let's say I have another example, instead of checking the existence, I would like to get all the results. Do I check the Empty results? Do I treat Empty as failure or success?
Option 1: only output them when it's not null or empty, otherwise, fail it
public void getItems(..., String itemType, Handler<AsyncResult<List<Item>>> resultHandler) {
// ....
doQuery(..., queryHandler -> {
if (queryHandler.succeeded()) {
List<Item> items = queryHandler.result();
if (items != null && !items.empty()) {
resultHandler.handle(Future.succeededFuture(items));
} else {
resultHandler.handle(Future.failedFuture("No items found!"));
}
} else {
resultHandler.handle(Future.failedFuture(queryHandler.cause().toString()));
}
});
}
Option 2: output results I got, even though it could be empty or null
public void getItems(..., String itemType, Handler<AsyncResult<List<Item>>> resultHandler) {
// ....
doQuery(..., queryHandler -> {
if (queryHandler.succeeded()) {
List<Item> items = queryHandler.result();
resultHandler.handle(Future.succeededFuture(items));
} else {
resultHandler.handle(Future.failedFuture(queryHandler.cause().toString()));
}
});
}
The 1st one option is better, because you can clearly say, that checkExistence returned True or False and completed successfully or it failed with some exception (database issue, etc.).
But lets say, you've decided to stick with 2nd option. Then, imagine you have another method:
void getEntity(int id, Handler<AsyncResult<Entity>> resultHandler);
If entity with provided id doesn't exists, will you throw exception (using Future.failedFuture) or return null (using Future.succeededFuture)? I think, you should throw exception to make your methods logic similar to each other. But again, is that exceptional situation?
For case with returning list of entities you can just return empty list, if there are no entities. Same goes to single entity: it's better to return Optional<Entity> instead of Entity, because in this way you avoid NullPointerException and don't have nullable variables in the code. What's better: Optional<List<Entity>> or empty List<Entity>, it's open question.
Particularly if you're writing this as reusable code, then definitely go with your first option. This method is simply determining whether an item exists, and so should simply return whether it does or not. How is this particular method to know whether it's an error condition that the item doesn't exist?
Some caller might determine that it is indeed an error; it that's the case, then it will throw an appropriate exception if the Future returns with false. But another caller might simply need to know whether the item exists before proceeding; in that case, you'll find yourself using exception handling to compose your business logic.

Resources