I'm trying to get all files from firebase's storage through listAll.
By the way..
storageReference.listAll().addOnSuccessListener { listResult ->
val image_task : FileDownloadTask
for (fileRef in listResult.items) {
fileRef.downloadUrl.addOnSuccessListener { Uri ->
image_list.add(Uri.toString())
println("size1 : " + image_list.size)
}
}
println("size2 : " + image_list.size)
}//addOnSuccessListener
enter image description here
Why is the execution order like this?
How do I solve it??
When you add a listener or callback to something, the code inside the listener will not be called until sometime later. Everything else in the current function will happen first.
You are adding listeners for each item using your for loop. No code in the listeners is running yet. Then your "size2" println call is made after the for loop. At some later time, all your listeners will fire.
If you want asynchronous code like this to be written sequentially, then you need to use coroutines. That's a huge topic, but your code would look something like this (but probably a little more involved than this if you want to properly handle errors). I'm using lifecycleScope from an Android Activity or Fragment for this example. If you're not on Android, you need to use some other CoroutineScope.
The calls to await() are an alternative to adding success and failure listeners. await() suspends the coroutine and then returns a result or throws an exception on failure.
lifecycleScope.launch {
val results = try {
storageReference.listAll().await()
} catch (e: Exception) {
println("Failed to get list: ${e.message}")
return#launch
}
val uris = try {
results.map { it.downloadUrl.await().toString() }
} catch (e: Exception) {
println("Failed to get at least one URI: ${e.message}")
return#launch
}
image_list.addAll(uris)
}
There is nothing wrong with the execution order here.
fileRef.downloadUrl.addOnSuccessListener { Uri ->
the downloadUrl is an asynchronous action which means it doesn't wait for the action to actually complete in order to move along with the code.
You receive the result with the success listener (at least in this case)
If you want to deal with it in a sequential way, look at coroutines.
I've made a function that calls on the FireBase database and will return a MutableList. However, when I try to make it return on a specific line, it says it requires a Unit instead of the MutableList.
fun firebaseCollect(key: String): MutableList<CustomList> {
var ref = FirebaseDatabase.getInstance().getReference(key)
var lessonList = mutableListOf<CustomList>()
ref.addValueEventListener(object: ValueEventListener{
override fun onCancelled(p0: DatabaseError?) {
}
override fun onDataChange(p0: DataSnapshot?) {
if (p0!!.exists()) {
lessonList.clear()
for (index in p0.children) {
val lesson = index.getValue(CustomList::class.java)
lessonList.add(lesson!!)
}
return lessonList
}
}
})
return lessonList
}
Type mismatch. Required: Unit, Found: MutableList< CustomList > is found at the first return lessonList since what I am asking for it to return is a MutableList not a Unit. I am confused as to why this happens. The last return would give an empty list. It is currently my first jab at FireBase and this is a practice I am doing. The rules for read and write have been set to public as well. How should I recode the function that I am able to return the data from FireBase into the function and passed back to the caller?
Firebase APIs are asynchronous. For your case, that means addValueEventListener returns immediately. Then, some time later, the listener you passed to it will be invoked with the data you're looking for. Your return statement in the callback doesn't actually return any data to the caller. In fact, you can't return anything from that callback. At the bottom of your function, when you return lessonList, you're actually returning an initially empty list to the caller, which may change later when the data finally arrives.
To get a better sense of how your code works, put log lines in various places, and see for yourself the order in which the code is invoked. You can read more about why Firebase APIs are asynchronous by reading this article. The bottom line is that you'll need to interact with the asynchronous APIs using asynchronous programming techniques. Don't try to make them synchronous.
Data is loaded asynchronously from Firebase. Once the data is fetched the method onDatachange() is invoked.
You are returning lessonList inside onDatachange(). Return type of onDatachange() is void(Unit in kotlin). This is the reason for the type mismatch error.
For returning the result from the method onDatachange() try this.
In the kotlinx.coroutines library you can start new coroutine using either launch (with join) or async (with await). What is the difference between them?
launch is used to fire and forget coroutine. It is like starting a new thread. If the code inside the launch terminates with exception, then it is treated like uncaught exception in a thread -- usually printed to stderr in backend JVM applications and crashes Android applications. join is used to wait for completion of the launched coroutine and it does not propagate its exception. However, a crashed child coroutine cancels its parent with the corresponding exception, too.
async is used to start a coroutine that computes some result. The result is represented by an instance of Deferred and you must use await on it. An uncaught exception inside the async code is stored inside the resulting Deferred and is not delivered anywhere else, it will get silently dropped unless processed. You MUST NOT forget about the coroutine you’ve started with async.
I find this guide to be useful. I will quote the essential parts.
🦄 Coroutines
Essentially, coroutines are light-weight threads.
So you can think of a coroutine as something that manages thread in a very efficient way.
🐤 launch
fun main(args: Array<String>) {
launch { // launch new coroutine in background and continue
delay(1000L) // non-blocking delay for 1 second (default time unit is ms)
println("World!") // print after delay
}
println("Hello,") // main thread continues while coroutine is delayed
Thread.sleep(2000L) // block main thread for 2 seconds to keep JVM alive
}
So launch starts a coroutine, does something, and returns a token immediately as Job. You can call join on this Job to block until this launch coroutine completes.
fun main(args: Array<String>) = runBlocking<Unit> {
val job = launch { // launch new coroutine and keep a reference to its Job
delay(1000L)
println("World!")
}
println("Hello,")
job.join() // wait until child coroutine completes
}
🦆 async
Conceptually, async is just like launch. It starts a separate coroutine which is a light-weight thread that works concurrently with all the other coroutines. The difference is that launch returns a Job and does not carry any resulting value, while async returns a Deferred -- a light-weight non-blocking future that represents a promise to provide a result later.
So async starts a background thread, does something, and returns a token immediately as Deferred.
fun main(args: Array<String>) = runBlocking<Unit> {
val time = measureTimeMillis {
val one = async { doSomethingUsefulOne() }
val two = async { doSomethingUsefulTwo() }
println("The answer is ${one.await() + two.await()}")
}
println("Completed in $time ms")
}
You can use .await() on a deferred value to get its eventual result, but Deferred is also a Job, so you can cancel it if needed.
So Deferred is actually a Job. Read this for more details.
interface Deferred<out T> : Job (source)
🦋 async is eager by default
There is a laziness option to async using an optional start parameter with a value of CoroutineStart.LAZY. It starts coroutine only when its result is needed by some await or if a start function is invoked.
launch and async are used to start new coroutines. But, they execute them in different manner.
I would like to show very basic example which will help you understand difference very easily
launch
class MainActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
btnCount.setOnClickListener {
pgBar.visibility = View.VISIBLE
CoroutineScope(Dispatchers.Main).launch {
val currentMillis = System.currentTimeMillis()
val retVal1 = downloadTask1()
val retVal2 = downloadTask2()
val retVal3 = downloadTask3()
Toast.makeText(applicationContext, "All tasks downloaded! ${retVal1}, ${retVal2}, ${retVal3} in ${(System.currentTimeMillis() - currentMillis)/1000} seconds", Toast.LENGTH_LONG).show();
pgBar.visibility = View.GONE
}
}
// Task 1 will take 5 seconds to complete download
private suspend fun downloadTask1() : String {
kotlinx.coroutines.delay(5000);
return "Complete";
}
// Task 1 will take 8 seconds to complete download
private suspend fun downloadTask2() : Int {
kotlinx.coroutines.delay(8000);
return 100;
}
// Task 1 will take 5 seconds to complete download
private suspend fun downloadTask3() : Float {
kotlinx.coroutines.delay(5000);
return 4.0f;
}
}
In this example, my code is downloading 3 data on click of btnCount button and showing pgBar progress bar until all download gets completed. There are 3 suspend functions downloadTask1(), downloadTask2() and downloadTask3() which downloads data. To simulate it, I've used delay() in these functions. These functions waits for 5 seconds, 8 seconds and 5 seconds respectively.
As we've used launch for starting these suspend functions, launch will execute them sequentially (one-by-one). This means that, downloadTask2() would start after downloadTask1() gets completed and downloadTask3() would start only after downloadTask2() gets completed.
As in output screenshot Toast, total execution time to complete all 3 downloads would lead to 5 seconds + 8 seconds + 5 seconds = 18 seconds with launch
async
As we saw that launch makes execution sequentially for all 3 tasks. The time to complete all tasks was 18 seconds.
If those tasks are independent and if they do not need other task's computation result, we can make them run concurrently. They would start at same time and run concurrently in background. This can be done with async.
async returns an instance of Deffered<T> type, where T is type of data our suspend function returns. For example,
downloadTask1() would return Deferred<String> as String is return type of function
downloadTask2() would return Deferred<Int> as Int is return type of function
downloadTask3() would return Deferred<Float> as Float is return type of function
We can use the return object from async of type Deferred<T> to get the returned value in T type. That can be done with await() call. Check below code for example
btnCount.setOnClickListener {
pgBar.visibility = View.VISIBLE
CoroutineScope(Dispatchers.Main).launch {
val currentMillis = System.currentTimeMillis()
val retVal1 = async(Dispatchers.IO) { downloadTask1() }
val retVal2 = async(Dispatchers.IO) { downloadTask2() }
val retVal3 = async(Dispatchers.IO) { downloadTask3() }
Toast.makeText(applicationContext, "All tasks downloaded! ${retVal1.await()}, ${retVal2.await()}, ${retVal3.await()} in ${(System.currentTimeMillis() - currentMillis)/1000} seconds", Toast.LENGTH_LONG).show();
pgBar.visibility = View.GONE
}
This way, we've launched all 3 tasks concurrently. So, my total execution time to complete would be only 8 seconds which is time for downloadTask2() as it is largest of all of 3 tasks. You can see this in following screenshot in Toast message
both coroutine builders namely launch and async are basically lambdas with receiver of type CoroutineScope which means their inner block is compiled as a suspend function, hence they both run in an asynchronous mode AND they both will execute their block sequentially.
The difference between launch and async is that they enable two different possibilities. The launch builder returns a Job however the async function will return a Deferred object. You can use launch to execute a block that you don't expect any returned value from it i.e writing to a database or saving a file or processing something basically just called for its side effect. On the other hand async which return a Deferred as I stated previously returns a useful value from the execution of its block, an object that wraps your data, so you can use it for mainly its result but possibly for its side effect as well. N.B: you can strip the deferred and get its value using the function await, which will block the execution of your statements until a value is returned or an exceptions is thrown! You could achieve the same thing with launch by using the function join()
both coroutine builder (launch and async) are cancelable.
anything more?: yep with launch if an exception is thrown within its block, the coroutine is automatically canceled and the exceptions is delivered. On the other hand, if that happens with async the exception is not propagated further and should be caught/handled within the returned Deferred object.
more on coroutines https://kotlinlang.org/docs/tutorials/coroutines/coroutines-basic-jvm.html
https://www.codementor.io/blog/kotlin-coroutines-6n53p8cbn1
Async and Launch, both are used to create coroutines that run in the background. In almost every situation one can use either of them.
tl;dr version:
When you dont care about the task's return value, and just want to run it, you may use Launch. If you need the return type from the task/coroutine you should use async.
Alternate:
However, I feel the above difference/approach is a consequence of thinking in terms of Java/one thread per request model. Coroutines are so inexpensive, that if you want to do something from the return value of some task/coroutine(lets say a service call) you are better off creating a new coroutine from that one. If you want a coroutine to wait for another coroutine to transfer some data, I would recommend using channels and not the return value from Deferred object. Using channels and creating as much number of coroutines as required, is the better way IMO
Detailed answer:
The only difference is in the return type and what functionality it provides.
Launch returns Job while Async returns Deferred. Interestingly enough, Deferred extends Job. Which implies it must be providing additional functionality on top of Job. Deferred is type parameterised over where T is the return type. Thus, Deferred object can return some response from the block of code executed by async method.
p.s. I only wrote this answer because I saw some factually incorrect answers on this question and wanted to clarify the concept for everyone. Also, while working on a pet project myself I faced similar problem because of previous Java background.
launch returns a job
async returns a result (deferred job)
launch with join is used to wait until the job gets finished. It simply suspends the coroutine calling join(), leaving the current thread free to do other work (like executing another coroutine) in the meantime.
async is used to compute some results. It creates a coroutine and returns its future result as an implementation of Deferred. The running coroutine is cancelled when the resulting deferred is cancelled.
Consider an async method that returns a string value. If the async method is used without await it will return a Deferred string but if await is used you will get a string as the result
The key difference between async and launch:
Deferred returns a particular value of type T after your Coroutine finishes executing, whereas Job doesn’t.
launch / async no result
Use when don't need the result,
Don't block the code where is called,
Run in sequential
async for result
When you need to wait for the result and can run in parallel for
efficiency,
Block the code where is called,
Run in concurrent
Alongside the other great answers, for the people familiar with Rx and getting into coroutines, async returns a Deferred which is akin to Single while launch returns a Job that is more akin to Completable. You can .await() to block and get the value of the first one, and .join() to block until the Job is completed.
I have to make N REST API calls and combine the results of all of them, or fail if at least one of the calls failed (returned an error or a timeout).
I want to use RxJava and I have some requirements:
Be able to configure a retry of each individual api call under some circumstances. I mean, if I have a retry = 2 and I make 3 requests each one has to be retried at most 2 times, with at most 6 requests in total.
Fail fast! If one API calls have failed N times (where the N is the configuration of the retries) it doesn't mater if the remaining requests hasn't ended, I want to return an error.
If I wish to make all the request with a single Thread, I would need an async Http Client, wouldn't?
Thanks.
You could use Zip operator to zip all request together once they ends and check there if all of them were success
private Scheduler scheduler;
private Scheduler scheduler1;
private Scheduler scheduler2;
/**
* Since every observable into the zip is created to subscribeOn a different thread, it´s means all of them will run in parallel.
* By default Rx is not async, only if you explicitly use subscribeOn.
*/
#Test
public void testAsyncZip() {
scheduler = Schedulers.newThread();
scheduler1 = Schedulers.newThread();
scheduler2 = Schedulers.newThread();
long start = System.currentTimeMillis();
Observable.zip(obAsyncString(), obAsyncString1(), obAsyncString2(), (s, s2, s3) -> s.concat(s2)
.concat(s3))
.subscribe(result -> showResult("Async in:", start, result));
}
private Observable<String> obAsyncString() {
return Observable.just("Request1")
.observeOn(scheduler)
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> "Hello");
}
private Observable<String> obAsyncString1() {
return Observable.just("Request2")
.observeOn(scheduler1)
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> " World");
}
private Observable<String> obAsyncString2() {
return Observable.just("Request3")
.observeOn(scheduler2)
.doOnNext(val -> {
System.out.println("Thread " + Thread.currentThread()
.getName());
})
.map(val -> "!");
}
In this example we just concat the results, but instead of do that, you can check the results and do your business logic there.
You can also consider merge or contact also.
you can take a look more examples here https://github.com/politrons/reactive
I would suggest to use an Observable to wrap all the calls.
Let's say you have your function to call the API:
fun restAPIcall(request: Request): Single<HttpResponse>
And you want to call this n times. I am assuming that you want to call them with a list of values:
val valuesToSend: List<Request>
Observable
.fromIterable(valuesToSend)
.flatMapSingle { valueToSend: Request ->
restAPIcall(valueToSend)
}
.toList() // This converts: Observable<Response> -> Single<List<Response>>
.map { responses: List<Response> ->
// Do something with the responses
}
So with this you can call the restAPI from the elements of your list, and have the result as a list.
The other problem is the retries. You said you wanted to retry when an individual cap is reached. This is tricky. I believe there is nothing out of the box in RxJava for this.
You can use retry(n) where you can retry n times in total, but that
is not what you wanted.
There's also a retryWhen { error -> ... } where you can do
something given an exception, but you would know what element throw
the error (unless you add the element to the exception I think).
I have not used the retries before, nevertheless it seems that it retries the whole observable, which is not ideal.
My first approach would be doing something like the following, where you save the count of each element in a dictionary or something like that and only retry if there is not a single element that exceeds your limit. This means that you have to keep a counter and search each time if any of the elements exceeded.
val counter = valuesToSend.toMap()
yourObservable
.map { value: String ->
counter[value] = counter[value]?.let { it + 1 }?: 0 // Update the counter
value // Return again the value so you can use it later for the api call
}
.map { restAPIcall(it) }
// Found a way to take yourObservable and readd the element if it doesn't exceeds
// your limit (maybe in an `onErrorResumeNext` or something). Else throw error
I'm trying to grok how C# 5's new async feature works. Suppose I want to develop an atomic increment function for incrementing an integer in a fictitious IntStore. Multiple calls are made to this function in one thread only.
async void IncrementKey(string key) {
int i = await IntStore.Get(key);
IntStore.Set(key, i+1);
}
It seems to me that this function is flawed. Two calls to IncrementKey could get the same number back from IntStore (say 5), and then set it to 6, thus losing one of the increments?
How could this be re-written, if IntStore.Get is asynchronous (returns Task) in order to work correctly?
Performance is critical, is there a solution that avoids locking?
If you are sure you are calling your function from only one thread, then there shouldn't be any problem, because only one call to IntStore.Get could be awaiting at at time. This because:
await IncrementKey("AAA");
await IncrementKey("BBB");
the second IncrementKey won't be executed until the first IncrementKey has finished. The code will be converted to a state machine. If you don't trust it, change the IntStore.Get(key) to:
async Task<int> IntStore(string str) {
Console.WriteLine("Starting IntStore");
await TaskEx.Delay(10000);
return 0;
}
You'll see that the second Starting IntStore will be written 10 seconds after the first.
To quote from here http://blogs.msdn.com/b/ericlippert/archive/2010/10/29/asynchronous-programming-in-c-5-0-part-two-whence-await.aspx The “await” operator ... means “if the task we are awaiting has not yet completed then sign up the rest of this method as the continuation of that task, and then return to your caller immediately; the task will invoke the continuation when it completes.”