I am trying to understand which is the best way to have an asynchronous job fired at a scheduled rate in Kotlin, while the application is normally running it's normal tasks. Let's say I have a simple application that only prints out "..." every second, but every 5 seconds I want another job / thread / coroutine (which ever suits best) to print "you have a message!". For the async job I have a class NotificationProducer and it looks like this.
class NotificationProducer {
fun produce() {
println("You have a message!")
}
}
Then, my main method looks like this.
while (true) {
println("...")
sleep(1000)
}
Should I use GlobalScope.async, Timer().schedule(...) or some Quartz job to achieve what I want? Any advice is highly appreciated. The point is that notification must come from another class (e.g. NotificationProducer)
If I correctly understand the issue, using Kotlin Coroutines you can implement it as the following:
class Presenter : CoroutineScope { // implement CoroutineScope to create local scope
private var job: Job = Job()
override val coroutineContext: CoroutineContext
get() = Dispatchers.Default + job
// this method will help to stop execution of a coroutine.
// Call it to cancel coroutine and to break the while loop defined in the coroutine below
fun cancel() {
job.cancel()
}
fun schedule() = launch { // launching the coroutine
var seconds = 1
val producer = NotificationProducer()
while (true) {
println("...")
delay(1000)
if (seconds++ == 5) {
producer.produce()
seconds = 1
}
}
}
}
Then you can use an instance of the Presenter class to launch the coroutine and stop it:
val presenter = Presenter()
presenter.schedule() // calling `schedule()` function launches the coroutine
//...
presenter.cancel() // cancel the coroutine when you need
For simple scheduling requirements, you can consider using coroutines:
class NotificationProducerScheduler(val service: NotificationProducer, val interval: Long, val initialDelay: Long?) :
CoroutineScope {
private val job = Job()
private val singleThreadExecutor = Executors.newSingleThreadExecutor()
override val coroutineContext: CoroutineContext
get() = job + singleThreadExecutor.asCoroutineDispatcher()
fun stop() {
job.cancel()
singleThreadExecutor.shutdown()
}
fun start() = launch {
initialDelay?.let {
delay(it)
}
while (isActive) {
service.produce()
delay(interval)
}
println("coroutine done")
}
}
Otherwise, the Java concurrency API is pretty solid too:
class NotificationProducerSchedulerJavaScheduler(
val service: NotificationProducer,
val interval: Long,
val initialDelay: Long = 0
) {
private val scheduler = Executors.newScheduledThreadPool(1)
private val task = Runnable { service.produce() }
fun stop() {
scheduler.shutdown()
}
fun start() {
scheduler.scheduleWithFixedDelay(task, initialDelay, interval, TimeUnit.MILLISECONDS)
}
}
This function will run a task in the background while proceeding with a "main" task that controls the lifecycle of the background job. Below is an example of usage.
/**
* Runs a task in the background in IO while the op proceeds.
* The job is canceled when op returns.
* This is useful for updating caches and the like.
*/
suspend fun withBackgroundTask(task: suspend () -> Unit, op: suspend () -> Unit) {
val job = CoroutineScope(Dispatchers.IO).launch { task() }
try {
op()
} finally {
job.cancel()
}
}
/**
* Updates the cache in a background task while op runs.
*/
suspend fun withCache(cache: Cache<*>, op: suspend () -> Unit) {
suspend fun cacheUpdate() {
cache.fetchInternal()
while (true) {
delay(cache.cycle)
cache.fetchInternal()
}
}
withBackgroundTask(::cacheUpdate, op)
}
Related
firebase method is working on worker thread automatically. but I have used coroutine and callbackflow to implement firebase listener code synchronously or get return from the listener.
below is my code that I explained
coroutine await with firebase for one shot
override suspend fun checkNickName(nickName: String): Results<Int> {
lateinit var result : Results<Int>
fireStore.collection("database")
.document("user")
.get()
.addOnCompleteListener { document ->
if (document.isSuccessful) {
val list = document.result.data?.get("nickNameList") as List<String>
if (list.contains(nickName))
result = Results.Exist(1)
else
result = Results.No(0)
//document.getResult().get("nickNameList")
}
else {
}
}.await()
return result
}
callbackflow with firebase listener
override fun getOwnUser(): Flow<UserEntity> = callbackFlow{
val document = fireStore.collection("database/user/userList/")
.document("test!!!!!")
val subscription = document.addSnapshotListener { snapshot,_ ->
if (snapshot!!.exists()) {
val ownUser = snapshot.toObject<UserEntity>()
if (ownUser != null) {
trySend(ownUser)
}
}
}
awaitClose { subscription.remove() }
}
so I really wonder these way is good or bad practice and its reason
Do not combine addOnCompleteListener with coroutines await(). There is no guarantee that the listener gets called before or after await(), so it is possible the code in the listener won't be called until after the whole suspend function returns. Also, one of the major reasons to use coroutines in the first place is to avoid using callbacks. So your first function should look like:
override suspend fun checkNickName(nickName: String): Results<Int> {
try {
val userList = fireStore.collection("database")
.document("user")
.get()
.await()
.get("nickNameList") as List<String>
return if (userList.contains(nickName)) Results.Exist(1) else Results.No(0)
} catch (e: Exception) {
// return a failure result here
}
}
Your use of callbackFlow looks fine, except you should add a buffer() call to the flow you're returning so you can specify how to handle backpressure. However, it's possible you will want to handle that downstream instead.
override fun getOwnUser(): Flow<UserEntity> = callbackFlow {
//...
}.buffer(/* Customize backpressure behavior here */)
I'm trying to run a Firebase Transaction under a suspended function in Kotlin and i see no documentation about it.
I'm using
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-play-services:1.5.2'
for coroutines with firebase (eg: setValue(*).await() ) but there seems to be no await function for runTransaction(*)
override suspend fun modifyProductStock(
product: ProductModel,
valueToModify: Long,
replace: Boolean
) {
CoroutineScope(Dispatchers.Main).launch {
val restaurantId = authRepository.restaurantId.value ?: throw Exception("No restaurant!")
val productId = product.id ?: throw Exception("No Product ID!")
val reference = FirebaseDatabase.getInstance().getReference("database/$restaurantId").child("products")
if (replace) {
reference.child(productId).child("stock").setValue(valueToModify).await()
} else {
reference.child(productId).child("stock")
.runTransaction(object : Transaction.Handler {
override fun doTransaction(p0: MutableData): Transaction.Result {
//any operation
return Transaction.success(p0)
}
override fun onComplete(p0: DatabaseError?, p1: Boolean, p2: DataSnapshot?) {
}
})
}
}
}
You could wrap it in suspendCoroutine:
val result: DataSnapshot? = suspendCoroutine { c ->
reference.child(productId).child("stock")
.runTransaction(object : Transaction.Handler {
override fun doTransaction(p0: MutableData): Transaction.Result {
//any operation
return Transaction.success(p0)
}
override fun onComplete(error: DatabaseError?, p1: Boolean, snapshot: DataSnapshot?) {
c.resume(snapshot)
}
})
}
suspendCoroutine
Obtains the current continuation instance inside suspend functions and suspends the currently running coroutine.
In this function both Continuation.resume and Continuation.resumeWithException can be used either synchronously in the same stack-frame where the suspension function is run or asynchronously later in the same thread or from a different thread of execution.
Given that the Kotlin example in the Firebase documentation on transactions uses the same callback style that you have, it seems indeed that there is no specific support for co-routines there.
It might be worth posting an issue on the Android SDK repo to get it added, or hear why it wasn't added.
okay, so I have a controller method which need to make a bunch of soap call to an external service, each one quite heavy. I am trying to do these one in parralel to save some time, but unless I build the async calls from GlobalScope, the deferred are resolved in sequence. Let me show you.
executing the following code
#ResponseBody
#GetMapping(path = ["/buildSoapCall"])
fun searchStations(): String = runBlocking {
var travels: List<Travel> = service.getTravels().take(500)
val deferred = travels
.map {
async() {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
}
println("Finished deferred")
val callResults = deferred.awaitAll()
println("Finished Awaiting")
""
}
get me the following console message :
Finished deferred
START-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-END.....
the - is printed by executeSoapCall
As you can see, the deferred are called in sequence.
But if I use GlobalScope, like this :
#ResponseBody
#GetMapping(path = ["/buildSoapCall"])
fun searchStations(): String = runBlocking {
var travels: List<Travel> = service.getTravels().take(500)
val deferred = travels
.map {
GlobalScope.async() {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
}
println("Finished deferred")
val callResults = deferred.awaitAll()
println("Finished Awaiting")
""
}
I get the following console message :
Finished Treating
STARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTFinished deferred
START-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART--ENDENDSTARTSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-END...START-END-END-END-END-END-END-END-END-END-END-END-ENDFinished Awaiting
showing that the Deferred are all starting in parallel. In addition, the treatment time is quite shorter.
I don't really understand why I have this behaviour.
Your call to service.executeSoapCall blocks the thread runBlocking coroutine is running on. You need to start async coroutine on a different thread everytime to get a concurrent behavior. You can achieve that by using a threadpool, e.g., Dispatchers.IO:
...
async(Dispatchers.IO) {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
...
or creating a new thread on every call:
...
async(newSingleThreadContext("MyThread")) {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
...
GlobalScope works because it uses a ThreadPool by default but you should avoid using it. You can read this article by Roman Elizarov about that topic.
I did some tests which compare speed of using async as a method of deferring results and CompletableDeferred with combination of Job or startCoroutine to do the same job.
In summary there are 3 use cases:
async with default type of start (right away) [async]
CompletableDeferred + launch (basically Job) [cdl]
CompletableDeferred + startCoroutine [ccdl]
results are presented here:
In short every iteration of each use case test generates 10000 of async / cdl / ccdl requests and waits for them to complete. This is repeated 225 times with 25 times as a warmUp (not included in results) and data points are collected over 100 iteration of process above (as min, max, avg).
here is a code:
import com.source.log.log
import kotlinx.coroutines.*
import kotlin.coroutines.Continuation
import kotlin.coroutines.startCoroutine
import kotlin.system.measureNanoTime
import kotlin.system.measureTimeMillis
/**
* #project Bricks
* #author SourceOne on 28.11.2019
*/
/*I know that there are better ways to benchmark speed
* but given the produced results this method is fine enough
* */
fun benchmark(warmUp: Int, repeat: Int, action: suspend () -> Unit): Pair<List<Long>, List<Long>> {
val warmUpResults = List(warmUp) {
measureNanoTime {
runBlocking {
action()
}
}
}
val benchmarkResults = List(repeat) {
measureNanoTime {
runBlocking {
action()
}
}
}
return warmUpResults to benchmarkResults
}
/* find way to cancel startedCoroutine when deferred is
* canceled (currently you have to cancel whole context)
* */
fun <T> CoroutineScope.completable(provider: suspend () -> T): Deferred<T> {
return CompletableDeferred<T>().also { completable ->
provider.startCoroutine(
Continuation(coroutineContext) { result ->
completable.completeWith(result)
}
)
}
}
suspend fun calculateAsyncStep() = coroutineScope {
val list = List(10000) {
async { "i'm a robot" }
}
awaitAll(*list.toTypedArray())
}
suspend fun calculateCDLStep() = coroutineScope {
val list = List(10000) {
CompletableDeferred<String>().also {
launch {
it.complete("i'm a robot")
}
}
}
awaitAll(*list.toTypedArray())
}
suspend fun calculateCCDLStep() = coroutineScope {
val list = List(10000) {
completable { "i'm a robot" }
}
awaitAll(*list.toTypedArray())
}
fun main() {
val labels = listOf("async", "cdl", "ccdl")
val collectedResults = listOf(
mutableListOf<Pair<List<Long>, List<Long>>>(),
mutableListOf(),
mutableListOf()
)
"stabilizing runs".log()
repeat(2) {
println("async $it")
benchmark(warmUp = 25, repeat = 200) {
calculateAsyncStep()
}
println("CDL $it")
benchmark(warmUp = 25, repeat = 200) {
calculateCDLStep()
}
println("CCDL $it")
benchmark(warmUp = 25, repeat = 200) {
calculateCCDLStep()
}
}
"\n#Benchmark start".log()
val benchmarkTime = measureTimeMillis {
repeat(100) {
println("async $it")
collectedResults[0] += benchmark(warmUp = 25, repeat = 200) {
calculateAsyncStep()
}
println("CDL $it")
collectedResults[1] += benchmark(warmUp = 25, repeat = 200) {
calculateCDLStep()
}
println("CCDL $it")
collectedResults[2] += benchmark(warmUp = 25, repeat = 200) {
calculateCCDLStep()
}
}
}
"\n#Benchmark completed in ${benchmarkTime}ms".log()
"#Benchmark results:".log()
val minMaxAvg = collectedResults.map { stageResults ->
stageResults.map { (_, benchmark) ->
arrayOf(
benchmark.minBy { it }!!, benchmark.maxBy { it }!!, benchmark.average().toLong()
)
}
}
minMaxAvg.forEachIndexed { index, list ->
"results for: ${labels[index]} [min, max, avg]".log()
list.forEach { results ->
"${results[0]}\t${results[1]}\t${results[2]}".log()
}
}
}
There is no surprise that the first two use cases (async and cdl) are very close to each other and async is always better (because you don't have the overhead of creating job to complete deferred object) but comparing async vs CompletableDeferred + startCoroutine there is a huge gap between them (almost 2 times) in favor of the last one. Why there is such a big difference and if anyone knows, why shouldn't we just be using CompletableDeferred + startCoroutine wrapper (like completable() here) instead of async?
Addition1:
Here is a sample for 1000 points:
There are constant spikes in async and cdl results and some in ccdl (maybe gc?) but still there is far less with ccdl. I will rerun these tests with changed order of tests interleaving but it seems that it's related to something under the coroutines machinery.
Edit1:
I've accepted Marko Topolnik answer, but in addition to it, you still can use this 'as he called' bare launch method if you await for the result within the scope you have launched it.
In example if you will launch few deffered coroutines (async) and at the end of that scope you will await them all then the ccdl method will work as expected (at least from what i've seen in my tests).
Since launch and async are built as a layer on top of the low-level primitive createCoroutineUnintercepted(), whereas startCoroutine is practically a direct call into it, there aren't any surprises in your benchmark results.
why shouldn't we just be using CompletableDeferred + startCoroutine wrapper (like completable() here) instead of async?
A comment in your code already hints to the answer:
/*
* find way to cancel startedCoroutine when deferred is
* canceled (currently you have to cancel whole context)
*/
The layer you short-circuited with startCoroutine is precisely the layer that handles things as cancellation, coroutine hierarchy, exception handling and propagation, and so on.
Here's a simple example that shows you one of the things that break when you replace launch with a bare coroutine:
fun main() = runBlocking {
bareLaunch {
try {
delay(1000)
println("Coroutine done")
} catch (e: CancellationException) {
println("Coroutine cancelled, the exception is: $e")
}
}
delay(10)
}
fun CoroutineScope.bareLaunch(block: suspend () -> Unit) =
block.startCoroutine(Continuation(coroutineContext) { Unit })
fun <T> CoroutineScope.bareAsync(block: suspend () -> T) =
CompletableDeferred<T>().also { deferred ->
block.startCoroutine(Continuation(coroutineContext) { result ->
result.exceptionOrNull()?.also {
deferred.completeExceptionally(it)
} ?: run {
deferred.complete(result.getOrThrow())
}
})
}
When you run this, you'll see the bare coroutine got cancelled after 10 milliseconds. The runBlocking builder didn't realize it had to wait for it to complete. If you replace bareLaunch { with launch {, you'll restore the designed behavior where the child coroutine completes normally. The same thing happens with bareAsync.
In SBT, I fork a Java process with:
class FilteredOutput extends FilterOutputStream(System.out) {
var buf = ArrayBuffer[Byte]()
override def write(b: Int) {
buf.append(b.toByte)
if (b == '\n'.toInt)
flush()
}
override def flush(){
if (buf.nonEmpty) {
val arr = buf.toArray
val txt = try new String(arr, "UTF-8") catch { case NonFatal(ex) ⇒ "" }
if (!txt.startsWith("pydev debugger: Unable to find real location for"))
out.write(arr)
buf.clear()
}
super.flush()
}
}
var process = Option.empty[Process]
process = Some(Fork.java.fork(ForkOptions(outputStrategy = new FilteredOutput()), Seq("my.company.MyClass")))
as a result of a custom task.
Later on, I terminate it with:
process.map { p =>
log info "Killing process"
p.destroy()
}
by means of another custom task.
The result is that SBT doesn't accept more input and gets blocked. Ctrl+C is the only way of restoring control back, but SBT dies as a consequence.
The problem has to do with the custom output strategy, that filters some annoying messages.
With jstack I haven't seen any deadlock.
SBT version 0.13.9.
The solution is to avoid closing System.out:
class FilteredOutput extends FilterOutputStream(System.out) {
var buf = ArrayBuffer[Byte]()
override def write(b: Int) {
...
}
override def flush(){
...
}
override def close() {}
}