In SBT, I fork a Java process with:
class FilteredOutput extends FilterOutputStream(System.out) {
var buf = ArrayBuffer[Byte]()
override def write(b: Int) {
buf.append(b.toByte)
if (b == '\n'.toInt)
flush()
}
override def flush(){
if (buf.nonEmpty) {
val arr = buf.toArray
val txt = try new String(arr, "UTF-8") catch { case NonFatal(ex) ⇒ "" }
if (!txt.startsWith("pydev debugger: Unable to find real location for"))
out.write(arr)
buf.clear()
}
super.flush()
}
}
var process = Option.empty[Process]
process = Some(Fork.java.fork(ForkOptions(outputStrategy = new FilteredOutput()), Seq("my.company.MyClass")))
as a result of a custom task.
Later on, I terminate it with:
process.map { p =>
log info "Killing process"
p.destroy()
}
by means of another custom task.
The result is that SBT doesn't accept more input and gets blocked. Ctrl+C is the only way of restoring control back, but SBT dies as a consequence.
The problem has to do with the custom output strategy, that filters some annoying messages.
With jstack I haven't seen any deadlock.
SBT version 0.13.9.
The solution is to avoid closing System.out:
class FilteredOutput extends FilterOutputStream(System.out) {
var buf = ArrayBuffer[Byte]()
override def write(b: Int) {
...
}
override def flush(){
...
}
override def close() {}
}
Related
I'm trying to run a Firebase Transaction under a suspended function in Kotlin and i see no documentation about it.
I'm using
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-play-services:1.5.2'
for coroutines with firebase (eg: setValue(*).await() ) but there seems to be no await function for runTransaction(*)
override suspend fun modifyProductStock(
product: ProductModel,
valueToModify: Long,
replace: Boolean
) {
CoroutineScope(Dispatchers.Main).launch {
val restaurantId = authRepository.restaurantId.value ?: throw Exception("No restaurant!")
val productId = product.id ?: throw Exception("No Product ID!")
val reference = FirebaseDatabase.getInstance().getReference("database/$restaurantId").child("products")
if (replace) {
reference.child(productId).child("stock").setValue(valueToModify).await()
} else {
reference.child(productId).child("stock")
.runTransaction(object : Transaction.Handler {
override fun doTransaction(p0: MutableData): Transaction.Result {
//any operation
return Transaction.success(p0)
}
override fun onComplete(p0: DatabaseError?, p1: Boolean, p2: DataSnapshot?) {
}
})
}
}
}
You could wrap it in suspendCoroutine:
val result: DataSnapshot? = suspendCoroutine { c ->
reference.child(productId).child("stock")
.runTransaction(object : Transaction.Handler {
override fun doTransaction(p0: MutableData): Transaction.Result {
//any operation
return Transaction.success(p0)
}
override fun onComplete(error: DatabaseError?, p1: Boolean, snapshot: DataSnapshot?) {
c.resume(snapshot)
}
})
}
suspendCoroutine
Obtains the current continuation instance inside suspend functions and suspends the currently running coroutine.
In this function both Continuation.resume and Continuation.resumeWithException can be used either synchronously in the same stack-frame where the suspension function is run or asynchronously later in the same thread or from a different thread of execution.
Given that the Kotlin example in the Firebase documentation on transactions uses the same callback style that you have, it seems indeed that there is no specific support for co-routines there.
It might be worth posting an issue on the Android SDK repo to get it added, or hear why it wasn't added.
okay, so I have a controller method which need to make a bunch of soap call to an external service, each one quite heavy. I am trying to do these one in parralel to save some time, but unless I build the async calls from GlobalScope, the deferred are resolved in sequence. Let me show you.
executing the following code
#ResponseBody
#GetMapping(path = ["/buildSoapCall"])
fun searchStations(): String = runBlocking {
var travels: List<Travel> = service.getTravels().take(500)
val deferred = travels
.map {
async() {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
}
println("Finished deferred")
val callResults = deferred.awaitAll()
println("Finished Awaiting")
""
}
get me the following console message :
Finished deferred
START-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-END.....
the - is printed by executeSoapCall
As you can see, the deferred are called in sequence.
But if I use GlobalScope, like this :
#ResponseBody
#GetMapping(path = ["/buildSoapCall"])
fun searchStations(): String = runBlocking {
var travels: List<Travel> = service.getTravels().take(500)
val deferred = travels
.map {
GlobalScope.async() {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
}
println("Finished deferred")
val callResults = deferred.awaitAll()
println("Finished Awaiting")
""
}
I get the following console message :
Finished Treating
STARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTSTARTFinished deferred
START-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART--ENDENDSTARTSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-ENDSTART-END...START-END-END-END-END-END-END-END-END-END-END-END-ENDFinished Awaiting
showing that the Deferred are all starting in parallel. In addition, the treatment time is quite shorter.
I don't really understand why I have this behaviour.
Your call to service.executeSoapCall blocks the thread runBlocking coroutine is running on. You need to start async coroutine on a different thread everytime to get a concurrent behavior. You can achieve that by using a threadpool, e.g., Dispatchers.IO:
...
async(Dispatchers.IO) {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
...
or creating a new thread on every call:
...
async(newSingleThreadContext("MyThread")) {
print("START")
val result = service.executeSoapCall(it)
print("END")
result
}
...
GlobalScope works because it uses a ThreadPool by default but you should avoid using it. You can read this article by Roman Elizarov about that topic.
I am trying to understand which is the best way to have an asynchronous job fired at a scheduled rate in Kotlin, while the application is normally running it's normal tasks. Let's say I have a simple application that only prints out "..." every second, but every 5 seconds I want another job / thread / coroutine (which ever suits best) to print "you have a message!". For the async job I have a class NotificationProducer and it looks like this.
class NotificationProducer {
fun produce() {
println("You have a message!")
}
}
Then, my main method looks like this.
while (true) {
println("...")
sleep(1000)
}
Should I use GlobalScope.async, Timer().schedule(...) or some Quartz job to achieve what I want? Any advice is highly appreciated. The point is that notification must come from another class (e.g. NotificationProducer)
If I correctly understand the issue, using Kotlin Coroutines you can implement it as the following:
class Presenter : CoroutineScope { // implement CoroutineScope to create local scope
private var job: Job = Job()
override val coroutineContext: CoroutineContext
get() = Dispatchers.Default + job
// this method will help to stop execution of a coroutine.
// Call it to cancel coroutine and to break the while loop defined in the coroutine below
fun cancel() {
job.cancel()
}
fun schedule() = launch { // launching the coroutine
var seconds = 1
val producer = NotificationProducer()
while (true) {
println("...")
delay(1000)
if (seconds++ == 5) {
producer.produce()
seconds = 1
}
}
}
}
Then you can use an instance of the Presenter class to launch the coroutine and stop it:
val presenter = Presenter()
presenter.schedule() // calling `schedule()` function launches the coroutine
//...
presenter.cancel() // cancel the coroutine when you need
For simple scheduling requirements, you can consider using coroutines:
class NotificationProducerScheduler(val service: NotificationProducer, val interval: Long, val initialDelay: Long?) :
CoroutineScope {
private val job = Job()
private val singleThreadExecutor = Executors.newSingleThreadExecutor()
override val coroutineContext: CoroutineContext
get() = job + singleThreadExecutor.asCoroutineDispatcher()
fun stop() {
job.cancel()
singleThreadExecutor.shutdown()
}
fun start() = launch {
initialDelay?.let {
delay(it)
}
while (isActive) {
service.produce()
delay(interval)
}
println("coroutine done")
}
}
Otherwise, the Java concurrency API is pretty solid too:
class NotificationProducerSchedulerJavaScheduler(
val service: NotificationProducer,
val interval: Long,
val initialDelay: Long = 0
) {
private val scheduler = Executors.newScheduledThreadPool(1)
private val task = Runnable { service.produce() }
fun stop() {
scheduler.shutdown()
}
fun start() {
scheduler.scheduleWithFixedDelay(task, initialDelay, interval, TimeUnit.MILLISECONDS)
}
}
This function will run a task in the background while proceeding with a "main" task that controls the lifecycle of the background job. Below is an example of usage.
/**
* Runs a task in the background in IO while the op proceeds.
* The job is canceled when op returns.
* This is useful for updating caches and the like.
*/
suspend fun withBackgroundTask(task: suspend () -> Unit, op: suspend () -> Unit) {
val job = CoroutineScope(Dispatchers.IO).launch { task() }
try {
op()
} finally {
job.cancel()
}
}
/**
* Updates the cache in a background task while op runs.
*/
suspend fun withCache(cache: Cache<*>, op: suspend () -> Unit) {
suspend fun cacheUpdate() {
cache.fetchInternal()
while (true) {
delay(cache.cycle)
cache.fetchInternal()
}
}
withBackgroundTask(::cacheUpdate, op)
}
I need to asynchronously fetch cats, dogs and mice and then do some post-processing. Here is something what I am doing:
Promise<List<Cat>> fetchCats = task {}
Promise<List<Mouse>> fetchMice = task { }
Promise<List<Dog>> fetchDogs = task {}
List promiseList = [fetchCats, fetchMice, fetchDogs]
List results = Promises.waitAll(promiseList)
The problem I am facing is, order of items in list results is not fixed, i.e. in one execution results can be [cats, dogs, mice] and in other execution, results can be [dogs, mice, cats].
Which means to access cats I need to explicitly check type of element of results, and similarly for dogs, and mice which makes my code look bad.
Upon going through documentation here, I found PromiseMap API can help me as it provides a pretty way of accessing results through key-value pairs. Here is what it offers:
import grails.async.*
def map = new PromiseMap()
map['one'] = { 2 * 2 }
map['two'] = { 4 * 4 }
map['three'] = { 8 * 8 }
map.onComplete { Map results ->
assert [one:4,two:16,three:64] == results
}
Though PromiseMap has an onComplete method, but it does not make current thread wait for all the promises to finish.
Using PromiseMap, how can I block the current thread till all the promises get finished?
If you are only concern about current thread to wait until PromiseMap complete, can use Thread : join()
import grails.async.*
def map = new PromiseMap()
map['one'] = { println "task one" }
map['two'] = { println "task two" }
map['three'] = { println "task three" }
Thread t = new Thread() {
public void run() {
println("pausing the current thread, let promiseMap complete first")
map.onComplete { Map results ->
println("Promisemap processing : " + results)
}
}
}
t.start()
t.join()
println("\n CurrentThread : I can won the race if you just comment t.join() line in code")
Use .get()
From the PromiseMap source:
/**
* Synchronously return the populated map with all values obtained from promises used
* inside the populated map
*
* #return A map where the values are obtained from the promises
*/
Map<K, V> get() throws Throwable {
I have a weird problem with a ChatServer program I am working on (don't know why I suddenly started it but I want to finish it). First, here is the relevant code:
sealed trait ServerMessage
case class Message(msg: String) extends ServerMessage
case object Quit extends ServerMessage
sealed trait ClientMessage
case class Incoming(conn: Connection, msg: String) extends ClientMessage
case class Remove(conn: Connection) extends ClientMessage
object Server extends App with Actor with Settings {
Console.println(greeting)
Console.println("Server starting up...")
val socket = new ServerSocket(defaultPort);
var connections: Set[Connection] = Set.empty
start
actor {
loop {
val s = socket.accept
val c = Connection(s)
Console.println("New Connection from " + s.getInetAddress)
c ! Message(greeting)
connections += c
}
}
def act = loop {
receive {
// For some reason, this only works once
case Incoming(conn, msg) => {
Console.println(conn.socket.getInetAddress.toString + " said: " + msg)
connections.foreach(_ ! Message(msg))
}
case Remove(conn) => connections -= conn; conn ! Quit
}
}
}
case class Connection(socket: Socket) extends Actor {
val in = new BufferedReader(new InputStreamReader(socket.getInputStream))
val out = new PrintWriter(socket.getOutputStream)
start
actor {
var s: String = in.readLine
while (s != null) {
// This output works
scala.Console.println(s)
if (s == "quit") Server ! Remove(this)
else Server ! Incoming(this, s)
s = in.readLine
}
}
def act = {
var done = false
while (!done) {
receive {
// This seems to work all the time (I can send several messages)
case Message(str) => out.println(str); out.flush
case Quit => done = true
}
}
in.close
out.close
socket.close
}
}
The problem I have is when I connect to it via telnet, I can send 1 message, and it comes back to me. But after that, when I send more messages, they don't come back to me. With the help of the debug messages I can identify where the problem is, but I can't see at all why it doesn't work.
Maybe someone can give me a hint? This is the first time I use actors in such a complex way.
EDIT: Could it have to do with the fact that the receive or react functions will never return?
Try replacing receive with react.