Vertx: How to combine third party events with Standard Verticle and Worker Verticle style event send and reply approach - asynchronous

I am trying to solve an issue whereby I need to combine third party events with the eventBus send and reply approach that Vertx provides for Standard and Worker Verticle setups. I am not sure if what I have laid out below is necessarily the correct approach.
Problem Statement:
I want to have standard verticle that sends a message to a worker verticle.
The worker verticle does some preprocessing and then uses a client method provided by a third party state management lib to publish an even (in an async manner). The result of which is only whether or not the event was successfully received or not (but does not contain any further info around processing etc).
Further processing takes place when the third party state management lib receives the event(this all happens on a separate thread) and a success or failure can occur at which point another event will be published to the cluster management tools output channel.
From the output channel listener I then want to be able to use the event to somehow use the message.reply() on the worker verticle to send back a response to the standard verticle that made the original request, thereby closing the loop of the entire request lifecycle but also using the async approach that vertx is built to use.
Now I conceptually know how to do 90% of what is described here but the missing piece for me is how to coordinate the event on the output channel listener and connect this to the worker verticle so that I can trigger the message.reply.
I have looked at possibly using SharedData and Clustering that Vertx has but was wondering if there is possibly another approach.
I have put a possible example implementation but would really appreciate if anyone has any insights/thoughts into how this can be accomplished and if I am on the right track.
class Order(val id: String)
class OrderCommand(val order: Order) : Serializable {
companion object {
const val name = "CreateOrderCommand"
}
}
class SuccessOrderEvent(val id: String) : Serializable {
companion object {
const val name = "OrderSuccessfulEvent"
}
}
interface StateManagementLib {
fun <T> send(
value: T,
success: Handler<AsyncResult<Int>>,
failure: Handler<AsyncResult<Exception>>
) {
val output = publish(value)
if (output == 1) {
success.handle(Future.succeededFuture())
} else {
failure.handle(Future.failedFuture("Failed"))
}
}
// non-blocking
fun <T> publish(value: T): Int // returns success/failure only
}
class WorkVerticle constructor(private val lib: StateManagementLib) : AbstractVerticle() {
override fun start(startPromise: Promise<Void>) {
workerHandler()
startPromise.complete()
}
private fun workerHandler() {
val consumer = vertx.eventBus().consumer<OrderCommand>(OrderCommand.name)
consumer.handler { message: Message<OrderCommand> ->
try {
vertx.sharedData().getClusterWideMap<String, Message<OrderCommand>>("OrderRequest") { mapIt ->
if (mapIt.succeeded()) {
lib.send(message.body(), {
// The StateManagementLib successfully propagated the event so, we try and store in this map (id -> Message)
mapIt.result().put(message.body().order.id, message)
}, {
message.fail(400, it.result().message)
})
}
}
} catch (e: Exception) {
message.fail(
HttpResponseStatus.INTERNAL_SERVER_ERROR.code(), "Failed to encode data."
)
}
}
// A consumer that will pick up an event that is received from the clusterTool output channel
vertx.eventBus().addInboundInterceptor { context: DeliveryContext<SuccessOrderEvent> ->
// Retrieve cluster map to get the previously stored message and try to respond with Message.reply
// This should go back to the Standard Verticle that sent the message
vertx.sharedData().getClusterWideMap<String, Message<OrderCommand>>("OrderRequest") {
if (it.succeeded()) {
val id = context.message().body().id
val mapResult = it.result().get(id)
it.result().remove(id)
// Try and reply so the original eventloop thread can pickup and respond to calling client
mapResult.result().reply(id)
}
}
}
}
}

Related

What is the correct way of handling collect in Kotlin?

I'm getting user data from Firestore using MVVM. In the repository class I use:
fun getUserData() = flow {
auth.currentUser?.apply {
val user = ref.document(uid).get().await().toObject(User::class.java)
user?.let {
emit(Success(user))
}
}
}. catch { error ->
error.message?.let { message ->
emit(Failure(message))
}
}
This method is called from the ViewModel class:
fun getUser() = repo.getUserData()
And in the activity class I use:
private fun getUser() {
lifecycleScope.launch {
viewModel.getUser().collect { data ->
when(data) {
is Success -> textView.text = data.user.name
is Failure -> print(data.message)
}
}
}
}
To display the name in the TextView. The code works fine. But is this the correct way if doing things? Or is it more correct to collect the data in the ViewModel class?
Any room for improvement? Thanks
My personal opinion is that data should be collected in the VM so it survives configuration changes.
The scope of the view (activity/fragment) should not drive your data flow.
The VM will outlive activities and fragments during configuration changes, so all the data you have collected and transformed, will still be there (or in progress if it's still being obtained).
StateFlow is good (in this last step) because it has the ability to tell the VM: This is no longer needed, don't waste resources.
But I haven't yet used StateFlow in production code so there's that.

Does channel.close() actually stop scheduled tasks by channel.eventLoop().scheduleAtFixedRate()?

I am developing a device control program (Tcp Server) using Netty.
I have to send status request message every 1 second, Therefor I am using the method channel.eventLoop().scheduleAtFixedRate() and it works fine.
channel.eventLoop().scheduleAtFixedRate( () -> {
channel.writeAndFlush("REQ MSG");
System.out.println("REQ MSG");
}, 1000, 1000, TimeUnit.MILLISECONDS);
and when I call channel.close(), the above scheduled task is automatically stopped.
But when I eliminate channel.writeAndFlush() and call channel.close(), the above scheduled task is not stopped.
channel.eventLoop().scheduleAtFixedRate( () -> {
// channel.writeAndFlush("REQ MSG");
System.out.println("REQ MSG");
}, 1000, 1000, TimeUnit.MILLISECONDS);
What's the difference between the two?
There shouldn't be any difference... In both cases you need to explicit cancel the task via future.cancel() once you want to stop it.
I couldn't find the exact cause in my case, and as I advised, I fixed it as follows.
private final CopyOnWriteArrayList<ScheduledFuture<?>> userTaskFutures = new CopyOnWriteArrayList<>();
public void scheduleAtFixedRate(Runnable task, long initialDelay, long period, TimeUnit unit) {
ScheduledFuture<?> future = channel.eventLoop().scheduleAtFixedRate(task, initialDelay, period, unit);
userTaskFutures.add(future);
}
public void disconnect() {
...
if (!userTaskFutures.isEmpty()) {
userTaskFutures.forEach(future -> future.cancel(true));
userTaskFutures.clear();
}
}
Had same situation, my setup: websocket server. When new connection created I've creating ttl handler like this
channel.eventLoop().scheduleWithFixedDelay(
TTLHandler(channel), 0, 10, TimeUnit.SECONDS
)
The code inside TTLHandler make internal logic and then try to drop expireded channel :
channel.deregister()
channel.disconnect()
channel.close()
None of this closing methods stopped eventLoop() and TTLHandler keep going.
My investigation lead me into ScheduledFutureTask.java where DefaultPromise checks cancellation by isCancelled() method for CancellationException
private static boolean isCancelled0(Object result) {
return result instanceof CauseHolder && ((CauseHolder) result).cause instanceof CancellationException;
}
so I directly added into my disconnect section additional
throw CancellationException()
and it works

gRPC client failing with "CANCELLED: io.grpc.Context was cancelled without error"

I have a gRPC server written in C++ and a client written in Java.
Everything was working fine using a blocking stub. Then I decided that I want to change one of the calls to be asynchronous, so I created an additional stub in my client, this one is created with newStub(channel) as opposed to newBlockingStub(channel). I didn't make any changes on the server side. This is a simple unary RPC call.
So I changed
Empty response = blockingStub.callMethod(request);
to
asyncStub.callMethod(request, new StreamObserver<Empty>() {
#Override
public void onNext(Empty response) {
logInfo("asyncStub.callMethod.onNext");
}
#Override
public void onError(Throwable throwable) {
logError("asyncStub.callMethod.onError " + throwable.getMessage());
}
#Override
public void onCompleted() {
logInfo("asyncStub.callMethod.onCompleted");
}
});
Ever since then, onError is called when I use this RPC (Most of the time) and the error it gives is "CANCELLED: io.grpc.Context was cancelled without error". I read about forking Context objects when making an RPC call from within an RPC call, but that's not the case here. Also, the Context seems to be a server side object, I don't see how it relates to the client. Is this a server side error propagating back to the client? On the server side everything seems to complete successfully, so I'm at a loss as to why this is happening. Inserting a 1ms sleep after calling asyncStub.callMethod seems to make this issue go away, but defeats the purpose. Any and all help in understanding this would be greatly appreciated.
Some notes:
The processing time on the server side is around 1 microsecond
Until now, the round trip time for the blocking call was several hundred microseconds (This is the time I'm trying to cut down, as this is essentially a void function, so I don't need to wait for a response)
This method is called multiple times in a row, so before it used to wait until the previous one finished, now they just fire off one after the other.
Some snippets from the proto file:
service EventHandler {
rpc callMethod(Msg) returns (Empty) {}
}
message Msg {
uint64 fieldA = 1;
int32 fieldB = 2;
string fieldC = 3;
string fieldD = 4;
}
message Empty {
}
So it turns out that I was wrong. The context object is used by the client too.
The solution was to do the following:
Context newContext = Context.current().fork();
Context origContext = newContext.attach();
try {
// Call async RPC here
} finally {
newContext.detach(origContext);
}
Hopefully this can help someone else in the future.

Go back to main-thread inside coroutine?

Im running this code since addListenerForSingleEvent is a long Running operation:
CoroutineScope(IO).launch {
userRef.addListenerForSingleValueEvent(object : ValueEventListener {
override fun onCancelled(p0: DatabaseError) {
}
override fun onDataChange(p0: DataSnapshot) {
if (p0.exists()) {
withContext(Main) {
toggleLoading()
val intent = Intent(this#LogInActivity, MainActivity::class.java)
startActivity(intent)
finish()
}
} else{
withContext(Main) {
var addUsernameIntent = Intent(this#LogInActivity,
AddUsernameActivity::class.java)
startActivityForResult(addUsernameIntent, CHOOSE_USERNAME_REQUEST)
}
}
}
})
}
I get an error where i write withContext(Main) that says :
Suspension functions can be called only within coroutine body
But I have a coroutine body right? Before i just had a Thread(runnable {..}) instead of a couroutine, but i read that i shouldn't do intents inside any other Thread than main-thread so i changed to coroutine.
The Firebase client already runs any network and disk I/O operations on a separate thread. There almost never is a need to run addListenerForSingleEvent on a separate thread yourself.
Also see:
Do we need to use background thread for retrieving data using firebase?
Firebase Android: onDataChange() event always executed in Main UI Thread?
Firebase database - run on different thread
Call method in another thread on completion
Function of an anonymous object may capture the variables of the scope, but it is not in enclosing coroutine body. Replace withContext(Main) with creating a new coroutine: <CoroutineScope>.launch(Main).

Read message with a consumer connected to an existing bus

I'm trying to understand how to connect consumers to an existing bus like explained here, but I don't understand how this thing should work
my code:
public interface IMessage
{
string Text { get; set; }
}
public class Program
{
public static void Main()
{
var bus = Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://localhost"), h =>
{
h.Username("guest");
h.Password("guest");
});
});
bus.Start();
bus.ConnectConsumer<TestConsumer>();
bus.Publish<IMessage>(new { Text = "Hi" });
Console.WriteLine("Press any key to exit");
Console.ReadKey();
bus.Stop();
}
}
public class TestConsumer : IConsumer<IMessage>
{
public Task Consume(ConsumeContext<IMessage> context)
{
Console.Out.WriteLineAsync($"Received: {context.Message.Text}");
return Task.CompletedTask;
}
}
I don't understand where the message is sent since aren't there any queue specified, and therefore I don't know how the message can be consumed.
Is there a way to specify on what queue the message will be sent and written OUTSIDE the bus definition?
What am I missing?
What you are publishing here
bus.Publish(new { Text = "Hi" });
is an anonymous type, not IMessage to which you are subscribed, i think you need to create IMessage interface, then create some class implementing IMessage and send over the bus
The docs you linked state that published messages are not received by the temporary queue and it has a limited set of use cases.
As far as I understand there's a distinction between senders and consumers in Mass Transit.
Senders send messages to an exchange not to queues.
Consumers are responsible for creating a queue and binding this queue to an exchange. The queue-exchange binding is done auto-magically based on the type of the message.
For this reason it's important that your consumers are created first. If the a message is sent without a consumer having created a queue the message has nowhere to go and will be lost.
Instead of using:
bus.ConnectConsumer<TestConsumer>();
Consider using:
sbc.ReceiveEndpoint("queue-name", e => e.Consumer<TestConsumer>());
That will create a queue named "queue-name" and connect your consumer to it.
If you need to connect a receive endpoint to the bus after the fact, you can use:
host.ConnectReceiveEndpoint("queue-name", e => e.Consumer<TestConsumer>());
You'll need to save the host from the configuration, but it will allow you to add the receive endpoint after the bus is started.

Resources