from what I can comprehend briefly, both of them processing flow asynchronously with VM scope using more resource as it create new context, separate properties and variables. Any particular reason other than that if the use is just to process the flow asynchronously?
Async is a scope that is executed immediately in parallel with respect to the flow, if there are resources available (ie threads). VM is a connector that implements an in-memory queue. I usually recommend to prefer to use the VM connector because with Async if there are no threads available it can fail to execute. With the VM connector the messages will be queued until the flow that reads from the VM queue is able to read the next message. Note that if the number of messages queued is greater than the number of messages processed it will run out of memory or exceed the queue allocation, causing another error.
Always remember that threads are a limited resource. In Mule it is not possible to control the number of threads used, only the concurrency. Also keep in mind that threads are not free, they consume memory and CPU.
Related
We have a bus reservation system running in GKE in which we are handling the creation of such reservations with different threads. Due to that, CRUD java methods can sometimes run simultaneously referring to the same bus, resulting in the save in our DB of the LAST simultaneous update only (so the other simultaneous updates are lost).
Even if the probabilities are low (the simultaneous updates need to be really close, 1-2 seconds), we need to avoid this. My question is about how to address the solution:
Lock the bus object and return error to the other simultaneous requests
In-memory map or Redis caché to track the bus requests
Use GCP Pub/Sub, Kafka or RabbitMQ as a queue system.
Try to focus the efforts on reducing the simultaneous time window (reduce from 1-2 seconds up to milliseconds)
Others?
Also, we are worried if in the future the GKE requests handling scalability may be an issue. If we manage a relatively higher number of buses, should we need to implement a queue system between the client and the server? Or GKE load balancer & ambassador will already manages it for us? In case we need a queue system in the future, could it be used also for the collision problem we are facing now?
Last, the reservation requests from the client often takes a while. Therefore, we are changing the requests to be handled asynchronously with a long polling approach from the client to know the task status. Could we link this solution to the current problem? For example, using the Redis caché or the queue system to know the task status? Or should we try to keep the requests synchronous and focus on reducing the processing time (it may be quite difficult).
An app makes 3 simultaneous HTTP requests to web server. using asynchronus technique, how many worker threads will be tied up waiting for the data
It wouldn't tied up, Because when you’re doing asynchronous work, you’re not always using a thread.
For example, if you made an async web service request, your client will not be using any threads between the “send” and “receive”.
You unwind after the “send”, and the “receive” occurs on an I/O completion port, at which time your callback is invoked and you will then be using a thread again. (Note that for this scenario your callback is executed on an i/o thread ASP.NET only uses worker threads, and only includes worker threads when it counts the threads in-use.)
Is there a standard name for this kind of design. and any existing frameworks in .net to make use of.
Multiple Process.exe are running in the server. There will be a ProcessPoolManager which is responsible for spawning these exes on a need basis. client(s) will send tasks to a queue. PoolManager reads the tasks queue and have them processed in the invoked process.exe and put response back into the queue. and client will get async response from queue when there is a response available.
Do you know what kind of design is this. and how to achieve it with any existing frameworks.
I understood that .net know to use multiple threads for multiple requests.
So, if probably our service wont get more request than the number of threads our server can produce (it look like huge number), the only reason I can see to use async is on single request that do multiple blocking operations which can done in parallel.
Am I right?
Another advantage may be that serve multiple requests with same thread is cheaper than use multiple threads. How significant is this difference?
(note: no UI exists in our service (I saw that there is single thread for this, but it isn't relevant))
thanks!
Am I right?
No, doing multiple independent blocking operations, is the job of Concurrent APIs anyway (though sometimes they need Synchronization (like lock, mutex) to maintain the object state and avoid Race condition), but the usage of Async-Await is to schedule the IO Operations, like File Read / Write, call a remote service or Database Read / Write, which doesn't need a thread, as they are queued on a queue in hardware called IO Completion ports.
Benefits of Async-Await:
Doesn't start a IO operation on a separate Thread, since Thread is a costly resource, in terms memory and resource allocation and would do little precious than wait for IO call to come back. Separate thread shall be used for the compute bound operations, no IO bound.
Free up the UI / caller thread to make it completely responsive to carry out other tasks / operations
This is the evolution of Asynchronous programming model (BeginXX, EndXX), which was fairly complex to understand and implement
Another advantage may be that serve multiple requests with same thread is cheaper than use multiple threads. How significant is this difference?
Its a good strategy depending on the kind of request from caller, if they are compute bound better invoke a Parallel API and finish them fast, IO bound there's Async-Await, only issue with multiple threads is Resource allocation and Context switching, which needs to be factored in, but on other end it efficiently utilize the processor cores, which are fairly under utilized in the current day systems, as you would see most of the time processor is lying idle
I need to process about 250.000 documents per day with an EJB 3.1 asynchronous method in order to face an overall long time task.
I do this to use more threads and process more documents concurrently. Here's an example in pseudo code:
// this returns about 250.000 documents per day
List<Document> documentList = Persistence.listDocumentsToProcess();
for(Document currentDocument: documentList){
//this is the asynchronous call
ejbInstance.processAsynchronously(currentDocument);
}
Suppose I have a thread pool of size 10 and 4 core processors, my questions are:
how many documents will the application server process SIMULTANEOUSLY?
what happen when all thread in pool are processing a documents and one more asynchronous call comes? Will this work like a sort of JMS Queue?
would I have any improvement adopting a JMS Queue solution
I work with Java EE 6 and WebSphere 8.5.5.2
The default configuration for asynchronous EJB method calls is as follows (from the infocenter):
The EJB container work manager has the following thread pool settings:
Minimum number of threads = 1
Maximum number of threads = 5
Work request queue size = 0 work objects
Work request queue full action = Block
Remote Future object duration = 86400 seconds
So trying to answer your questions:
how many documents will the application server process SIMULTANEOUSLY? (assuming 10 size thread pool)
This thread pool is for all EJB async calls, so first you need to assume that your application is the only one using EJB async calls. Then you will potentially have 10 runnable instances, that will be processed in parallel. Whether they will be processed concurrently depends on the number of cores/threads available in the system, so you cant have accurate number (some cores/threads may be doing web work for example, or other process using cpu).
what happen when all thread in pool are processing a documents and one more asynchronous call comes?
It depends on the Work request queue size and Work request queue full action, settings. If there are no available threads in the pool, then requests will be queued till the queue size is reached. Then it depends on the action, which might be Block or Fail.
would I have any improvement adopting a JMS Queue solution
Depends on your needs. Here are some pros/cons JMS solution.
Pros:
Persistence - if using JMS your asynchronous task can be persistent, so in case of the server failure you will not lost them, and will be processed after restart or by other cluster member. EJB async queue is held only in memory, so tasks in queue are lost in case of failure.
Scalability - if you put tasks to the queue, they might be concurrently processed by many servers in the cluster, not limited to single JVM
Expiration and priorities - you can define different expiration time or priorities for your messages.
Cons:
More complex application - you will need to implement MDB to process your tasks.
More complex infrastructure - you will need database to store the queues (file system can be used for single server, and shared filesystem can be used for clusters), or external messaging solution like WebSphere MQ
a bit lower performance for processing single item and higher load on server, as it will have to be serialized/deserialized to persistent storage