Suggestions for simple ways to do Asynchronous processing in Grails - servlets

Let's say I have a simple controller like this:
class FooController {
def index = {
someVeryLongCompution() //e.g crawl a set of web pages
render "Long computation was launched."
}
}
When the index action is invoked, I want the method to return immediately to the user while running the long computation asynchronously.
I understand the most robust way to do this would be to use a message broker in the architecture, but I was wondering if there is a simpler way to do it.
I tried the Executor plugin but that blocks the http request from returning until the long computation is done.
I tried the Quartz plugin, but that seems to be good for periodic tasks (unless there is a way to run a job just once?)
How are you guys handling such requests in Grails?

Where do you want to process veryLongComputation(), on the same Grails server, or a different server?
If the same server, you don't need JMS, another option is to just create a new thread and process the computation asynchronously.
def index = {
def asyncProcess = new Thread({
someVeryLongComputation()
} as Runnable )
asyncProcess.start()
render "Long computation was launched."
}

If you use a simple trigger in Grails Quartz and set the repeatCount to 0, the job will only run once. It runs separate from user requests, however, so you'd need to figure out some way to communicate to user when it completed.

I know this is a very old question, just wanted to give an updated answer.
Since grails 2.3, the framework supports async calls using Servlet 3.0 async request handling (of course, a servlet 3.0 container must be used and the servlet version should be 3.0 in the configuration, which it is per default)
It is documented here : http://grails.org/doc/latest/guide/async.html
In general, there are two ways achieving what you asked for:
import static grails.async.Promises.*
def index() {
tasks books: Book.async.list(),
totalBooks: Book.async.count(),
otherValue: {
// do hard work
}
}
or the Servlet Async way:
def index() {
def ctx = startAsync()
ctx.start {
new Book(title:"The Stand").save()
render template:"books", model:[books:Book.list()]
ctx.complete()
}
}
Small note - The grails method is using promises, which is a major (async) leap forward. Any Promise can be chained to further promised, have a callback on success and fail etc'

Have you tried Grails Promisses API? It should be as simple as
import static grails.async.Promise
import static grails.async.Promises
class FooController {
def index = {
Promise p = Promises.task {
someVeryLongCompution() //e.g crawl a set of web pages
}
render "Long computation was launched."
}
}

Try Spring events plugin - It supports asynchronous event listeners.

If you'd like to use the Quartz plugin (as we always do), you can do it like this. It works well for us:
DEFINE A JOB (with no timed triggers)
static triggers = {
simple name:'simpleTrigger', startDelay:0, repeatInterval: 0, repeatCount: 0
}
CALL <job>.triggerNow() to manually execute the job in asynchronous mode.
MyJob.triggerNow([key1:value1, key2: value2]);
Pro Tip #1
To get the named parameters out the other side...
def execute(context) {
def value1 = context.mergedJobDataMap.get('key1');
def value2 = context.mergedJobDataMap.get('key2');
...
if (value1 && value2) {
// This was called by triggerNow(). We know this because we received the parameters.
} else {
// This was called when the application started up. There are no parameters.
}
}
Pro Tip #2
The execute method always gets called just as the application starts up, but the named parameters come through as null.
Documentation: http://grails.org/version/Quartz%20plugin/24#Dynamic%20Jobs%20Scheduling

The best solution for this kind of problem is to use JMS, via the JMS plugin.
For a simpler implementation, that doesn't requires an externel server/service, you can try the Spring Events plugin.

Grails 2.2.1 Solution I had an additional requirement that said the report had to auto pop a window when it was complete. So I chose the servlet way from above with a twist. I replaced the render a view with a json formatted string return it looked like this.
Additionally my client side is not a gsp view, it is ExtJS 4.1.1 (a HTML5 product)
enter code here
def index() {
def ctx = startAsync()
ctx.start ({
Map retVar = [reportId: reportId, success: success];
String jsonString = retVar as JSON;
log.info("generateTwoDateParamReport before the render out String is: " + jsonString);
ctx.getOriginalWebRequest().getCurrentResponse().setContentType("text/html");
ctx.getOriginalWebRequest().getCurrentResponse().setCharacterEncoding("UTF-8");
log.info("current contentType is: "ctx.getOriginalWebRequest().getCurrentResponse().contentType);
try {
ctx.getOriginalWebRequest().getCurrentResponse().getWriter().write(jsonString);
ctx.getOriginalWebRequest().getCurrentResponse().getWriter().flush();
ctx.getOriginalWebRequest().getCurrentResponse().setStatus(HttpServletResponse.SC_OK);
}
catch (IOException ioe)
{
log.error("generateTwoDateParamReport flush data to client failed.");
}
ctx.complete();
log.info("generateNoUserParamsReport after complete");
});
}

Related

Solution for asynchronous notification upon future completion in GridGain needed

We are evaluating Grid Gain 6.5.5 at the moment as a potential solution for distribution of compute jobs over a grid.
The problem we are facing at the moment is a lack of a suitable asynchronous notification mechanism that will notify the sender asynchronously upon job completion (or future completion).
The prototype architecture is relatively simple and the core issue is presented in the pseudo code below (the full code cannot be published due to an NDA). *** Important - the code represents only the "problem", the possible solution in question is described in the text at the bottom together with the question.
//will be used as an entry point to the grid for each client that will submit jobs to the grid
public class GridClient{
//client node for submission that will be reused
private static Grid gNode = GridGain.start("config xml file goes here");
//provides the functionality of submitting multiple jobs to the grid for calculation
public int sendJobs2Grid(GridJob[] jobs){
Collection<GridCallable<GridJobOutput>> calls = new ArrayList<>();
for (final GridJob job : jobs) {
calls.add(new GridCallable<GridJobOutput>() {
#Override public GridJobOutput call() throws Exception {
GridJobOutput result = job.process();
return result;
}
});
}
GridFuture<Collection<GridJobOutput>> fut = this.gNode.compute().call(calls);
fut.listenAsync(new GridInClosure<GridFuture<Collection<GridJobOutput>>>(){
#Override public void apply(GridFuture<Collection<GridJobOutput>> jobsOutputCollection) {
Collection<GridJobOutput> jobsOutput;
try {
jobsOutput = jobsOutputCollection.get();
for(GridJobOutput currResult: jobsOutput){
//do something with the current job output BUT CANNOT call jobFinished(GridJobOutput out) method
//of sendJobs2Grid class here
}
} catch (GridException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
return calls.size();
}
//This function should be invoked asynchronously when the GridFuture is
//will invoke some processing/aggregation of the result for each submitted job
public void jobFinished(GridJobOutput out) {}
}
}
//represents a job type that is to be submitted to the grid
public class GridJob{
public GridJobOutput process(){}
}
Description:
The idea is that a GridClient instance will be used to in order to submit a list/array of jobs to the grid, notify the sender how many jobs were submitted and when the jobs are finished (asynchronously) is will perform some processing of the results. For the results processing part the "GridClient.jobFinished(GridJobOutput out)" method should be invoked.
Now getting to question at hand, we are aware of the GridInClosure interface that can be used with "GridFuture.listenAsync(GridInClosure> lsnr)"
in order to register a future listener.
The problem (if my understanding is correct) is that it is a good and pretty straightforward solution in case the result of the future is to be "processed" by code that is within the scope of the given GridInClosure. In our case we need to use the "GridClient.jobFinished(GridJobOutput out)" which is out of the scope.
Due to the fact that GridInClosure has a single argument R and it has to be of the same type as of GridFuture result it seems impossible to use this approach in a straightforward manner.
If I got it right till now then in order to use "GridFuture.listenAsync(..)" aproach the following has to be done:
GridClient will have to implement an interface granting access to the "jobFinished(..)" method let's name it GridJobFinishedListener.
GridJob will have to be "wrapped" in new class in order to have an additional property of GridJobFinishedListener type.
GridJobOutput will have to be "wrapped" in new class in order to have an addtional property of GridJobFinishedListener type.
When the GridJob will be done in addition to the "standard" result GridJobOutput will contain the corresponding GridJobFinishedListener reference.
Given the above modifications now GridInClosure can be used now and in the apply(GridJobOutput) method it will be possible to call the GridClient.jobFinished(GridJobOutput out) method through the GridJobFinishedListener interface.
So if till now I got it all right it seems a bit clumsy work around so I hope I have missed something and there is a much better way to handle this relatively simple case of asynchronous call back.
Looking forward to any helpful feedback, thanks a lot in advance.
Your code looks correct and I don't see any problems in calling jobFinished method from the future listener closure. You declared it as an anonymous class which always has a reference to the external class (GridClient in your case), therefore you have access to all variables and methods of GridClient instance.

How to rewrite synchronous controller to be asynchronous in Play?

I'm using Play framework 2.2 for one of my upcoming web application. I have implemented my controllers in synchronous pattern, with several blocking calls (mainly, database).
For example,
Synchronous version:
public static Result index(){
User user = db.getUser(email); // blocking
User anotherUser = db.getUser(emailTwo); // blocking
...
user.sendEmail(); // call to a webservice, blocking.
return ok();
}
So, while optimising the code, decided to make use of Asynchronous programming support of Play. Gone through the documentation, but the idea is still vague to me, as I'm confused about how to properly convert the above synchronous block of code to Async.
So, I came up with below code:
Asynchronous version:
public static Promise<Result> index(){
return Promise.promise(
new Function0<Result>(){
public Result apply(){
User user = db.getUser(email); // blocking
User anotherUser = db.getUser(emailTwo); // blocking
...
user.sendEmail(); // call to a webservice, blocking.
return ok();
}
}
);
}
So, I just wrapped the entire control logic inside a promise block.
Is my approach correct?
Should I convert each and every blocking request inside the controller, as Asynchronous, or wrapping several blocking calls inside single Async block is enough?
The play framework is asynchronous by nature and it allows the creation of fully non-blocking code. But in order to be non-blocking - with all its benefits - you can't just wrap your blocking code and expect magic to happen...
In an ideal scenario, your complete application is written in a non-blocking manner. If this is not possible (for whatever reason), you might want to abstract your blocking code in Akka actors or behind async interfaces which return scala.concurrent.Future's. This way you can execute your blocking code (simultaneously) in a dedicated Execution Context, without impacting other actions. After all, having all your actions share the same ExecutionContext means they share the same Thread pool. So an Action that blocks Threads might drastically impact other Actions doing pure CPU while having CPU not fully utilized!
In your case, you probably want to start at the lowest level. It looks like the database calls are blocking so start by refactoring these first. You either need to find an asynchronous driver for whatever database you are using or if there is only a blocking driver available, you should wrap them in a future to execute using a DB-specific execution context (with a ThreadPool that's the same size as the DB ConnectionPool).
Another advantage of abstracting the DB calls behind an async interface is that, if at some point in the future, you switch to a non-blocking driver, you can just change the implementation of your interface without having to change your controllers!
In your re-active controller, you can then handle these futures and work with them (when they complete). You can find more about working with Futures here
Here's a simplified example of your controller method doing non-blocking calls, and then combining the results in your view, while sending an email asynchronous:
public static Promise<Result> index(){
scala.concurrent.Future<User> user = db.getUser(email); // non-blocking
scala.concurrent.Future<User> anotherUser = db.getUser(emailTwo); // non-blocking
List<scala.concurrent.Future<User>> listOfUserFutures = new ArrayList<>();
listOfUserFutures.add(user);
listOfUserFutures.add(anotherUser);
final ExecutionContext dbExecutionContext = Akka.system().dispatchers().lookup("dbExecutionContext");
scala.concurrent.Future<Iterable<User>> futureListOfUsers = akka.dispatch.Futures.sequence(listOfUserFutures, dbExecutionContext);
final ExecutionContext mailExecutionContext = Akka.system().dispatchers().lookup("mailExecutionContext");
user.andThen(new OnComplete<User>() {
public void onComplete(Throwable failure, User user) {
user.sendEmail(); // call to a webservice, non-blocking.
}
}, mailExecutionContext);
return Promise.wrap(futureListOfUsers.flatMap(new Mapper<Iterable<User>, Future<Result>>() {
public Future<Result> apply(final Iterable<User> users) {
return Futures.future(new Callable<Result>() {
public Result call() {
return ok(...);
}
}, Akka.system().dispatcher());
}
}, ec));
}
It you don't have anything to not block on then there may not be a reason to make your controller async. Here is a good blog about this from one of the creators of Play: http://sadache.tumblr.com/post/42351000773/async-reactive-nonblocking-threads-futures-executioncont

How can i get an immediate response from a long running process in j2ee?

I can't seem to find a solid answer anywhere. I THINK i found one with respect to JMS but it was confusing.
It really depends what stack of j2EE are you using? Is it just web, ejb layer or both?. If we are talking about the web then you can use asynchronous servlet introduced in the newest Java EE specification, if you are using plain EJB's then the natural choice would be Messege driven beans (mentioned JMS). You can of course design a custom solution where for example you send some data to process and then the j2ee application itself calls your application (with http request for example) to notify that its done running the job. Possibilities are endless and if one is better than other always depends on the specific scenario.
If I understand correctly what you are talking about is the ability to start a task (that will take some time) then respond to the user while that task is still doing it's stuff. Depending on your requirements it is really quite simple and you can use a plain old Java Thread to perform the operation.
public class DoSillyCounting extends Thread {
private volatile int counter;
public int getCounter() { return counter; }
public run() {
while (counter < 10) {
counter ++;
try { Thread.sleep(1000); }
catch (InterruptedException ie) { }
}
}
}
In your setup page you might do this: (session is an HttpSession)
DoSillyCounting doSillyCounting = new DoSillyCounting();
doSillyCounting.start();
session.putValue("tenSecondsCounter", doSillyCounting);
/* Here you can respond to the user while the Thread is executing */
And in your status page you might do this:
DoSillyCounting doSillyCounting =
(DoSillyCounting)session.getValue("tenSecondsCounter");
out.print(Integer.toString(doSillyCounting.getCounter());
if (doSillyCounting.isAlive()) {
out.print("Still Working on it");
} else {
out.print("Yippee, I finished");
}
Of course, this is a rather useless example and this model is not a good idea when you may have a large number of requests to satisfy, it would then be worth looking at a ThreadPool implementation or using something like JMS.

OpenRasta unit testing

I’m about to start work on an OpenRasta project (an xml over http web service). OpenRasta looks great but unfortunately worked examples seem few and far between on the internet. Looking at the test side of the project, if my handlers are returning strongly typed objects (not OperationResult), i.e.:
public class PersonHandler
...
public Person Get(int id)
{
...
How can I test for http status codes? (For example if the handler throws an uncaught exception). I’m not sure what level the tests pitch in at, and what needs mocking (using moq btw)
Any help appreciated, particularly coded examples!
I faced the same problem, and ended up writing my tests as integration tests at a much higher level, actually making real REST/HTTP calls through a simple HttpWebRequest client. This allowed me to check the HTTP response headers / status codes and double-check the JSON/XML serialization from the client's perspective, which was just as important as whether or not the operations succeeded.
I started by returning OperationResult from all my handlers, and used these to wrap the strongly-typed objects. My handlers all inherit from a base class with a few helper methods that make it easier to return a custom error with a user-friendly error message. The more I coded this up, the more my handlers resembled a ASP.NET MVC controller. e.g.:
public OperationResult GetById(int id)
{
try
{
// do stuff here
return OKResult( // some strongly-typed resource );
}
catch(SomeException ex)
{
return BadRequestResult(SomeErrorCode, ex.Message);
}
}
Then in the test client, it's pretty easy to just check the HTTP status code. Obviously this doesn't help much with mocking. I'm not sure what the best solution is, in fact I've favorited this question in the hope that someone answers it better than I can - but this has worked pretty well for me so far.
The handler is just a class--ideally with minimal dependencies--so your unit tests can just test the isolated logic in the class.
If you want to test for status codes, I recommend (based on very little experience!) using OpenRasta self-hosting.
Here's a test (somewhat changed) that I wrote recently:
[TestMethod]
public void POST_with_inaccurate_contentLength_returns_405()
{
var resource = GetResource();
IRequest request = new InMemoryRequest
{
HttpMethod = "POST",
Uri = new Uri("http://localhost/Resource"),
};
request.Headers.ContentLength = 16; //wrong!
request.Entity.Stream.Write(resource.Content, 0, resource.Content.Length);
var response = _host.ProcessRequest(request);
Assert.AreEqual(405, response.StatusCode);
}
I should add that the host is set up in the TestInitialize method as such:
_host = new InMemoryHost(new Configuration());
_host.Resolver.AddDependencyInstance(typeof(IFileResourceRepository), _repository, DependencyLifetime.Singleton);
...and is cleaned up in the TestCleanup method.

How does ASP.NET AJAX work when retrieving values from the server?

If I want to call a server function from JavaScript to retrieve a name of a person from a database (just as an example)... and I went...
name = myServices.getName(userId);
If I have a script manager with a service reference to a .asmx file that has the web method getName( int userId ) {} then this function should be called properly and would, eventually, return the name for that userId.
Unfortunately, I want to do...
name = myServices.getName(userId);
alert(name);
however, when doing ASP.NET AJAX, it would call the web method and continue executing before waiting for a response from the server (which I understand is the point of AJAX, to stop the browser from freezing while waiting for data)
I need to get the name back from the server before I can continue executing... How can I approach this to fix this issue?
There is a method you can add as a parameter to the service method that will call the method on success where you can do other stuff.
For example:
function test() {
PageMethods.MyMethod("name", OnMyMethodComplete);
}
function OnMyMethodComplete(result, userContext, methodName) {
alert(result);
}
If you want to call a Web method synchronously, you'll need to set up the request manually and use a Sys.Net.XMLHttpSyncExecutor.
Here's an example (see ExecuteSynchronously function).
For a JavaScript solution, you could loop until name has a value. Adjust time-based on latency to keep app responsive
var time = 100;
window.setTimeout(name == '' ? wait : continue, time);
function wait() {
window.setTimeout(name == '' ? wait : continue, time);
}
function continue() {
//code having to do with name
alert(name);
}
Congratulations! You've taken your first step into a larger asynchronous world. I'd definitely go with using the callbacks that CSharpAtl suggested.

Resources