play framework parallel WSClient calls error management - asynchronous

I have an action in which I make three parallel HTTP calls (to other services), then I merge the contents of the responses into one document and finally I send it back to the client.
This is a working sample of the code:
#Inject
WSClient wsc;
public CompletionStage<Result> getUrlData() throws Exception {
List<CompletionStage<WSResponse>> stages = new ArrayList<>();
stages.add(wsc.url("http://jsonplaceholder.typicode.com/posts/1").get());
stages.add(wsc.url("http://jsonplaceholder.typicode.com/posts/2").get());
stages.add(wsc.url("http://jsonplaceholder.typicode.com/posts/3").get());
return Futures
.sequence(stages)
.thenApply(responses -> {
StringBuilder builder = new StringBuilder("[");
responses.stream().forEach(response -> builder.append(response.getBody()).append(","));
builder.deleteCharAt(builder.length()-1).append("]");
return ok(builder.toString());
})
.exceptionally(ex -> ok("{\"error\": \"An error has occurred\"}"));
If one of services is not available (you can simulate this behavior modifying the domain name of one of the URLs to a non existing one), the page returned contains only the message contained in the exceptionally() part, while I need to return the contents of the correct calls plus the error message of the not succeeded call. Any hint on how to do it?
I'm using Play 2.5.1.
Thanks,
Andrea

Basically you just want to handle the .exceptionally(..) individually for each call. Something like this should work:
create a function that returns a CompletionStage for an individual URL, incorporating your error handling (returning JSON of the error)
convert that to a list of completion stages to pass to Futures.sequence
As an aside, you can make the JSON manipulation a bit nicer by building the objects programatically using Jackson's ObjectMapper.createObjectNode() and ObjectMapper.createArrayNode():
private static final ObjectMapper mapper = new ObjectMapper();
private CompletionStage<JsonNode> getDataFromUrl(String url) {
return wsc.url(url)
.get()
.thenApply(WSResponse::asJson)
.exceptionally(ex -> {
ObjectNode error = mapper.createObjectNode();
error.put("error", ex.getMessage());
return error;
});
}
public CompletionStage<Result> getUrlData() throws Exception {
List<String> urls = new ArrayList<>();
urls.add("http://jsonplaceholder.typicode.com/posts/1");
urls.add("http://jsonplaceholder.typicode.com/posts/2");
urls.add("http://jsonplaceholder.typicode.com/posts/3");
// Convert to a list of promises
List<CompletionStage<JsonNode>> stages = urls
.stream()
.map(this::getDataFromUrl)
.collect(Collectors.toList());
return Futures
.sequence(stages)
.thenApply(responses -> {
ArrayNode arrayNode = mapper.createArrayNode();
responses.stream().forEach(arrayNode::add);
return ok(arrayNode);
});
}

Related

How to use CompletableFuture to execute the threads parallaly without waiting and combine the result?

I have executeGetCapability method which is executed in different threads but these threads runs sequentially..meaning one is completed after the other
#Async("threadPoolCapabilitiesExecutor")
public CompletableFuture<CapabilityDTO> executeGetCapability(final String id, final LoggingContextData data){...}
and this method is called in following way:
public CapabilityResponseDTO getCapabilities(final List<String> ids) {
final CapabilityResponseDTO responseDTO = new CapabilityResponseDTO();
final List<CapabilityDTO> listOfCapabilityDTOS = new ArrayList<>();
try {
for (String id: ids) {
listOfCapabilityDTOS .add(
asyncProcessService.executeGetCapability(id, LoggingContext.getLoggingContextData()).get());
}
} catch (Exception e) {
....
}
responseDTO.setDTOS(listOfCapabilityDTOS );
return responseDTO;
}
How can i call executeGetCapability method using CompletableFuture so that thread runs in parallel without waiting for each other and then the result is combined ?? how can i use here CompletableFuture.supplyAsync and or .allOf methods ? Can someone explain me
Thanks
The reduce helper function from this answer converts a CompletableFuture<Stream<T>> to a Stream<CompletableFuture<T>>. You can use it to asynchronously combine the results for multiple calls to executeGetCapability:
// For each id create a future to asynchronously execute executeGetCapability
Stream<CompletableFuture<CapabilityDTO>> futures = ids.stream()
.map(id -> executeGetCapability(id, LoggingContext.getLoggingContextData()));
// Reduce the stream of futures to a future for a stream
// and convert the stream to list
CompletableFuture<List<CapabilityDTO>> capabilitiesFuture = reduce(futures)
.thenApply(Stream::toList);
responseDTO.setDTOS(capabilitiesFuture.get());

Iterate with asynchronous functions

Using the Twitch API and vert.x - I'm looking to continuously send requests to Twitch's API using a WebClient and Twitch's cursor response to go page by page. However I'm not sure how to go back and keep doing queries until a condition is met due to vert.x's asynchronous nature.
Here's my code so far
public void getEntireStreamList(Handler<AsyncResult<JsonObject>> handler) {
JsonObject data = new JsonObject();
getLiveChannels(100, result -> {
if(result.succeeded()) {
JsonObject json = result.result();
String cursor = json.getJsonObject("pagination").getString("cursor");
data.put("data", json.getJsonArray("data"));
if(json.getJsonArray("data").size() < 100) { // IF NOT LAST PAGE
// GO BACK AND DO AGAIN WITH CURSOR IN REQUEST
}
handler.handle(Future.succeededFuture(data));
} else
handler.handle(Future.failedFuture(result.cause()));
});
}
Ideally I'd be able to call getLiveChannels with the cursor String from the previous request to continue the search.
You will need to use Future composition.
Here's my code for your problem:
public void getEntireStreamList(Handler<AsyncResult<JsonObject>> handler) {
JsonArray data = new JsonArray();
// create initial Future for first function call
Future<JsonObject> initFuture = Future.future();
// complete Future when getLiveChannels completes
// fail on exception
getLiveChannels(100, initFuture.completer());
// Create a callback that returns a Future
// for composition.
final AtomicReference<Function<JsonObject, Future<JsonObject>>> callback = new AtomicReference<>();
// Create Function that calls composition with itself.
// This is similar to recursion.
Function<JsonObject, Future<JsonObject>> cb = new Function<JsonObject, Future<JsonObject>>() {
#Override
public Future<JsonObject> apply(JsonObject json) {
// new Future to return
Future<JsonObject> f = Future.future();
// Do what you wanna do with the data
String cursor = json.getJsonObject("pagination").getString("cursor");
data.addAll(json.getJsonArray("data"));
// IF NOT LAST PAGE
if(json.getJsonArray("data").size() == 100) {
// get more live channels with cursor
getLiveChannels(100, cursor, f.completer());
// return composed Future
return f.compose(this);
}
// Otherwise return completed Future with results.
f.complete(new JsonObject().put("data", data));
return f;
}
};
Future<JsonObject> composite = initFuture.compose(cb);
// Set handler on composite Future (ALL composed futures together)
composite.setHandler(result -> handler.handle(result));
}
The code + comments should speak for themselves if you read the Vert.x docs on sequential Future composition.

How to send List of objects to google execution api and handle it in google apps script

In general I want to export data from asp.net mvc application to Google Sheets for example list of people. I've already set up connection and authenticated app with my Google account (trough OAuth2) but now I'm trying to send my list of objects to api and then handle it in script (by putting all data in new file) and couldn't get my head around this.
Here is some sample code in my app that sends the request.
public async Task<ActionResult> SendTestData()
{
var result = new AuthorizationCodeMvcApp(this, new AppFlowMetadata()).
AuthorizeAsync(CancellationToken.None).Result;
if (result.Credential != null)
{
string scriptId = "MY_SCRIPT_ID";
var service = new ScriptService(new BaseClientService.Initializer
{
HttpClientInitializer = result.Credential,
ApplicationName = "Test"
});
IList<object> parameters = new List<object>();
var people= new List<Person>(); // next i'm selecting data from db.Person to this variable
parameters.Add(people);
ExecutionRequest req = new ExecutionRequest();
req.Function = "testFunction";
req.Parameters = parameters;
ScriptsResource.RunRequest runReq = service.Scripts.Run(req, scriptId);
try
{
Operation op = runReq.Execute();
if (op.Error != null)
{
// The API executed, but the script returned an error.
// Extract the first (and only) set of error details
// as a IDictionary. The values of this dictionary are
// the script's 'errorMessage' and 'errorType', and an
// array of stack trace elements. Casting the array as
// a JSON JArray allows the trace elements to be accessed
// directly.
IDictionary<string, object> error = op.Error.Details[0];
if (error["scriptStackTraceElements"] != null)
{
// There may not be a stacktrace if the script didn't
// start executing.
Newtonsoft.Json.Linq.JArray st =
(Newtonsoft.Json.Linq.JArray)error["scriptStackTraceElements"];
}
}
else
{
// The result provided by the API needs to be cast into
// the correct type, based upon what types the Apps
// Script function returns. Here, the function returns
// an Apps Script Object with String keys and values.
// It is most convenient to cast the return value as a JSON
// JObject (folderSet).
Newtonsoft.Json.Linq.JObject folderSet =
(Newtonsoft.Json.Linq.JObject)op.Response["result"];
}
}
catch (Google.GoogleApiException e)
{
// The API encountered a problem before the script
// started executing.
AddAlert(Severity.error, e.Message);
}
return RedirectToAction("Index", "Controller");
}
else
{
return new RedirectResult(result.RedirectUri);
}
}
The next is how to handle this data in scripts - are they serialized to JSON there?
The execution API calls are essentially REST calls so the payload should be serialized as per that. Stringified JSON is typically fine. Your GAS function should then parse that payload to consume the encoded lists
var data = JSON.parse(payload);

Some questions concerning the combination of ADO.NET, Dapper QueryAsync and Glimpse.ADO

I have been experimenting with a lightweight solution for handling my business logic. It consists of a vanilla ADO.NET connection that is extended with Dapper, and monitored by Glimpse.ADO. The use case for this setup will be a web application that has to process a handful of queries asynchronously per request. Below a simple implementation of my setup in an MVC controller.
public class CatsAndDogsController : Controller
{
public async Task<ActionResult> Index()
{
var fetchCatsTask = FetchCats(42);
var fetchDogsTask = FetchDogs(true);
await Task.WhenAll(fetchCatsTask, fetchDogsTask);
ViewBag.Cats = fetchCatsTask.Result;
ViewBag.Dogs = fetchDogsTask.Result;
return View();
}
public async Task<IEnumerable<Cat>> FetchCats(int breedId)
{
IEnumerable<Cat> result = null;
using (var connection = CreateAdoConnection())
{
await connection.OpenAsync();
result = await connection.QueryAsync<Cat>("SELECT * FROM Cat WHERE BreedId = #bid;", new { bid = breedId });
connection.Close();
}
return result;
}
public async Task<IEnumerable<Dog>> FetchDogs(bool isMale)
{
IEnumerable<Dog> result = null;
using (var connection = CreateAdoConnection())
{
await connection.OpenAsync();
result = await connection.QueryAsync<Dog>("SELECT * FROM Dog WHERE IsMale = #im;", new { im = isMale });
connection.Close();
}
return result;
}
public System.Data.Common.DbConnection CreateAdoConnection()
{
var sqlClientProviderFactory = System.Data.Common.DbProviderFactories.GetFactory("System.Data.SqlClient");
var dbConnection = sqlClientProviderFactory.CreateConnection();
dbConnection.ConnectionString = "SomeConnectionStringToAwesomeData";
return dbConnection;
}
}
I have some questions concerning the creation of the connection in the CreateAdoConnection() method. I assume the following is happening behind the scenes.
The call to sqlClientProviderFactory.CreateConnection() returns an instance of System.Data.SqlClient.SqlConnection passed as a System.Data.Common.DbConnection. At this point Glimpse.ADO.AlternateType.GlimpseDbProviderFactory kicks in and wraps this connection in an instance of Glimpse.Ado.AlternateType.GlimpseDbConnection, which is also passed as a System.Data.Common.DbConnection. Finally, this connection is indirectly extended by the Dapper library with its query methods, among them the QueryAsync<>() method used to fetch the cats and dogs.
The questions:
Is the above assumption correct?
If I use Dapper's async methods with this connection - or create a System.Data.Common.DbCommand with this connection's CreateCommand() method, and use it's async methods - will those calls internally always end up using the vanilla async implementations of these methods as Microsoft has written them for System.Data.SqlClient.SqlConnection and System.Data.SqlClient.SqlCommand? And not some other implementations of these methods that are actually blocking?
How much perf do I lose with this setup compared to just returning a new System.Data.SqlClient.SqlConnection directly? (So, without the Glimpse.ADO wrapper)
Any suggestions on improving this setup?
Yes pretty much. GlimpseDbProviderFactory wraps/decorates/proxies all the registered factories. We then pass any calls we get through to the factory we wrap (in this case SQL Server). In the case of CreateConnection() we ask the inner factory we have, to create a connection, when we get that connection, we wrap it and then return it to the originating caller
Yes. Glimpse doesn't turn what was an async request into a blocking request. We persevere the async chain all the way though. If you are interested, the code in question is here.
Very little. In essence, using a decorator pattern like this adds only one or two frames to the call stack. Compared to most operations performed during the request lifecycle, the time to observe whats happening here is extremely minimal.
What you have looks great. Only suggestion is to maybe us this code to build the factory. This code means that you can shift your connection string, etc to the web.config.

Alfresco: Custom Share Evaluator based on some custom repo webscripts

So I'd like to write a new set of evaluators in Share based on the result of some repository webscripts.
The current existing Share evaluators are usable through some XML configuration and are related to Alfresco usual meta-data.
But, I'd like to know how to write my own Java evaluator while re using most of the logic already here (BaseEvaluator).
Suppose I have a repository webscript that returns some JSON like {"result" : "true"}:
How do I access it from my custom Evaluator? Mainly how do I access the proxy URL to alfresco webapp from the Spring context?
Do I need to write my own async call in Java?
Where do I find this JSONObject parameter of the required evaluate method?
thanks for your help
See if this helps. This goes into a class that extends BaseEvaluator. Wire the bean in through Spring, then set the evaluator on your actions.
public boolean evaluate(JSONObject jsonObject) {
boolean result = false;
final RequestContext rc = ThreadLocalRequestContext.getRequestContext();
final String userId = rc.getUserId();
try {
final Connector conn = rc.getServiceRegistry().getConnectorService().getConnector("alfresco", userId, ServletUtil.getSession());
final Response response = conn.call("/someco/example?echo=false");
if (response.getStatus().getCode() == Status.STATUS_OK) {
System.out.println(response.getResponse());
try {
org.json.JSONObject json = new org.json.JSONObject(response.getResponse());
result = Boolean.parseBoolean(((String) json.get("result")));
} catch (JSONException je) {
je.printStackTrace();
return false;
}
} else {
System.out.println("Call failed, code:" + response.getStatus().getCode());
return false;
}
} catch (ConnectorServiceException cse) {
cse.printStackTrace();
return false;
}
return result;
}
In this example I am using a simple example web script that echoes back your JSON and switches the result based on the value of the "echo" argument. So when it is called with "false", the JSON returns false and the evaluator returns false.
I should probably point out the name collision between the org.json.simple.JSONObject that the evaluate method expects and the org.json.JSONObject I am using to snag the result from the response JSON.

Resources