I'm calling this function every 50 ms :
def send() = {
val myData = generateRandomData()
val response = pipeline(Post("http://slow-website.com/send", myData))
response onComplete {
case Success(r) => ? how to access myData ?
case Failure(error) => print(error.getMessage)
}
}
I would like to know what data was sent in my successfull request.
How can I achieve this?
Just refer to myData.
What happens behind the scenes is that the Scala compiler creates a closure for the onComplete handler argument that captures the reference to myData so that you can use it.
Related
I am using a web API that uses an authentication that is similar to, but not quite, OAuth 2.0. As a consequence, I would like to implement my own Token class for use by ‘httr’.
Unfortunate there are several issues:
The Token base class seems to be specific to OAuth and contains members/logic that don’t apply to me.
The documentation is incomplete and incorrect: at the very least it is missing the method for restoring a token from the cache, and it is also missing the specification for the other methods. It also specifies an incorrect return type for sign() (the calling code inside ‘httr’ expects a request S3 object).
To implement sign() correctly, I need to use an unexported function, httr:::request().
The expected usage for me would be as follows, given a valid token from a previous authorisation request:
httr::GET(endpoint_url, bearer_auth(token))
Where:
bearer_auth = function (token) {
httr::config(token = BearerToken$new(token))
}
This “works” when creating a minimal BearerToken R6 class that doesn’t inherit from httr::Token, implements only the sign() method, and cheats by accessing httr:::request():
BearerToken = R6::R6Class("BearerToken", list(
token = NULL,
initialize = function (token) {
self$token = token
},
sign = function (method, url) {
httr:::request(
method = method,
url = url,
headers = c(Authorization = paste("Bearer", self$token))
)
}
))
But that’s obviously insufficient for real usage.
I am making some http requests in kotlin with fuel library. I want to test that code using mockk library. I figured out how to mock http requests. Below is the code for that.
val client = mockk<Client>()
every { client.executeRequest(any()).statusCode } returns 200
every { client.executeRequest(any()).responseMessage } returns "test"
every { client.executeRequest(any()).data } returns "abc".toByteArray()
FuelManager.instance.client = client
assertEquals("abc" , testHttpRequest())
I do not like that any() here. I want to be specific about the http method and the url. I would like to return specific responses based on the url being called and the http method being used.
I figured may be I could do following
val req = Request(Method.POST, "my/test", URL("https://testRequest.com"), timeoutInMillisecond = 3000, timeoutReadInMillisecond = 3000)
every { client.executeRequest(req).statusCode } returns 200
every { client.executeRequest(req).responseMessage } returns "OK"
every { client.executeRequest(req).data } returns "abc".toByteArray()
FuelManager.instance.client = client
But I am getting following error.
io.mockk.MockKException: no answer found for: Client(#1).executeRequest(-->
https://testRequest.com/my/test
"Body : abc"
"Headers : (3)"
Accept-Encoding : compress;q=0.5, gzip;q=1.0
Content-Type : application/json
Authorization : Basic xxx)
What am I missing here?
To all those people that ended up here trying to find a solution to this, I've found something that solves the problem for my use case (but there are likely many use cases it's not appropriate for and I accept that it may not be the nicest...).
Provided you always have the calls to different endpoints in the same order every time you can do -
every { client.executeRequest(any()).data} returnsMany listOf(responseBody1, responseBody2, ... , responseBodyN)
Which will return the next response body for every subsequent call to the Fuel client.
The full code would look like (using OP's example) -
val response1 = "This is response one"
val response2 = "This is response two"
val client = mockk<Client>()
every { client.executeRequest(any()).statusCode } returns 200
every { client.executeRequest(any()).responseMessage } returns "test"
every { client.executeRequest(any()).data } returnsMany listOf(response1.toByteArray(), response2.toByteArray())
FuelManager.instance.client = client
assertEquals("This is response one", testHttpRequest())
assertEquals("This is response two", testHttpRequest())
I suspect the correct way to do this is with a 'CustomMatcher' extension function on the MockKMatcherScope as detailed here. I could only get the mock to response with the last item that'd been mocked when doing that, rather than the correct item but YMMV...
Try to use following:
every { client.executeRequest(req) } returns <mock object>
You could try these lines. Regarding how to intercept fuel request.
fun interceptFuel(method: Method, url: String) {
val interceptor = { next: (Request) -> Request ->
{ req: Request ->
if (req.method == method && req.url.toString() == url) {
val client = mockk<Client>()
/*mock fuel into whatever you like*/
FuelManager.instance.client = client
}
next(req)
}
}
FuelManager.instance.addRequestInterceptor(interceptor)
}
then, use it like this
interceptFuel(Method.GET, "https://google.com")
BTW, this code not fully tested. Use at your own risk
First function observe any process. However, if first function will not send any response, i want to stop observer manually according to second function.When i call second function, observer does not stop in first function. How can i handle this situation ?
exports.startObserverFunc = functions.https.onRequest((req,res) => {
var userFirebaseID = req.query.firebaseID;
var ref = admin.database().ref("Quickplay");
let callback = ref.child(userFirebaseID).on("value",function(snapshot){
if (snapshot.exists()){
// Some Codes
ref.child(userFirebaseID).off("value", callback);
res.status(200).send({
resultName: "ok"
});
}
});
});
exports.stopObserverFunc = functions.https.onRequest((req,res) => {
var userFirebaseID = req.query.firebaseID;
var ref = admin.database().ref("Quickplay");
ref.child(userFirebaseID).off("value");
res.status(200).send({
resultName: "ok"
});
});
You should avoid using observers/listeners like this in Cloud Functions. It will likely not do what you want, given that your code could be running on any number of server instances, based on the load on your functions. Also, two function invocations are definitely not going to be running on the same server instances, so they have no knowledge of each other.
It's almost certainly the case that you just want to use once() to query for a single object just one time, and use that in your response. This is what all the official samples do.
I understand how to make a message based non-blocking application in akka, and can easily mock up examples that perform
concurrent operations and pass back the aggregated results in a message. Where I have difficulty is understanding what my
non-blocking options are when my application has to respond to an HTTP request. The goal is to receive a request and
immediately hand it over to a local or remote actor to do the work, which in turn will hand it off to get a result that
could take some time. Unfortunatly under this model, I don't understand how I could express this with a non-blocking
series of "tells" rather than blocking "asks". If at any point in the chain I use a tell, I no longer have a future to
use as the eventual response content (required by the http framework interface which in this case is finagle - but that is not
important). I understand the request is on its own thread, and my example is quite contrived, but just trying to
understand my design options.
In summary, If my contrived example below can be reworked to block less I very much love to understand how. This is my
first use of akka since some light exploration a year+ ago, and in every article, document, and talk I have viewed says
not to block for services.
Conceptual answers may be helpful but may also be the same as what I have already read. Working/Editing my example
would likely be key to my understanding of the exact problem I am attempting to solve. If the current example is generally
what needs to be done that confirmation is helpful too, so I don't search for magic that does not exist.
Note The following aliases: import com.twitter.util.{Future => TwitterFuture, Await => TwitterAwait}
object Server {
val system = ActorSystem("Example-System")
implicit val timeout = Timeout(1 seconds)
implicit def scalaFuture2twitterFuture[T](scFuture: Future[T]): TwitterFuture[T] = {
val promise = TwitterPromise[T]
scFuture onComplete {
case Success(result) ⇒ promise.setValue(result)
case Failure(failure) ⇒ promise.setException(failure)
}
promise
}
val service = new Service[HttpRequest, HttpResponse] {
def apply(req: HttpRequest): TwitterFuture[HttpResponse] = req.getUri match {
case "/a/b/c" =>
val w1 = system.actorOf(Props(new Worker1))
val r = w1 ? "take work"
val response: Future[HttpResponse] = r.mapTo[String].map { c =>
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
response
}
}
//val server = Http.serve(":8080", service); TwitterAwait.ready(server)
class Worker1 extends Actor with ActorLogging {
def receive = {
case "take work" =>
val w2 = context.actorOf(Props(new Worker2))
pipe (w2 ? "do work") to sender
}
}
class Worker2 extends Actor with ActorLogging {
def receive = {
case "do work" =>
//Long operation...
sender ! "The Work"
}
}
def main(args: Array[String]) {
val r = service.apply(
com.twitter.finagle.http.Request("/a/b/c")
)
println(TwitterAwait.result(r).getContent.toString(CharsetUtil.UTF_8)) // prints The Work
}
}
Thanks in advance for any guidance offered!
You can avoid sending a future as a message by using the pipe pattern—i.e., in Worker1 you'd write:
pipe(w2 ? "do work") to sender
Instead of:
sender ! (w2 ? "do work")
Now r will be a Future[String] instead of a Future[Future[String]].
Update: the pipe solution above is a general way to avoid having your actor respond with a future. As Viktor points out in a comment below, in this case you can take your Worker1 out of the loop entirely by telling Worker2 to respond directly to the actor that it (Worker1) got the message from:
w2.tell("do work", sender)
This won't be an option if Worker1 is responsible for operating on the response from Worker2 in some way (by using map on w2 ? "do work", combining multiple futures with flatMap or a for-comprehension, etc.), but if that's not necessary, this version is cleaner and more efficient.
That kills one Await.result. You can get rid of the other by writing something like the following:
val response: Future[HttpResponse] = r.mapTo[String].map { c =>
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
Now you just need to turn this Future into a TwitterFuture. I can't tell you off the top of my head exactly how to do this, but it should be fairly trivial, and definitely doesn't require blocking.
You definitely don't have to block at all here. First, update your import for the twitter stuff to:
import com.twitter.util.{Future => TwitterFuture, Await => TwitterAwait, Promise => TwitterPromise}
You will need the twitter Promise as that's the impl of Future you will return from the apply method. Then, follow what Travis Brown said in his answer so your actor is responding in such a way that you do not have nested futures. Once you do that, you should be able to change your apply method to something like this:
def apply(req: HttpRequest): TwitterFuture[HttpResponse] = req.getUri match {
case "/a/b/c" =>
val w1 = system.actorOf(Props(new Worker1))
val r = (w1 ? "take work").mapTo[String]
val prom = new TwitterPromise[HttpResponse]
r.map(toResponse) onComplete{
case Success(resp) => prom.setValue(resp)
case Failure(ex) => prom.setException(ex)
}
prom
}
def toResponse(c:String):HttpResponse = {
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
This probably needs a little more work. I didn't set it up in my IDE, so I can't guarantee you it compiles, but I believe the idea to be sound. What you return from the apply method is a TwitterFuture that is not yet completed. It will be completed when the future from the actor ask (?) is done and that's happing via a non-blocking onComplete callback.
I need to do two $http.get call and I need to send returned response data to my service for doing further calculation.
I want to do something like below:
function productCalculationCtrl($scope, $http, MyService){
$scope.calculate = function(query){
$http.get('FIRSTRESTURL', {cache: false}).success(function(data){
$scope.product_list_1 = data;
});
$http.get('SECONDRESTURL', {'cache': false}).success(function(data){
$scope.product_list_2 = data;
});
$scope.results = MyService.doCalculation($scope.product_list_1, $scope.product_list_2);
}
}
In my markup I am calling it like
<button class="btn" ng-click="calculate(query)">Calculate</button>
As $http.get is asynchronous, I am not getting the data when passing in doCalculation method.
Any idea how can I implement multiple $http.get request and work like above implementation to pass both the response data into service?
What you need is $q.all.
Add $q to controller's dependencies, then try:
$scope.product_list_1 = $http.get('FIRSTRESTURL', {cache: false});
$scope.product_list_2 = $http.get('SECONDRESTURL', {'cache': false});
$q.all([$scope.product_list_1, $scope.product_list_2]).then(function(values) {
$scope.results = MyService.doCalculation(values[0], values[1]);
});
There's a simple and hacky way: Call the calculation in both callbacks. The first invocation (whichever comes first) sees incomplete data. It should do nothing but quickly exit. The second invocation sees both product lists and does the job.
I had a similar problem recently so I'm going to post my answer also:
In your case you only have two calculations and it seems to be the case this number is not mutable.
But hey, this could be any case with two or more requests being triggered at once.
So, considering two or more cases, this is how I would implement:
var requests = [];
requests.push($http.get('FIRSTRESTURL', {'cache': false}));
requests.push($http.get('SECONDRESTURL', {'cache': false}));
$q.all(requests).then(function (responses) {
var values = [];
for (var x in responses) {
responses[x].success(function(data){
values.push(data);
});
}
$scope.results = MyService.doCalculation(values);
});
Which, in this case, would force doCalculation to accept an array instead.