I'm working with Play Framework (2.0.4) and Scala, and I have a problem.
I call my backend to get a list of users (in json), and for each user, I have to get extra info from the backend (one request per user).
So in my services, I have :
def getUsers(/*different uninteresting parameters*/ ): Promise[List[Option[User]]]
and
def getExtraUserInfo(user:User):Promise[Option[Double]]
So for each User which is returned by getUsers, I want to call getExtraUserInfo, and return the user plus the extra info about each user.
So in my controller, i've tried to do something like that :
def getUsers(/*parameters*/) = AuthenticatedAsync{ request =>
val users = UserService.getUsers(/*parameters*/)
users.flatMap {
case Some(userList) =>
Ok(Success("users" -> Json.toJson(userList.flatMap{
user => UserService.getExtraUserInfo(user).map {
case Some(price) => user.price = price
user
case _ => user
}
}.map(_.json))))
case _ => InternalServerError(Error("error while getting users", Errors.TECHNICAL))
}
}
Do you guys have any idea how to do it ? (this code doesn't work, but that's all I managed to do...)
Try Promise.sequence to sequence a list of promise.
Promise.sequence transforms a List[Promise[T]] to a Promise[List[T]]...
see the Play! Scala API here
Related
I am trying to make an API GET request, using React Query's useInfiniteQuery hook, that uses data from a Next Auth session token in the query string.
I have a callback in /api/auth/[...nextauth.ts] to send extra userData to my session token.
There are two relevant pages on the client side. Let's call them /pages/index.tsx and /hooks/useApiData.ts. This is what they look like, for all intents and purposes:
// pages/index.tsx
export default function Page() {
const {data, fetchNextPage, isLoading, isError} = useCourseData()
if (isLoading) return <main />
return <main>
<InfiniteScroller fetchMore={fetchNextPage}>
{data?.pages?.map(page => page?.results?.map(item: string => item))}
</InfiniteScroller>
</main>
}
// hooks/useApiData.ts
async function fetchPage(pageParam: string) {
const response = await fetch(pageParam)
return await response.json()
}
export default function useApiData() {
const {data: session} = useSession()
const init = `/api?userData=${session?.user?.userData}`
return useInfiniteQuery('query',
({pageParam = init}) => fetchPage(pageParam),
{getNextPageParam: prevPage => prevPage.next ?? undefined}
)
}
My initial request gets sent to the API as /api?userData=undefined. The extra data is definitely making its way into the token.
I can place the data from my session in the DOM via the render function of /pages/index.tsx, so I figure the problem is something to do with custom hooks running before the session context is ready, or something like that... I don't understand the mechanics of hooks well enough to figure that out.
I've been looking for answers for a long time, and I'm surprised not to have found a single person with the same issue. These are not unpopular packages and I guess a lot of people are using them in conjunction to achieve what I'm attempting here, so I figure I must be doing something especially dumb. But what?!
How can I get the data from my Next Auth session into my React Query request? And for bonus points, why is the session data not available when the request is sent in my custom hook?
I'm attempting to get a list of items from a MongoDB Atlas instance via Realm into my Xamarin Forms application. I'm pretty sure I've followed the instructions correctly, however I can't seem to see any values come in.
I'm using Realm and Realm.Fody version: 10.2.0
I've uploaded my data with the following script
mongoimport --uri mongodb+srv://[user]:[pass]#cluster0.[wxyz].mongodb.net/[company] --collection products --type json --file products --jsonArray
1057 document(s) imported successfully. 0 document(s) failed to import.
When I go to MongoDB Atlas, I see this
In my App.xaml.cs constructor, I create a realm app and log in with anonymous credentials
public const string MasterDataPartitionKey = "master_data";
ctor()
{
var app = Realms.Sync.App.Create("my-app-id");
app.LogInAsync(Credentials.Anonymous()).ContinueWith(task => User = task.Result);
}
After this, in my first ViewModel constructor (it's Rx, but isn't doing anything fancy)
ctor()
{
_listProductsCommand = ReactiveCommand.Create<Realm, IEnumerable<Product>>(ExecuteLoadProducts);
_listProductsCommand.ThrownExceptions.Subscribe(x => { /* checking for errors here */ }).DisposeWith(TrashBin);
Initialize
.ObserveOn(RxApp.MainThreadScheduler)
// note MasterDataPartitionKey is the value I've set on every single record's `_partitionKey` property.
.Select(_ => new SyncConfiguration(MasterDataPartitionKey, App.User))
.SelectMany(Realm.GetInstanceAsync)
.Do(_ => { }, ex => { /* checking for errors here */ })
.ObserveOn(RxApp.MainThreadScheduler)
.InvokeCommand(this, x => x._listProductsCommand)
.DisposeWith(TrashBin);
}
And later, a simple query
private static IEnumerable<Product> ExecuteLoadProducts(Realm realm)
{
var data = realm.All<ProductDto>();
var productDtos = data.ToList();
var products = productDtos.Select(x => x.Map());
return products;
}
Expected:
productDtos (and products) should have a count of 1057
Actual:
productDtos (and products) have a count of 0
I've looked in my Realm logs and I can see the connections and sync logs
My anonymous authentication is turned on
and I've made it so that all of my Collections have read access (in fact developer mode is on)
Here's an example of one of the records
Here's a snipped of the dto I'm trying to pull down
I feel as though I must be missing something simple. Can anyone see anything obvious that could be sending me sideways?
So, Im working on an app with a concept of "Plans" and each plan you can add a comment. That part works fine, but it seems to fail and get confused if i try to run this in a loop.
The Action:
export class AddComment implements Action {
readonly type = CommentActionTypes.AddComment;
constructor(public payload: Comment) {}
}
export class AddCommentSuccess implements Action {
readonly type = CommentActionTypes.AddCommentSuccess;
constructor(public payload: Comment) {}
}
Effect
#Effect()
addComment$: Observable<Action> = this.actions$
.ofType<AddComment>(CommentActionTypes.AddComment).pipe(
switchMap(action => this.commentService.addComment(this.disciplineType, action.payload)),
map((comment: any) => new AddCommentSuccess(comment)),
catchError(err => of(new AddCommentFail(err)))
);
Implementation
What im struggling with is firing this off in rapid success/ I have a situation where I want to add a duplicate comment to multiple plans.
saveSet.forEach(x => {
comment.plan_id = x.id;
this.store.dispatch(this.buildAddCommentAction(comment));
});
For reference:
buildAddCommentAction(comment: DisciplineComment) : Action {
return new CommentActions.AddComment(comment);
}
What is Happening
If i have a list of 5 plans, and want to add a duplicate comment to all of them, Im only getting a successful response for the last item in the loop.
Now i know that is overly chatty, that is 5 separate client/service calls. What I cant figure out, its what the prescribed approach to this should be?
1.) A new BulkAddComment Action, effect, etc. Im loathe to do this becuase I have Comments, Concerns (similar in function and need), and one of each for every "discipline". Thatd be about 36 new effects and twice that in actions. A serious refactor is needed.
2.) Modify the actions and effects for 1 or multiple
3.)?
Thanks for input
This is because you're using the switchMap operator which will cancel the current running observable, in your case the service call.
You'll have to use concatMap or mergeMap. If the order is important use concatMap, if not use mergeMap because this will make your service calls in parallel.
For more info, watch this.
I understand how to make a message based non-blocking application in akka, and can easily mock up examples that perform
concurrent operations and pass back the aggregated results in a message. Where I have difficulty is understanding what my
non-blocking options are when my application has to respond to an HTTP request. The goal is to receive a request and
immediately hand it over to a local or remote actor to do the work, which in turn will hand it off to get a result that
could take some time. Unfortunatly under this model, I don't understand how I could express this with a non-blocking
series of "tells" rather than blocking "asks". If at any point in the chain I use a tell, I no longer have a future to
use as the eventual response content (required by the http framework interface which in this case is finagle - but that is not
important). I understand the request is on its own thread, and my example is quite contrived, but just trying to
understand my design options.
In summary, If my contrived example below can be reworked to block less I very much love to understand how. This is my
first use of akka since some light exploration a year+ ago, and in every article, document, and talk I have viewed says
not to block for services.
Conceptual answers may be helpful but may also be the same as what I have already read. Working/Editing my example
would likely be key to my understanding of the exact problem I am attempting to solve. If the current example is generally
what needs to be done that confirmation is helpful too, so I don't search for magic that does not exist.
Note The following aliases: import com.twitter.util.{Future => TwitterFuture, Await => TwitterAwait}
object Server {
val system = ActorSystem("Example-System")
implicit val timeout = Timeout(1 seconds)
implicit def scalaFuture2twitterFuture[T](scFuture: Future[T]): TwitterFuture[T] = {
val promise = TwitterPromise[T]
scFuture onComplete {
case Success(result) ⇒ promise.setValue(result)
case Failure(failure) ⇒ promise.setException(failure)
}
promise
}
val service = new Service[HttpRequest, HttpResponse] {
def apply(req: HttpRequest): TwitterFuture[HttpResponse] = req.getUri match {
case "/a/b/c" =>
val w1 = system.actorOf(Props(new Worker1))
val r = w1 ? "take work"
val response: Future[HttpResponse] = r.mapTo[String].map { c =>
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
response
}
}
//val server = Http.serve(":8080", service); TwitterAwait.ready(server)
class Worker1 extends Actor with ActorLogging {
def receive = {
case "take work" =>
val w2 = context.actorOf(Props(new Worker2))
pipe (w2 ? "do work") to sender
}
}
class Worker2 extends Actor with ActorLogging {
def receive = {
case "do work" =>
//Long operation...
sender ! "The Work"
}
}
def main(args: Array[String]) {
val r = service.apply(
com.twitter.finagle.http.Request("/a/b/c")
)
println(TwitterAwait.result(r).getContent.toString(CharsetUtil.UTF_8)) // prints The Work
}
}
Thanks in advance for any guidance offered!
You can avoid sending a future as a message by using the pipe pattern—i.e., in Worker1 you'd write:
pipe(w2 ? "do work") to sender
Instead of:
sender ! (w2 ? "do work")
Now r will be a Future[String] instead of a Future[Future[String]].
Update: the pipe solution above is a general way to avoid having your actor respond with a future. As Viktor points out in a comment below, in this case you can take your Worker1 out of the loop entirely by telling Worker2 to respond directly to the actor that it (Worker1) got the message from:
w2.tell("do work", sender)
This won't be an option if Worker1 is responsible for operating on the response from Worker2 in some way (by using map on w2 ? "do work", combining multiple futures with flatMap or a for-comprehension, etc.), but if that's not necessary, this version is cleaner and more efficient.
That kills one Await.result. You can get rid of the other by writing something like the following:
val response: Future[HttpResponse] = r.mapTo[String].map { c =>
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
Now you just need to turn this Future into a TwitterFuture. I can't tell you off the top of my head exactly how to do this, but it should be fairly trivial, and definitely doesn't require blocking.
You definitely don't have to block at all here. First, update your import for the twitter stuff to:
import com.twitter.util.{Future => TwitterFuture, Await => TwitterAwait, Promise => TwitterPromise}
You will need the twitter Promise as that's the impl of Future you will return from the apply method. Then, follow what Travis Brown said in his answer so your actor is responding in such a way that you do not have nested futures. Once you do that, you should be able to change your apply method to something like this:
def apply(req: HttpRequest): TwitterFuture[HttpResponse] = req.getUri match {
case "/a/b/c" =>
val w1 = system.actorOf(Props(new Worker1))
val r = (w1 ? "take work").mapTo[String]
val prom = new TwitterPromise[HttpResponse]
r.map(toResponse) onComplete{
case Success(resp) => prom.setValue(resp)
case Failure(ex) => prom.setException(ex)
}
prom
}
def toResponse(c:String):HttpResponse = {
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
This probably needs a little more work. I didn't set it up in my IDE, so I can't guarantee you it compiles, but I believe the idea to be sound. What you return from the apply method is a TwitterFuture that is not yet completed. It will be completed when the future from the actor ask (?) is done and that's happing via a non-blocking onComplete callback.
I'm making a real push to understand the async powers of Play but finding a lot of conflict with regard to places where async invocation fits and places where the framework seems to conspire against its use.
The example I have relates to form validation. Play allows for ad-hoc constraints to be defined - see this from the docs:
val loginForm = Form(
tuple(
"email" -> email,
"password" -> text
) verifying("Invalid user name or password", fields => fields match {
case (e, p) => User.authenticate(e,p).isDefined
})
)
Nice and clean. However, if I'm using a fully async data access layer (e.g. ReactiveMongo), such a call to User.authenticate(...) would return a Future and I'm thus in the dark as to how I can utilise the power of both the built in form binding features and the async tools.
It's all well and good to publicise the async approach but I'm getting frustrated that certain parts of the framework don't play so well with it. If the validation has to be done synchronously, it seems to defeat the point of the async approach. I've come across a similar problem when using Action composition - e.g. a security related Action that would make a call to ReactiveMongo.
Can anyone shed any light on where my comprehension is falling short?
Yes, validation in Play is designed synchronously. I think it's because assumed that most of time there is no I/O in form validation: field values are just checked for size, length, matching against regexp, etc.
Validation is built over play.api.data.validation.Constraint that store function from validated value to ValidationResult (either Valid or Invalid, there is no place to put Future here).
/**
* A form constraint.
*
* #tparam T type of values handled by this constraint
* #param name the constraint name, to be displayed to final user
* #param args the message arguments, to format the constraint name
* #param f the validation function
*/
case class Constraint[-T](name: Option[String], args: Seq[Any])(f: (T => ValidationResult)) {
/**
* Run the constraint validation.
*
* #param t the value to validate
* #return the validation result
*/
def apply(t: T): ValidationResult = f(t)
}
verifying just adds another constraint with user-defined function.
So I think Data Binding in Play just isn't designed for doing I/O while validation. Making it asynchronous would make it more complex and harder to use, so it kept simple. Making every piece of code in framework to work on data wrapped in Futures is overkill.
If you need to use validation with ReactiveMongo, you can use Await.result. ReactiveMongo returns Futures everywhere, and you can block until completion of these Futures to get result inside verifying function. Yes, it will waste a thread while MongoDB query runs.
object Application extends Controller {
def checkUser(e:String, p:String):Boolean = {
// ... construct cursor, etc
val result = cursor.toList().map( _.length != 0)
Await.result(result, 5 seconds)
}
val loginForm = Form(
tuple(
"email" -> email,
"password" -> text
) verifying("Invalid user name or password", fields => fields match {
case (e, p) => checkUser(e, p)
})
)
def index = Action { implicit request =>
if (loginForm.bindFromRequest.hasErrors)
Ok("Invalid user name")
else
Ok("Login ok")
}
}
Maybe there's way to not waste thread by using continuations, not tried it.
I think it's good to discuss this in Play mailing list, maybe many people want to do asynchronous I/O in Play data binding (for example, for checking values against database), so someone may implement it for future versions of Play.
I've been struggling with this, too. Realistic applications are usually going to have some sort of user accounts and authentication. Instead of blocking the thread, an alternative would be to get the parameters out of the form and handle the authentication call in the controller method itself, something like this:
def authenticate = Action { implicit request =>
Async {
val (username, password) = loginForm.bindFromRequest.get
User.authenticate(username, password).map { user =>
user match {
case Some(u: User) => Redirect(routes.Application.index).withSession("username" -> username)
case None => Redirect(routes.Application.login).withNewSession.flashing("Login Failed" -> "Invalid username or password.")
}
}
}
}
Form validation means syntactic validation of fields, one by one.
If a filed does not pass the validation it can be marked (eg. red bar with message).
Authentication should be placed in the body of the action, which may be in an Async block.
It should be after the bindFromRequest call, so there must me after the validation, so after each field is not empty, etc.
Based on the result of the async calls (eg. ReactiveMongo calls) the result of the action can be either BadRequest or Ok.
Both with BadRequest and Ok can redisplay the form with error message if the authentication failed. These helpers only specify the HTTP status code of the response, independently to the response body.
It would be an elegant solution to do the Authentication with play.api.mvc.Security.Authenticated (or write a similar, customized action compositor), and use Flash scoped messages. Thus the user always would be redirected to login page if she is not authenticated, but if she submits the login form with wrong credentials the error message would be shown besides the redirect.
Please take a look on the ZenTasks example of your play installation.
The same question was asked in the Play mailing list with Johan Andrén replying:
I'd move the actual authentication out of the form validation and do it in your action instead and use the validation only for validation of required fields etc. Something like this:
val loginForm = Form(
tuple(
"email" -> email,
"password" -> text
)
)
def authenticate = Action { implicit request =>
loginForm.bindFromRequest.fold(
formWithErrors => BadRequest(html.login(formWithErrors)),
auth => Async {
User.authenticate(auth._1, auth._2).map { maybeUser =>
maybeUser.map(user => gotoLoginSucceeded(user.get.id))
.getOrElse(... failed login page ...)
}
}
)
}
I've seen on theguardian's GH repo how they handle this case scenario in a asynchronous way while still having the support of the form error helpers from play. From a quick look, seems like they are storing the form errors in an encrypted cookie in a way as to display those errors back to the user the next time the user goes to the login page.
Extracted from: https://github.com/guardian/facia-tool/blob/9ec455804edbd104861117d477de9a0565776767/identity/app/controllers/ReauthenticationController.scala
def processForm = authenticatedActions.authActionWithUser.async { implicit request =>
val idRequest = idRequestParser(request)
val boundForm = formWithConstraints.bindFromRequest
val verifiedReturnUrlAsOpt = returnUrlVerifier.getVerifiedReturnUrl(request)
def onError(formWithErrors: Form[String]): Future[Result] = {
logger.info("Invalid reauthentication form submission")
Future.successful {
redirectToSigninPage(formWithErrors, verifiedReturnUrlAsOpt)
}
}
def onSuccess(password: String): Future[Result] = {
logger.trace("reauthenticating with ID API")
val persistent = request.user.auth match {
case ScGuU(_, v) => v.isPersistent
case _ => false
}
val auth = EmailPassword(request.user.primaryEmailAddress, password, idRequest.clientIp)
val authResponse = api.authBrowser(auth, idRequest.trackingData, Some(persistent))
signInService.getCookies(authResponse, persistent) map {
case Left(errors) =>
logger.error(errors.toString())
logger.info(s"Reauthentication failed for user, ${errors.toString()}")
val formWithErrors = errors.foldLeft(boundForm) { (formFold, error) =>
val errorMessage =
if ("Invalid email or password" == error.message) Messages("error.login")
else error.description
formFold.withError(error.context.getOrElse(""), errorMessage)
}
redirectToSigninPage(formWithErrors, verifiedReturnUrlAsOpt)
case Right(responseCookies) =>
logger.trace("Logging user in")
SeeOther(verifiedReturnUrlAsOpt.getOrElse(returnUrlVerifier.defaultReturnUrl))
.withCookies(responseCookies:_*)
}
}
boundForm.fold[Future[Result]](onError, onSuccess)
}
def redirectToSigninPage(formWithErrors: Form[String], returnUrl: Option[String]): Result = {
NoCache(SeeOther(routes.ReauthenticationController.renderForm(returnUrl).url).flashing(clearPassword(formWithErrors).toFlash))
}