I just start my FP journey while learning Scala.
Now a requirement is to maintain a List[String] in an unfiltered web application. When a POST request is sent to an endpoint, the List should be updated from a file. And the list will be used when a GET request is sent to the same endpoint.
Now, I am trying to avoid using var to saving the List. I know sometimes we have to use var but just curious that is there an elegant way to deal with the case. I've tried using scalaz.State Iterator and Steam. But got stuck since I have no idea how to pass the current immutable state to the next request. Any suggestion, please?
def update = State( l => {
retrieve(filepath) match {
case Success(lines) => (lines.split("[,\n\r]").toVector.map (_.trim), true)
case Failure(_) => {
log.error(s"Cannot retrieve the file.")
(l, false)
}
}
})
def isContained(message: String) = State(l => (l, l.exists(message.contains)))
/* assume the following get or post method will be invoked when GET or POST request is sent to the endpoint */
def post() = update(Vector.empty) // how can I pass the updated state to the get method
def get(msg: String): Boolean = isContained(msg)(???)._2
Then I don't know how can I pass the current state to the next visit as input without using var.
There is no free lunch. If you want to avoid mutability and avoid storing the state somewhere, you need to work with returned values.
State is no more than a function A => B (I simplified it a bit for a purpose), where A is initial state and B is the endresult
So in your case, the model would be looking like:
def post(newMessage: String, state: List[String]): List[String] = {
newMessage :: state
}
def get(msg: String, state: List[String]): Boolean = {
state.contains(msg)
}
As you can see here, you need to provide the current state to every post and get. Posts would just the add a new message from the file (put your business logic here), and return a new state. And for get, you need to provide current state, to be able to retrieve something you want.
You can rewrite it this way:
def post(newMessage: String): List[String] ⇒ List[String] = state ⇒ {
newMessage :: state
}
def get(msg: String): List[String] ⇒ Boolean =
_.contains(msg)
Notice that post returns you exactly A ⇒ A (where A = List[String]).
ScalaZ State gives you Monad for chaining the functions inside for comprehension as well as some additional conveniences (like map, gets, put etc..) . But essentially - the underlying model would be the same.
This code represents more precisely what State does:
type YourListState = List[String] ⇒ List[String]
def post(newMessage: String, state: YourListState): YourListState = li ⇒ {
newMessage :: state(li)
}
def get(msg: String, state: YourListState): List[String] ⇒ Boolean = {
state(_).contains(msg)
}
This allows you to combine states and to provide initial value and "run" it whenever you want, not sure that you really need this.
Related
I was wondering if anyone could give me any pointers on the best way to go about handling wake ups when writing a wrapper for a Stream.
For context I've got a Byte stream coming in via a HTTP request (using reqwest) and I'm doing some filtering and mapping on that stream to handle validation and deserialization. Effectively whenever the inner stream produces a value I want this stream to (potentially) emit a value.
** Edit **
An additional caveat is the stream needs to also hold a small amount of state (A Vec<String>) that it needs to be able to reference on each poll - (the columns property)
The Solution
This turned out to be me just not understanding how the stream was working under the hood. Rodrigo's answer below was completely correct. I did just need to return Poll::Pending from the inner stream, however I was making the mistake of matching on that and returning my own Poll::Pending which was why the stream wasn't being appropriately woken up.
If it's useful to anyone, instead of matching on the output of inner_stream.poll_next(), I ended up just mapping the Some value and returning that to ensure that I was building off the Polls of the inner stream eg:
return Pin::new(&mut this.stream).poll_next(cx).map(|data| { ... })
Thanks for everyone who commented and helped out!
Context for the original question
The wrapper type:
pin_project! {
#[derive(Default)]
struct QueryStream<T, S> where S: Stream, T: DeserializeOwned {
columns: Vec<String>,
#[pin]
stream: S,
has_closed: bool,
_marker: PhantomData<T>
}
}
The only implementation of Stream that I've managed to get to work on the wrapper type is one that spins on the inner stream when it returns Poll::Pending. This doesn't seem ideal though as I believe it would block until a value is emitted?
impl<T, S> Stream for QueryStream<T, S>
where
T: DeserializeOwned,
S: Stream<Item = std::result::Result<Bytes, reqwest::Error>>,
{
type Item = Result<T>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let mut this = self.project();
loop {
if *this.has_closed {
return Poll::Ready(None);
}
match Pin::new(&mut this.stream).poll_next(cx) {
Poll::Ready(Some(data)) => {
// Parsing Logic Here
return Poll::Ready(Some(Ok::<_, Error>(resp)));
}
Poll::Ready(None) => return Poll::Ready(None),
Poll::Pending => {}
}
}
}
}
Trying to remove the loop (and changing the Poll::Pending match arm to Poll::Pending => Poll::Pending) generally results in poll only being called once before hanging, from my very rough understanding of why this is, it's because I'm dropping the reference to the waker when I return from this function, as it's not stored anywhere.
However I'm struggling to work out how to arrange my struct/code to enable the storage of that reference or alternatively what the best way to enable the use of that waker is? Is anyone able to explain how this problem can be solved?
Many thanks in advance!
I'm writing a (toy) hash-and-cache decorator in TypeScript and can't find a good means of creating a solid generic one.
The code I have so far is
function cache
(target: Object,
propertyKey: string,
// Likely we can do better than <any> here -- <Function<any>> maybe?
descriptor: TypedPropertyDescriptor<any>)
{
let cacheMap = new Map();
let wrappedFn = descriptor.value;
descriptor.value = function (...args: any[]) {
if (cacheMap.has(args)) {
console.log("Short-circuiting with result: " + cacheMap.get(args));
return cacheMap.get(args);
}
let result = wrappedFn.apply(this, args);
cacheMap.set(args, result);
console.log("cacheMap %o", cacheMap);
return result;
}
return descriptor;
}
Naturally this fails, since args is not a tuple but a list, which is mutable[1]. So each input, even if it's the same over and over, gets its own list/array in its own memory location with its own hash value, wherever that comes from.
I haven't found a Tuple type in TypeScript (or JS) yet -- is there one? Is there another workaround for this sort of problem?
Shouldn't this be an error? Map<T, U> should constrain T to implementing IHashable or something, right? That's the point of types -- to raise this issue before it takes a bunch of time out of your life.
Shouldn't this be an error? Map<T, U> should constrain T to implementing IHashable or something, right?
No. Object identity is a real and well-defined thing in JavaScript; TypeScript doesn't attempt to force you to pretend it doesn't exist.
If the ECMAScript committee thought it was appropriate to enforce non-object-identity-based keying in maps, they could have restricted Map keys, but they didn't.
When I use FFI to wrap some API (for example DOM API) is there any rule of thumb that could help me to decide whether function should be effectful or not?
Here is an example:
foreign import querySelectorImpl """
function querySelectorImpl (Nothing) {
return function (Just) {
return function (selector) {
return function (src) {
return function () {
var result = src.querySelector(selector);
return result ? Just(result) : Nothing;
};
};
};
};
}
""" :: forall a e. Maybe a -> (a -> Maybe a) -> String -> Node -> Eff (dom :: DOM | e) (Maybe Node)
querySelector :: forall e. String -> Node -> Eff (dom :: DOM | e) (Maybe Node)
querySelector = querySelectorImpl Nothing Just
foreign import getTagName """
function getTagName (n) {
return function () {
return n.tagName;
};
}
""" :: forall e. Node -> Eff (dom :: DOM | e) String
It feels right for querySelector to be effectful, but I'm not quite sure about getTagName
Update:
I understand what a pure function is and that it should not change the state of the program and maybe DOM was a bad example.
I ask this question because in most libraries that wrap existing js libraries pretty much every function is effectful even if it doesn't feels right. So maybe my actual question is - does this effect represent the need in this wrapped js lib or is it there just in case it is stateful inside?
If a function does not change state, and it always (past, present, and future) returns the same value when given the same arguments, then it does not need to return Eff, otherwise it does.
n.tagName is read-only, and as far as I know, it never changes. Therefore, getTagName is pure, and it's okay to not return Eff.
On the other hand, a getTextContent function must return Eff. It does not change state, but it does return different values at different times.
The vast vast vast majority of JS APIs (including the DOM) are effectful. getTagName is one of the very few exceptions. So when writing an FFI, PureScript authors just assume that all JS functions return Eff, even in the rare situations where they don't need to.
Thankfully the most recent version of purescript-dom uses non-Eff functions for nodeName, tagName, localName, etc.
Effectful functions are functions that are not pure, from Wikipedia:
In computer programming, a function may be described as a pure function if both these statements about the function hold:
The function always evaluates the same result value given the same argument value(s). The function result value cannot depend on any hidden information or state that may change as program execution proceeds or between different executions of the program, nor can it depend on any external input from I/O devices [...].
Evaluation of the result does not cause any semantically observable side effect or output, such as mutation of mutable objects or output to I/O devices [...].
Since the DOM stores state, functions wrapping calls to the DOM are almost always effectful.
For more details regarding PureScript, see Handling Native Effects with the Eff Monad.
I currently have an agent that does heavy data processing by constantly posting "work" messages to itself.
Sometimes clients to this agent wants to interrupt this processing to access the data in a safe manner.
For this I thought that posting an async to the agent that the agent can run whenever it's in a safe state would be nice. This works fine and the message looks like this:
type Message = |Sync of Async<unit>*AsyncReplyChannel<unit>
And the agent processing simply becomes:
match mailbox.Receive () with
| Sync (async, reply) -> async |> Async.RunSynchronously |> reply.Reply
This works great as long as clients don't need to return some value from the async as I've constrained the async/reply to be of type unit and I cannot use a generic type in the discriminated union.
My best attempts to solve this has involved wrapper asyncs and waithandles, but this seems messy and not as elegant as I've come to expect from F#. I'm also new to async workflows in F# so it's very possible that I've missed/misunderstood some concepts here.
So the question is; how can I return generic types in a agent response?
The thing that makes this difficult is that, in your current version, the agent would somehow have to calculate the value and then pass it to the channel, without knowing what is the type of the value. Doing that in a statically typed way in F# is tricky.
If you make the message generic, then it will work, but the agent will only be able to handle messages of one type (the type T in Message<T>).
An alternative is to simply pass Async<unit> to the agent and let the caller do the value passing for each specific type. So, you can write message & agent just like this:
type Message = | Sync of Async<unit>
let agent = MailboxProcessor.Start(fun inbox -> async {
while true do
let! msg = inbox.Receive ()
match msg with
| Sync (work) -> do! work })
When you use PostAndReply, you get access to the reply channel - rather than passing the channel to the agent, you can just use it in the local async block:
let num = agent.PostAndReply(fun chan -> Sync(async {
let ret = 42
chan.Reply(ret) }))
let str = agent.PostAndReply(fun chan -> Sync(async {
let ret = "hi"
chan.Reply(ret) }))
I understand how to make a message based non-blocking application in akka, and can easily mock up examples that perform
concurrent operations and pass back the aggregated results in a message. Where I have difficulty is understanding what my
non-blocking options are when my application has to respond to an HTTP request. The goal is to receive a request and
immediately hand it over to a local or remote actor to do the work, which in turn will hand it off to get a result that
could take some time. Unfortunatly under this model, I don't understand how I could express this with a non-blocking
series of "tells" rather than blocking "asks". If at any point in the chain I use a tell, I no longer have a future to
use as the eventual response content (required by the http framework interface which in this case is finagle - but that is not
important). I understand the request is on its own thread, and my example is quite contrived, but just trying to
understand my design options.
In summary, If my contrived example below can be reworked to block less I very much love to understand how. This is my
first use of akka since some light exploration a year+ ago, and in every article, document, and talk I have viewed says
not to block for services.
Conceptual answers may be helpful but may also be the same as what I have already read. Working/Editing my example
would likely be key to my understanding of the exact problem I am attempting to solve. If the current example is generally
what needs to be done that confirmation is helpful too, so I don't search for magic that does not exist.
Note The following aliases: import com.twitter.util.{Future => TwitterFuture, Await => TwitterAwait}
object Server {
val system = ActorSystem("Example-System")
implicit val timeout = Timeout(1 seconds)
implicit def scalaFuture2twitterFuture[T](scFuture: Future[T]): TwitterFuture[T] = {
val promise = TwitterPromise[T]
scFuture onComplete {
case Success(result) ⇒ promise.setValue(result)
case Failure(failure) ⇒ promise.setException(failure)
}
promise
}
val service = new Service[HttpRequest, HttpResponse] {
def apply(req: HttpRequest): TwitterFuture[HttpResponse] = req.getUri match {
case "/a/b/c" =>
val w1 = system.actorOf(Props(new Worker1))
val r = w1 ? "take work"
val response: Future[HttpResponse] = r.mapTo[String].map { c =>
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
response
}
}
//val server = Http.serve(":8080", service); TwitterAwait.ready(server)
class Worker1 extends Actor with ActorLogging {
def receive = {
case "take work" =>
val w2 = context.actorOf(Props(new Worker2))
pipe (w2 ? "do work") to sender
}
}
class Worker2 extends Actor with ActorLogging {
def receive = {
case "do work" =>
//Long operation...
sender ! "The Work"
}
}
def main(args: Array[String]) {
val r = service.apply(
com.twitter.finagle.http.Request("/a/b/c")
)
println(TwitterAwait.result(r).getContent.toString(CharsetUtil.UTF_8)) // prints The Work
}
}
Thanks in advance for any guidance offered!
You can avoid sending a future as a message by using the pipe pattern—i.e., in Worker1 you'd write:
pipe(w2 ? "do work") to sender
Instead of:
sender ! (w2 ? "do work")
Now r will be a Future[String] instead of a Future[Future[String]].
Update: the pipe solution above is a general way to avoid having your actor respond with a future. As Viktor points out in a comment below, in this case you can take your Worker1 out of the loop entirely by telling Worker2 to respond directly to the actor that it (Worker1) got the message from:
w2.tell("do work", sender)
This won't be an option if Worker1 is responsible for operating on the response from Worker2 in some way (by using map on w2 ? "do work", combining multiple futures with flatMap or a for-comprehension, etc.), but if that's not necessary, this version is cleaner and more efficient.
That kills one Await.result. You can get rid of the other by writing something like the following:
val response: Future[HttpResponse] = r.mapTo[String].map { c =>
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
Now you just need to turn this Future into a TwitterFuture. I can't tell you off the top of my head exactly how to do this, but it should be fairly trivial, and definitely doesn't require blocking.
You definitely don't have to block at all here. First, update your import for the twitter stuff to:
import com.twitter.util.{Future => TwitterFuture, Await => TwitterAwait, Promise => TwitterPromise}
You will need the twitter Promise as that's the impl of Future you will return from the apply method. Then, follow what Travis Brown said in his answer so your actor is responding in such a way that you do not have nested futures. Once you do that, you should be able to change your apply method to something like this:
def apply(req: HttpRequest): TwitterFuture[HttpResponse] = req.getUri match {
case "/a/b/c" =>
val w1 = system.actorOf(Props(new Worker1))
val r = (w1 ? "take work").mapTo[String]
val prom = new TwitterPromise[HttpResponse]
r.map(toResponse) onComplete{
case Success(resp) => prom.setValue(resp)
case Failure(ex) => prom.setException(ex)
}
prom
}
def toResponse(c:String):HttpResponse = {
val resp = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)
resp.setContent(ChannelBuffers.copiedBuffer(c, CharsetUtil.UTF_8))
resp
}
This probably needs a little more work. I didn't set it up in my IDE, so I can't guarantee you it compiles, but I believe the idea to be sound. What you return from the apply method is a TwitterFuture that is not yet completed. It will be completed when the future from the actor ask (?) is done and that's happing via a non-blocking onComplete callback.