F# Pattern-matching & recursion vs looping & if..then's for parsing nested structures - recursion

I'm using a 3rd party vendor's API in F#. On initialization the API returns a C# object that is nested msg container. It is populated with status messages and may include errors message. The vendor provides a C# sample parsing routine which I have ported F#.
The code sample loops through a nested msg container extracting fatal and nonfatal errors, and then return a List of tuples of type BBResponseType * string
Response Enum:
type BBResponseType =
| Status = 0
| Data = 1
| Error = 2
| FatalError = -1
My port to F# looks like this:
member private this.ProcessStatusMsg(eventObj: Blpapi.Event) =
let responseLst = List<(BBResponseType * string)>()
for msg in eventObj do
if msg.MessageType.Equals(SUBSTARTED) then
if msg.GetElement(EXCEPTIONS).NumValues > 0 then // <- check for errors/exceptions
let e = msg.GetElement(EXCEPTIONS)
for i in 0..e.NumValues-1 do
let error = e.GetValueAsElement(i)
let field = error.GetElementAsString(FieldID)
let reason = error.GetElement(REASON)
let message = sprintf "Subscription Started w/errors( Field: %s \n Reason: %s)" field (reason.GetElementAsString(DESCRIPTION))
responseLst.Add(BBResponseType.Error, message)
else
let message = sprintf "Subscription Started"
responseLst.Add(BBResponseType.Status, message)
if msg.MessageType.Equals(SUBSCFAILURE) then // <- check for subscriptions failure
if msg.HasElement(REASON) then
let reason = msg.GetElement(REASON)
let desc = reason.GetElementAsString(DESCRIPTION)
let message = sprintf "Real-time Subscription Failure: %s" desc
responseLst.Add(BBResponseType.FatalError, message)
else
let message = sprintf "Subscription Failure: (reason unknown) "
responseLst.Add(BBResponseType.FatalError, message)
responseLst
After I finished it, I looked at it and thought, "Wow, that's about as non-functional as you can get and still code in F#."
It does seem a lot clearer and succinct than the C# version, but I was thinking that there must be a better way to do all this without using so many loops and if/then's.
How can I do a better job of parsing these nested structures using pattern matching and recursion?

Few pointers:
Instead of returning a List of tuple return a seq of tuple - using the seq { } computation expression for creating sequence.
Extract the if/else parts as a function of type Message -> (BBResponseType * string) and use this function inside the seq expression
Inside this new function (which transforms the Message to tuple) use pattern matching to figure out what kind of (BBResponseType * string) to return.

To complement #Ankur's answer:
member private this.ProcessStatusMsg(eventObj: Blpapi.Event) =
// 0. Define a parameterized active pattern to turn if/else into pattern matching
let (|Element|_|) e msg =
if msg.HasElement(e) then
Some <| msg.GetElement(e)
else None
// 1. Wrapping the whole method body in a sequence expression
seq {
for msg in eventObj do
// 2. Extracting if/else part and using it in sequence expression
match msg.MessageType with
// 3. Using pattern matching to figure out what kind (BBResponseType * string) to return
| SUBSTARTED ->
match msg with
// 4. Use active pattern to pattern match on message directly
| Element EXCEPTIONS e when e.NumValues > 0 ->
for i in 0..e.NumValues-1 do
let error = e.GetValueAsElement(i)
let field = error.GetElementAsString(FieldID)
let reason = error.GetElement(REASON)
let message = sprintf "Subscription Started w/errors( Field: %s \n Reason: %s)" field (reason.GetElementAsString(DESCRIPTION))
yield (BBResponseType.Error, message)
| _ ->
let message = sprintf "Subscription Started"
yield (BBResponseType.Status, message)
| SUBSCFAILURE ->
match msg with
| Element REASON reason ->
let desc = reason.GetElementAsString(DESCRIPTION)
let message = sprintf "Real-time Subscription Failure: %s" desc
yield (BBResponseType.FatalError, message)
| _ ->
let message = sprintf "Subscription Failure: (reason unknown) "
yield (BBResponseType.FatalError, message)
// There are probably more cases, processing them here
| _ -> ()
}
Point 1, 2 and 3 in comments are from the other answer. I added point 0 and 4 to use active patterns for easy pattern matching.

Related

How do I check 'T for types not being allowed to be null?

given
let inline deserialize<'t> x :'t option =
printfn "Attempting to deserialize %A" typeof<'t>.Name
try
JsonConvert.DeserializeObject<'t>(x)
|> Some
with ex ->
System.Diagnostics.Trace.WriteLine(sprintf "Error deserialization failed:%s" ex.Message)
None
is returning for example an obj list as null. FSharpList<_> is not allowed to be null. How can I, without knowing what 't is ask F# if the type I'm about to return supports null so that I can halt/throw/act accordingly? Is there a reflection flag or Microsoft.FSharp.Reflection... method for this?
The full answer involves checking if the type is a record (in which case null is never allowed), or if it's a union (in which case null is allowed if the type has a CompilationRepresentation CustomAttribute whose flags contain the UseNullAsTrueValue member (https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/core.compilationrepresentationflags-enumeration-%5Bfsharp%5D for more details)).
To answer the first question you can use the IsRecord function in the FSharpType module (https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual/reflection.fsharptype-class-%5Bfsharp%5D) and to answer the second you can use a combination of the IsUnion function on that same module and CustomAttribute hunting.
In case the type is a union with UseNullAsTrueValue set, you should be good to go, just send the value along.
Best I can think of is to box the result (in case you are deserializing structs) and pattern match it with null:
let inline deserialize<'t> x :'t option =
printfn "Attempting to deserialize %A" typeof<'t>.Name
try
let obj = Newtonsoft.Json.JsonConvert.DeserializeObject<'t>(x)
match box obj with
| null -> None
| _ -> Some obj
with ex ->
System.Diagnostics.Trace.WriteLine(sprintf "Error deserialization failed:%s" ex.Message)
None
let r1 = deserialize<obj list> ("[1,2,3]") //val r1 : obj list option = Some [1L; 2L; 3L]
let r2 = deserialize<obj list> ("null") //val r2 : obj list option = None

sendMessage in functional style

Suppose I have 2 messenger objects that can send and receive messages. When message is received it is appended to some file:
class Messenger:
def sendMessage(anotherMessenger, message):
anotherMessenger.receiveMessage(message)
def receiveMessage(msg):
# appending msg to file
a = Messenger()
b = Messenger()
a.sendMessage(b, "Hello b!")
b.sendMessage(a, "Hi a!")
What would be a functional programming equivalent of the program above?
In FP your 'objects' tend to be functions. In this case, instead of multiple objects, each with two member functions you would have multiple functions to handle different scenarios.
In this case you have:
receiveMessage: string -> ()
That is, receiveMessage is a function that takes a string and returns nothing - unit. Likewise:
sendMessage: (f: string -> ()) -> (msg: string) -> ()
I.e. sendMessage takes a receiveMessage implementation and a string and returns unit.
So, using a language like F# we can have the following:
let sendMessage (f: string -> ()) (msg: string) = f(msg)
let aRecv (msg: string) = // Do something with msg
let bRecv (msg: string) = // Do something else with msg
...
sendMessage bRecv "Hello b!"
sendMessage aRecv "Hi a!"
...
Since your question is kind of generic I will leave this as a kind of generic answer. If you have something more specific in mind this answer can be expanded on.

Defining a Message Passing domain with very many message types

Most F# Message Passing examples I've seen so far are working with 2-4 message types, and are able to utilize pattern matching to direct each message to its proper handler function.
For my application, I need hundreds of unique message types due to the different nature of their handling and required parameters. So far, each message type is its own record type with a marker interface attached, because including hundreds of types in a single discriminated union would not be very pretty - and neither would the pattern matching of these be. As a result, I'm currently using reflection to find the correct handler functions of messages.
Is there a better, and more functional way of doing this? Perhaps even a smarter way to define such a domain? I'd like to enforce as much correctness as possible at compile time, but currently I'm finding the handler functions based on a custom attribute, as well as checking their signature at run time.
As far as I know, I cannot enforce a function's signature with a .NET custom attribute, and since there are too many types to realistically pattern match, I can't (to my knowledge) use a single generic message handler function either. I tried using a generic wrapper function as an "interface" for all handlers, and only attaching the custom attribute to this one, but this didn't grant the wrapped functions the attribute and make them visible through reflection based on that attribute (I'm very new to .NET).
I have thought about the possibility of attaching the handler functions to their respective record type as a member, which would circumvent the need for reflection and enforce some additional correctness at compile time. However, it doesn't make much sense to have all those functions present client side.
The question is a little broad but I will give it a try.
Lets start with the types. Our message will have a type and a content. You can add more fields like messageId, sender, receiver, etc.
type MessageType = MessageType of string
type Message<'T> = {
messageType : MessageType
message : 'T
}
Similarly our handler type will pair both type and handler function.
type HandlerResult = Result<string, string>
type MessageHandler<'T> = {
messageType : MessageType
handlerF : Message<'T> -> HandlerResult
}
We want to have someplace to register all handlers with their types. A Dictionary is ideal because it is fast:
let Handlers = System.Collections.Generic.Dictionary<MessageType, MessageHandler<obj>>()
The only thing is the dictionary cannot have a generic type so all handlers here are going to be of type MessageHandler<obj>. Because of that we need to be able to convert <'T> messages and handlers to <obj> messages and handlers and back. Se here we have a helper function:
let ofMessageGen (msg: Message<obj>) : Message<_> = {
messageType = msg.messageType
message = unbox msg.message
}
and a function to register the handler function as an <obj> handler:
let registerHandler (handlerF:Message<'T> -> HandlerResult) =
let handler = {
messageType = MessageType <| (typeof<'T>).FullName
handlerF = ofMessageGen >> handlerF
}
Handlers.Add(handler.messageType, handler )
With that we can register handlers for any types:
registerHandler (fun msg -> sprintf "String message: %s" msg.message |> Ok )
registerHandler (fun msg -> sprintf "int message: %d" msg.message |> Ok )
registerHandler (fun msg -> sprintf "float message: %f" msg.message |> Ok )
and here is our generic message handler:
let genericHandler (msg:Message<obj>) : HandlerResult =
match Handlers.TryGetValue msg.messageType with
| false, _ -> Error <| sprintf "No Handler for message: %A" msg
| true , handler -> handler.handlerF msg
to create a message:
let createMessage (m:'T) = {
messageType = MessageType <| (typeof<'T>).FullName
message = box m
}
and test it like this:
createMessage "Hello" |> genericHandler |> printfn "%A"
createMessage 123 |> genericHandler |> printfn "%A"
createMessage 123.4 |> genericHandler |> printfn "%A"
createMessage true |> genericHandler |> printfn "%A"
// Ok "String message: Hello"
// Ok "int message: 123"
// Ok "float message: 123.400000"
// Error
// "No Handler for message: {messageType = MessageType "System.Boolean";
// message = true;}"

Railway oriented programming with Async operations

Previously asked similar question but somehow I'm not finding my way out, attempting again with another example.
The code as a starting point (a bit trimmed) is available at https://ideone.com/zkQcIU.
(it has some issue recognizing Microsoft.FSharp.Core.Result type, not sure why)
Essentially all operations have to be pipelined with the previous function feeding the result to the next one. The operations have to be async and they should return error to the caller in case an exception occurred.
The requirement is to give the caller either result or fault. All functions return a Tuple populated with either Success type Article or Failure with type Error object having descriptive code and message returned from the server.
Will appreciate a working example around my code both for the callee and the caller in an answer.
Callee Code
type Article = {
name: string
}
type Error = {
code: string
message: string
}
let create (article: Article) : Result<Article, Error> =
let request = WebRequest.Create("http://example.com") :?> HttpWebRequest
request.Method <- "GET"
try
use response = request.GetResponse() :?> HttpWebResponse
use reader = new StreamReader(response.GetResponseStream())
use memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(reader.ReadToEnd()))
Ok ((new DataContractJsonSerializer(typeof<Article>)).ReadObject(memoryStream) :?> Article)
with
| :? WebException as e ->
use reader = new StreamReader(e.Response.GetResponseStream())
use memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(reader.ReadToEnd()))
Error ((new DataContractJsonSerializer(typeof<Error>)).ReadObject(memoryStream) :?> Error)
Rest of the chained methods - Same signature and similar bodies. You can actually reuse the body of create for update, upload, and publish to be able to test and compile code.
let update (article: Article) : Result<Article, Error>
// body (same as create, method <- PUT)
let upload (article: Article) : Result<Article, Error>
// body (same as create, method <- PUT)
let publish (article: Article) : Result<Article, Error>
// body (same as create, method < POST)
Caller Code
let chain = create >> Result.bind update >> Result.bind upload >> Result.bind publish
match chain(schemaObject) with
| Ok article -> Debug.WriteLine(article.name)
| Error error -> Debug.WriteLine(error.code + ":" + error.message)
Edit
Based on the answer and matching it with Scott's implementation (https://i.stack.imgur.com/bIxpD.png), to help in comparison and in better understanding.
let bind2 (switchFunction : 'a -> Async<Result<'b, 'c>>) =
fun (asyncTwoTrackInput : Async<Result<'a, 'c>>) -> async {
let! twoTrackInput = asyncTwoTrackInput
match twoTrackInput with
| Ok s -> return! switchFunction s
| Error err -> return Error err
}
Edit 2 Based on F# implementation of bind
let bind3 (binder : 'a -> Async<Result<'b, 'c>>) (asyncResult : Async<Result<'a, 'c>>) = async {
let! result = asyncResult
match result with
| Error e -> return Error e
| Ok x -> return! binder x
}
Take a look at the Suave source code, and specifically the WebPart.bind function. In Suave, a WebPart is a function that takes a context (a "context" is the current request and the response so far) and returns a result of type Async<context option>. The semantics of chaining these together are that if the async returns None, the next step is skipped; if it returns Some value, the next step is called with value as the input. This is pretty much the same semantics as the Result type, so you could almost copy the Suave code and adjust it for Result instead of Option. E.g., something like this:
module AsyncResult
let bind (f : 'a -> Async<Result<'b, 'c>>) (a : Async<Result<'a, 'c>>) : Async<Result<'b, 'c>> = async {
let! r = a
match r with
| Ok value ->
let next : Async<Result<'b, 'c>> = f value
return! next
| Error err -> return (Error err)
}
let compose (f : 'a -> Async<Result<'b, 'e>>) (g : 'b -> Async<Result<'c, 'e>>) : 'a -> Async<Result<'c, 'e>> =
fun x -> bind g (f x)
let (>>=) a f = bind f a
let (>=>) f g = compose f g
Now you can write your chain as follows:
let chain = create >=> update >=> upload >=> publish
let result = chain(schemaObject) |> Async.RunSynchronously
match result with
| Ok article -> Debug.WriteLine(article.name)
| Error error -> Debug.WriteLine(error.code + ":" + error.message)
Caution: I haven't been able to verify this code by running it in F# Interactive, since I don't have any examples of your create/update/etc. functions. It should work, in principle — the types all fit together like Lego building blocks, which is how you can tell that F# code is probably correct — but if I've made a typo that the compiler would have caught, I don't yet know about it. Let me know if that works for you.
Update: In a comment, you asked whether you need to have both the >>= and >=> operators defined, and mentioned that you didn't see them used in the chain code. I defined both because they serve different purposes, just like the |> and >> operators serve different purposes. >>= is like |>: it passes a value into a function. While >=> is like >>: it takes two functions and combines them. If you would write the following in a non-AsyncResult context:
let chain = step1 >> step2 >> step3
Then that translates to:
let asyncResultChain = step1AR >=> step2AR >=> step3AR
Where I'm using the "AR" suffix to indicate versions of those functions that return an Async<Result<whatever>> type. On the other hand, if you had written that in a pass-the-data-through-the-pipeline style:
let result = input |> step1 |> step2 |> step3
Then that would translate to:
let asyncResult = input >>= step1AR >>= step2AR >>= step3AR
So that's why you need both the bind and compose functions, and the operators that correspond to them: so that you can have the equivalent of either the |> or the >> operators for your AsyncResult values.
BTW, the operator "names" that I picked (>>= and >=>), I did not pick randomly. These are the standard operators that are used all over the place for the "bind" and "compose" operations on values like Async, or Result, or AsyncResult. So if you're defining your own, stick with the "standard" operator names and other people reading your code won't be confused.
Update 2: Here's how to read those type signatures:
'a -> Async<Result<'b, 'c>>
This is a function that takes type A, and returns an Async wrapped around a Result. The Result has type B as its success case, and type C as its failure case.
Async<Result<'a, 'c>>
This is a value, not a function. It's an Async wrapped around a Result where type A is the success case, and type C is the failure case.
So the bind function takes two parameters:
a function from A to an async of (either B or C)).
a value that's an async of (either A or C)).
And it returns:
a value that's an async of (either B or C).
Looking at those type signatures, you can already start to get an idea of what the bind function will do. It will take that value that's either A or C, and "unwrap" it. If it's C, it will produce an "either B or C" value that's C (and the function won't need to be called). If it's A, then in order to convert it to an "either B or C" value, it will call the f function (which takes an A).
All this happens within an async context, which adds an extra layer of complexity to the types. It might be easier to grasp all this if you look at the basic version of Result.bind, with no async involved:
let bind (f : 'a -> Result<'b, 'c>) (a : Result<'a, 'c>) =
match a with
| Ok val -> f val
| Error err -> Error err
In this snippet, the type of val is 'a, and the type of err is 'c.
Final update: There was one comment from the chat session that I thought was worth preserving in the answer (since people almost never follow chat links). Developer11 asked,
... if I were to ask you what Result.bind in my example code maps to your approach, can we rewrite it as create >> AsyncResult.bind update? It worked though. Just wondering i liked the short form and as you said they have a standard meaning? (in haskell community?)
My reply was:
Yes. If the >=> operator is properly written, then f >=> g will always be equivalent to f >> bind g. In fact, that's precisely the definition of the compose function, though that might not be immediately obvious to you because compose is written as fun x -> bind g (f x) rather than as f >> bind g. But those two ways of writing the compose function would be exactly equivalent. It would probably be very instructive for you to sit down with a piece of paper and draw out the function "shapes" (inputs & outputs) of both ways of writing compose.
Why do you want to use Railway Oriented Programming here? If you just want to run a sequence of operations and return information about the first exception that occurs, then F# already provides a language support for this using exceptions. You do not need Railway Oriented Programming for this. Just define your Error as an exception:
exception Error of code:string * message:string
Modify the code to throw the exception (also note that your create function takes article but does not use it, so I deleted that):
let create () = async {
let ds = new DataContractJsonSerializer(typeof<Error>)
let request = WebRequest.Create("http://example.com") :?> HttpWebRequest
request.Method <- "GET"
try
use response = request.GetResponse() :?> HttpWebResponse
use reader = new StreamReader(response.GetResponseStream())
use memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(reader.ReadToEnd()))
return ds.ReadObject(memoryStream) :?> Article
with
| :? WebException as e ->
use reader = new StreamReader(e.Response.GetResponseStream())
use memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(reader.ReadToEnd()))
return raise (Error (ds.ReadObject(memoryStream) :?> Error)) }
And then you can compose functions just by sequencing them in async block using let! and add exception handling:
let main () = async {
try
let! created = create ()
let! updated = update created
let! uploaded = upload updated
Debug.WriteLine(uploaded.name)
with Error(code, message) ->
Debug.WriteLine(code + ":" + message) }
If you wanted more sophisticated exception handling, then Railway Oriented Programming might be useful and there is certainly a way of integrating it with async, but if you just want to do what you described in your question, then you can do that much more easily with just standard F#.

Monadic Retry logic w/ F# and async?

I've found this snippet:
http://fssnip.net/8o
But I'm working not only with retriable functions, but also with asynchronous such, and I was wondering how I make this type properly. I have a tiny piece of retryAsync monad that I'd like to use as a replacement for async computations, but that contains retry logic, and I'm wondering how I combine them?
type AsyncRetryBuilder(retries) =
member x.Return a = a // Enable 'return'
member x.ReturnFrom a = x.Run a
member x.Delay f = f // Gets wrapped body and returns it (as it is)
// so that the body is passed to 'Run'
member x.Bind expr f = async {
let! tmp = expr
return tmp
}
member x.Zero = failwith "Zero"
member x.Run (f : unit -> Async<_>) : _ =
let rec loop = function
| 0, Some(ex) -> raise ex
| n, _ ->
try
async { let! v = f()
return v }
with ex -> loop (n-1, Some(ex))
loop(retries, None)
let asyncRetry = AsyncRetryBuilder(4)
Consuming code is like this:
module Queue =
let desc (nm : NamespaceManager) name = asyncRetry {
let! exists = Async.FromBeginEnd(name, nm.BeginQueueExists, nm.EndQueueExists)
let beginCreate = nm.BeginCreateQueue : string * AsyncCallback * obj -> IAsyncResult
return! if exists then Async.FromBeginEnd(name, nm.BeginGetQueue, nm.EndGetQueue)
else Async.FromBeginEnd(name, beginCreate, nm.EndCreateQueue)
}
let recv (client : MessageReceiver) timeout =
let bRecv = client.BeginReceive : TimeSpan * AsyncCallback * obj -> IAsyncResult
asyncRetry {
let! res = Async.FromBeginEnd(timeout, bRecv, client.EndReceive)
return res }
Error is:
This expression was expected to have type Async<'a> but here has type 'b -> Async<'c>
Your Bind operation behaves like a normal Bind operation of async, so your code is mostly a re-implementation (or wrapper) over async. However, your Return does not have the right type (it should be 'T -> Async<'T>) and your Delay is also different than normal Delay of async. In general, you should start with Bind and Return - using Run is a bit tricky, because Run is used to wrap the entire foo { .. } block and so it does not give you the usual nice composability.
The F# specification and a free chapter 12 from Real-World Functional Programming both show the usual types that you should follow when implementing these operations, so I won't repeat that here.
The main issue with your approach is that you're trying to retry the computation only in Run, but the retry builder that you're referring to attempts to retry each individual operation called using let!. Your approach may be sufficient, but if that's the case, you can just implement a function that tries to run normal Async<'T> and retries:
let RetryRun count (work:Async<'T>) = async {
try
// Try to run the work
return! work
with e ->
// Retry if the count is larger than 0, otherwise fail
if count > 0 then return! RetryRun (count - 1) work
else return raise e }
If you actually want to implement a computation builder that will implicitly try to retry every single asynchronous operation, then you can write something like the following (it is just a sketch, but it should point you in the right direction):
// We're working with normal Async<'T> and
// attempt to retry it until it succeeds, so
// the computation has type Async<'T>
type RetryAsyncBuilder() =
member x.ReturnFrom(comp) = comp // Just return the computation
member x.Return(v) = async { return v } // Return value inside async
member x.Delay(f) = async { return! f() } // Wrap function inside async
member x.Bind(work, f) =
async {
try
// Try to call the input workflow
let! v = work
// If it succeeds, try to do the rest of the work
return! f v
with e ->
// In case of exception, call Bind to try again
return! x.Bind(work, f) }

Resources