I'm a newbie to Elixir and am trying to write a GenServer that stores key, value pairs in a HashDict. Storing a compound key and value is fine. here's the code:
#Initialise the HashDict GenServer.start_link
def init(:ok) do
{:ok, HashDict.new}
end
#Implement the server call back for GenServer.cast
def handle_cast({:add, event}, dict) do
{foo, bar, baz, qux} = event
key = %{key1: foo, key2: bar}
value = %{val1: baz, val2: qux}
{:noreply, HashDict.put(dict, key, value) }
end
All good. But I'm having trouble implementing the handle_call behaviour that I want. So here I'd like:
For a given key1 value, retrieve all corresponding value entries in HashDict. This will mean ignoring the value for key2 (so kind of a select all).
Having returned all the val2s, add them all up (assuming they are integers, ignoring val1) to give an overall sum.
So I've got this far:
def handle_call({:get, getKey}, _from, dict) do
key = %{key1: getKey, key2: _}
{:reply, HashDict.fetch(dict, key), dict}
end
This doesn't work, as it's not possible to pattern match on _. Presumably I would use some kind of Enumeration over the map such as the following to achieve my second objective:
Enum.map(mymap, fn {k, v} -> v end)|> Enum.sum{}
But I can't seem to quite crack the syntax to achieve my two aims. Thanks for any help!
If I understand your question correctly, the following should accomplish what you are wanting to do:
def handle_call({:get, getKey}, _from, dict) do
sum = Enum.reduce(dict, 0, fn
({%{key1: key1}, %{val2: val2}}, acc)
when key1 === getKey
and is_integer(val2) ->
val2 + acc
(_, acc) ->
acc
end)
{:reply, sum, dict}
end
See the documentation of Enum.reduce/3 for more information.
Related
I am new to functional programming and so can not imagen how to build the new dictionary based on two other dictionaries with similar set of keys. The new dictionary will have the entries with all keys but values will be selected/computed based on some condition.
For example, having two dictionaries:
D1: [(1,100);(2,50);(3,150)]
D2: [(1,20);(2,30);(3,0);(4,10)]
and condition to get the average of two values, the resulting dictionary will be
DR: [(1,60);(2,40);(3,75);(4,10)]
I need implementation in F#.
Please could you give me some advise.
View them as two (or more...) lists of tuples that we concat makes it easier. The below solves your specfic problem. To generalise the process aggeragting a list of values to something specific you would need to change averageBy to fold and provide a fold function instead of float. Assuming d1 and d2 mataches your exmaple.
Seq.concat [ d1 ; d2 ]
|> Seq.map (|KeyValue|)
|> Seq.groupBy fst
|> Seq.map (fun (k, c) -> k, Seq.averageBy (snd >> float) c |> int)
|> dict
If you wanted to use an external library, you could do this using Deedle series, which has various operations for working with (time) series of data.
Here, you have two data series that have different keys. Deedle lets you zip series based on keys and handle the cases where one of the values is missing using the opt type:
#r "nuget:Deedle"
open Deedle
let s1 = series [(1,100);(2,50);(3,150)]
let s2 = series [(1,20);(2,30);(3,0);(4,10)]
Series.zip s1 s2
|> Series.mapValues (fun (v1, v2) ->
( (OptionalValue.defaultArg 0 v1) +
(OptionalValue.defaultArg 0 v2) ) / 2)
This may not make sense if this is a thing that you need just in one or two places, but if you're working with key-value series of data more generally, it may be worth checking out.
Solution 1
From a functional perspective I would use a Map data-structure, instead of a dictionary. You can convert a dictionary to a Map like this
let d1 = dict [(1,100);(2,50);(3,150)]
let m1 = Map [for KeyValue (key,value) in d1 -> key, value]
But i wouldn't use a Dictionary and convert it, I would use a Map diretly.
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
Next, you need a way to get all keys from both Maps. You can get the keys of a map with Map.keys but you need all the keys from both. You could get them by using a Set.
let keys = Set (Map.keys m1) + Set (Map.keys m2)
By adding two Sets you get a Set.union of both sets. Once you have them, you can traverse the keys, and try to get both values from both keys. If you use Map.find then you get an optional. You can Pattern match on both cases at once.
let result = Map [
for key in keys do
match Map.tryFind key m1, Map.tryFind key m2 with
| Some x, Some y -> key, (x + y) / 2
| Some x, None -> key, x
| None , Some y -> key, y
| None , None -> failwith "Cannot happen"
]
This creates a new Map data-structure and saves it into result. If both cases are Some then you compute the average, otherwise you just keep the value. As you iterate the keys of both Maps the None,None case cannot happen. A Key always must be in either one or the other.
After all of this, result will be:
Map [(1, 60); (2, 40); (3, 75); (4, 10)]
Again, here is the whole code at once:
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
let keys = Set (Map.keys m1) + Set (Map.keys m2)
let result = Map [
for key in keys do
match Map.tryFind key m1, Map.tryFind key m2 with
| Some x, Some y -> key, (x + y) / 2
| Some x, None -> key, x
| None , Some y -> key, y
| None , None -> failwith "Cannot happen"
]
You also can inline the keys variable, if you want.
Solution 2
When you have a Map then you can make use of the fact that adding a value always to a map, always creates a new Map data-structure. This way you are able to use Map.fold that traverses a Map data-structure and uses one of the map as the starting state while you traverse the other Map.
With Map.change you then can read and change a value in one step. If a key is already available you calculate the average, otherwise just add the value.
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
let result =
(m1,m2) ||> Map.fold (fun state key y ->
state |> Map.change key (function
| Some x -> Some ((x + y) / 2)
| None -> Some y
)
)
Bonus: Adding Functions to Modules
It's sad sometimes that F# has so few functions on Map. But you need the a lot, you always can add a union function youself to the Module. For example:
module Map =
let union f map1 map2 =
let keys = Set (Map.keys map1) + Set (Map.keys map2)
Map [
for key in keys do
match Map.tryFind key map1, Map.tryFind key map2 with
| Some x, Some y -> key, (f x y)
| Some x, None -> key, x
| None , Some y -> key, y
| None , None -> failwith "Cannot happen"
]
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
This way you get a Map.union and you can specify a lambda-function that is executed if both keys are present in both maps, otherwise the value is used unchanged.
There have been a couple of useful suggestions:
Group by keys with standard library functions from the Seq module, by user1981
Use a specialized library for dealing with data series, by Tomas Petricek
Use a map instead (a functional data structure based on comparison), by David Raab
To this I'd like to add
An imperative way, filling a combined dictionary by iterating through the keys of the source data structures, and finally
A query expression
An imperative way
The average calculation is hard-coded with the type int. You can still have generic keys, as their type does not figure in the function, except for the equality constraint required for dictionary keys. You could make the function generic for values too, by marking it inline, but that won't be a pretty sight as it will introduce a host of other constraints onto the type of values.
open System.Collections.Generic
let unionAverage (d1 : IDictionary<_,_>) (d2 : IDictionary<_,_>) =
let d = Dictionary<_,_>()
for k in Seq.append d1.Keys d2.Keys |> Seq.distinct do
match d1.TryGetValue k, d2.TryGetValue k with
| (true, v1), (true, v2) -> d.Add(k, (v1 + v2) / 2)
| (true, v), _ | _, (true, v) -> d.Add(k, v)
| _ -> failwith "Key not found"
d
let d1 = dict[1, 100; 2, 50; 3, 150]
let d2 = dict[1, 20; 2, 30; 3, 0; 4, 10]
unionAverage d1 d2
A query expression
It operates on the same principle as the answer from user1981, but for re-usability the average function has been factored out. It expects an arbitrary number of #seq<KeyValuePair<_,_>> elements, which is just another way to represent dictionaries that are accessed through their enumerators.
As the query expression uses System.Linq.IGrouping under the hood, this is upcast to a regular sequence to reduce confusion. Then there's the conversion to float for Seq.average to operate on, because the type int does not have the required member DivideByInt.
module Dict =
let unionByMany f src =
query{
for KeyValue(k, v) in Seq.concat src do
groupValBy v k into group
select (group.Key, f (group :> seq<_>)) }
|> dict
Dict.unionByMany (Seq.averageBy float >> int) [d1; d2]
Dict.unionByMany Seq.sum [d1; d2]
Dict.unionByMany Seq.min [d1; d2]
How does one get the first key,value pair from F# Map without knowing the key?
I know that the Map type is used to get a corresponding value given a key, e.g. find.
I also know that one can convert the map to a list and use List.Head, e.g.
List.head (Map.toList map)
I would like to do this
1. without a key
2. without knowing the types of the key and value
3. without using a mutable
4. without iterating through the entire map
5. without doing a conversion that iterates through the entire map behind the seen, e.g. Map.toList, etc.
I am also aware that if one gets the first key,value pair it might not be of use because the map documentation does not note if using map in two different calls guarantees the same order.
If the code can not be written then an existing reference from a site such as MSDN explaining and showing why not would be accepted.
TLDR;
How I arrived at this problem was converting this function:
let findmin l =
List.foldBack
(fun (_,pr1 as p1) (_,pr2 as p2) -> if pr1 <= pr2 then p1 else p2)
(List.tail l) (List.head l)
which is based on list and is used to find the minimum value in the associative list of string * int.
An example list:
["+",10; "-",10; "*",20; "/",20]
The list is used for parsing binary operator expressions that have precedence where the string is the binary operator and the int is the precedence. Other functions are preformed on the data such that using F# map might be an advantage over list. I have not decided on a final solution but wanted to explore this problem with map while it was still in the forefront.
Currently I am using:
let findmin m =
if Map.isEmpty m then
None
else
let result =
Map.foldBack
(fun key value (k,v) ->
if value <= v then (key,value)
else (k,v))
m ("",1000)
Some(result)
but here I had to hard code in the initial state ("",1000) when what would be better is just using the first value in the map as the initial state and then passing the remainder of the map as the starting map as was done with the list:
(List.tail l) (List.head l)
Yes this is partitioning the map but that did not work e.g.,
let infixes = ["+",10; "-",10; "*",20; "/",20]
let infixMap = infixes |> Map.ofList
let mutable test = true
let fx k v : bool =
if test then
printfn "first"
test <- false
true
else
printfn "rest"
false
let (first,rest) = Map.partition fx infixMap
which results in
val rest : Map<string,int> = map [("*", 20); ("+", 10); ("-", 10)]
val first : Map<string,int> = map [("/", 20)]
which are two maps and not a key,value pair for first
("/",20)
Notes about answers
For practical purposes with regards to the precedence parsing seeing the + operations before - in the final transformation is preferable so returning + before - is desirable. Thus this variation of the answer by marklam
let findmin (map : Map<_,_>) = map |> Seq.minBy (fun kvp -> kvp.Value)
achieves this and does this variation by Tomas
let findmin m =
Map.foldBack (fun k2 v2 st ->
match st with
| Some(k1, v1) when v1 < v2 -> st
| _ -> Some(k2, v2)) m None
The use of Seq.head does return the first item in the map but one must be aware that the map is constructed with the keys sorted so while for my practical example I would like to start with the lowest value being 10 and since the items are sorted by key the first one returned is ("*",20) with * being the first key because the keys are strings and sorted by such.
For me to practically use the answer by marklam I had to check for an empty list before calling and massage the output from a KeyValuePair into a tuple using let (a,b) = kvp.Key,kvp.Value
I don't think there is an answer that fully satisfies all your requirements, but:
You can just access the first key-value pair using m |> Seq.head. This is lazy unlike converting the map to list. This does not guarantee that you always get the same first element, but realistically, the implementation will guarantee that (it might change in the next version though).
For finding the minimum, you do not actually need the guarantee that Seq.head returns the same element always. It just needs to give you some element.
You can use other Seq-based functons as #marklam mentioned in his answer.
You can also use fold with state of type option<'K * 'V>, which you can initialize with None and then you do not have to worry about finding the first element:
m |> Map.fold (fun st k2 v2 ->
match st with
| Some(k1, v1) when v1 < v2 -> st
| _ -> Some(k2, v2)) None
Map implements IEnumerable<KeyValuePair<_,_>> so you can treat it as a Seq, like:
let findmin (map : Map<_,_>) = map |> Seq.minBy (fun kvp -> kvp.Key)
It's even simpler than the other answers. Map internally uses an AVL balanced tree so the entries are already ordered by key. As mentioned by #marklam Map implements IEnumerable<KeyValuePair<_,_>> so:
let m = Map.empty.Add("Y", 2).Add("X", 1)
let (key, value) = m |> Seq.head
// will return ("X", 1)
It doesn't matter what order the elements were added to the map, Seq.head can operate on the map directly and return the key/value mapping for the min key.
Sometimes it's required to explicitly convert Map to Seq:
let m = Map.empty.Add("Y", 2).Add("X", 1)
let (key, value) = m |> Map.toSeq |> Seq.head
The error message I've seen for this case says "the type 'a * 'b does not match the type Collections.Generic.KeyValuePair<string, int>". It may also be possible add type annotations rather than Map.toSeq.
I'm trying to write some code in a functional paradigm for practice. There is one case I'm having some problems wrapping my head around. I am trying to create an array of 5 unique integers from 1, 100. I have been able to solve this without using functional programming:
let uniqueArray = [];
while (uniqueArray.length< 5) {
const newNumber = getRandom1to100();
if (uniqueArray.indexOf(newNumber) < 0) {
uniqueArray.push(newNumber)
}
}
I have access to lodash so I can use that. I was thinking along the lines of:
const uniqueArray = [
getRandom1to100(),
getRandom1to100(),
getRandom1to100(),
getRandom1to100(),
getRandom1to100()
].map((currentVal, index, array) => {
return array.indexOf(currentVal) > -1 ? getRandom1to100 : currentVal;
});
But this obviously wouldn't work because it will always return true because the index is going to be in the array (with more work I could remove that defect) but more importantly it doesn't check for a second time that all values are unique. However, I'm not quite sure how to functionaly mimic a while loop.
Here's an example in OCaml, the key point is that you use accumulators and recursion.
let make () =
Random.self_init ();
let rec make_list prev current max accum =
let number = Random.int 100 in
if current = max then accum
else begin
if number <> prev
then (number + prev) :: make_list number (current + 1) max accum
else accum
end
in
make_list 0 0 5 [] |> Array.of_list
This won't guarantee that the array will be unique, since its only checking by the previous. You could fix that by hiding a hashtable in the closure between make and make_list and doing a constant time lookup.
Here is a stream-based Python approach.
Python's version of a lazy stream is a generator. They can be produced in various ways, including by something which looks like a function definition but uses the key word yield rather than return. For example:
import random
def randNums(a,b):
while True:
yield random.randint(a,b)
Normally generators are used in for-loops but this last generator has an infinite loop hence would hang if you try to iterate over it. Instead, you can use the built-in function next() to get the next item in the string. It is convenient to write a function which works something like Haskell's take:
def take(n,stream):
items = []
for i in range(n):
try:
items.append(next(stream))
except StopIteration:
return items
return items
In Python StopIteration is raised when a generator is exhausted. If this happens before n items, this code just returns however much has been generated, so perhaps I should call it takeAtMost. If you ditch the error-handling then it will crash if there are not enough items -- which maybe you want. In any event, this is used like:
>>> s = randNums(1,10)
>>> take(5,s)
[6, 6, 8, 7, 2]
of course, this allows for repeats.
To make things unique (and to do so in a functional way) we can write a function which takes a stream as input and returns a stream consisting of unique items as output:
def unique(stream):
def f(s):
items = set()
while True:
try:
x = next(s)
if not x in items:
items.add(x)
yield x
except StopIteration:
raise StopIteration
return f(stream)
this creates an stream in a closure that contains a set which can keep track of items that have been seen, only yielding items which are unique. Here I am passing on any StopIteration exception. If the underlying generator has no more elements then there are no more unique elements. I am not 100% sure if I need to explicitly pass on the exception -- (it might happen automatically) but it seems clean to do so.
Used like this:
>>> take(5,unique(randNums(1,10)))
[7, 2, 5, 1, 6]
take(10,unique(randNums(1,10))) will yield a random permutation of 1-10. take(11,unique(randNums(1,10))) will never terminate.
This is a very good question. It's actually quite common. It's even sometimes asked as an interview question.
Here's my solution to generating 5 integers from 0 to 100.
let rec take lst n =
if n = 0 then []
else
match lst with
| [] -> []
| x :: xs -> x :: take xs (n-1)
let shuffle d =
let nd = List.map (fun c -> (Random.bits (), c)) d in
let sond = List.sort compare nd in
List.map snd sond
let rec range a b =
if a >= b then []
else a :: range (a+1) b;;
let _ =
print_endline
(String.concat "\t" ("5 random integers:" :: List.map string_of_int (take (shuffle (range 0 101)) 5)))
How's this:
const addUnique = (ar) => {
const el = getRandom1to100();
return ar.includes(el) ? ar : ar.concat([el])
}
const uniqueArray = (numberOfElements, baseArray) => {
if (numberOfElements < baseArray.length) throw 'invalid input'
return baseArray.length === numberOfElements ? baseArray : uniqueArray(numberOfElements, addUnique(baseArray))
}
const myArray = uniqueArray(5, [])
I'm currently generating a sequence in a similar way to:
migrators
|> Seq.map (fun m -> m())
The migrator function is ultimately returning a discriminated union like:
type MigratorResult =
| Success of string * TimeSpan
| Error of string * Exception
I want to stop the map once I encounter my first Error but I need to include the Error in the final sequence.
I have something like the following to display a final message to the user
match results |> List.rev with
| [] -> "No results equals no migrators"
| head :: _ ->
match head with
| Success (dt, t) -> "All migrators succeeded"
| Error (dt, ex) -> "Migration halted owing to error"
So I need:
A way to stop the mapping when one of the map steps produces an Error
A way to have that error be the final element added to the sequence
I appreciate there may be a different sequence method other than map that will do this, I'm new to F# and searching online hasn't yielded anything as yet!
I guess there are multiple approaches here, but one way would be to use unfold:
migrators
|> Seq.unfold (fun ms ->
match ms with
| m :: tl ->
match m () with
| Success res -> Some (Success res, tl)
| Error res -> Some (Error res, [])
| [] -> None)
|> List.ofSeq
Note the List.ofSeq at the end, that's just there for realizing the sequence. A different way to go would be to use sequence comprehensions, some might say it results in a clearer code.
The ugly things Tomaš alludes to are 1) mutable state, and 2) manipulation of the underlying enumerator. A higher-order function which returns up to and including when the predicate holds would then look like this:
module Seq =
let takeUntil pred (xs : _ seq) = seq{
use en = xs.GetEnumerator()
let flag = ref true
while !flag && en.MoveNext() do
flag := not <| pred en.Current
yield en.Current }
seq{1..10} |> Seq.takeUntil (fun x -> x % 5 = 0)
|> Seq.toList
// val it : int list = [1; 2; 3; 4; 5]
For your specific application, you'd map the cases of the DU to a boolean.
(migrators : seq<MigratorResult>)
|> Seq.takeUntil (function Success _ -> false | Error _ -> true)
I think the answer from #scrwtp is probably the nicest way to do this if your input is reasonably small (and you can turn it into an F# list to use pattern matching). I'll add one more version, which works when your input is just a sequence and you do not want to turn it into a list.
Essentially, you want to do something that's almost like Seq.takeWhile, but it gives you one additional item at the end (the one, for which the predicate fails).
To use a simpler example, the following returns all numbers from a sequence until one that is divisible by 5:
let nums = [ 2 .. 10 ]
nums
|> Seq.map (fun m -> m % 5)
|> Seq.takeWhile (fun n -> n <> 0)
So, you basically just need to look one element ahead - to do this, you could use Seq.pairwise which gives you the current and the next element in the sequence"
nums
|> Seq.map (fun m -> m % 5)
|> Seq.pairwise // Get sequence of pairs with the next value
|> Seq.takeWhile (fun (p, n) -> p <> 0) // Look at the next value for test
|> Seq.mapi (fun i (p, n) -> // For the first item, we return both
if i = 0 then [p;n] else [n]) // for all other, we return the second
|> Seq.concat
The only ugly thing here is that you then need to flatten the sequence again using mapi and concat.
This is not very nice, so a good thing to do would be to define your own higher-order function like Seq.takeUntilAfter that encapsulates the behavior you need (and hides all the ugly things). Then your code could just use the function and look nice & readable (and you can experiment with other ways of implementing this).
I recently started with F# and implemented a very basic recursive function that represents the Sieve of Eratosthenes. I came up with the following, working code:
static member internal SieveOfEratosthenesRecursive sequence accumulator =
match sequence with
| [] -> accumulator
| head::tail -> let rest = tail |> List.filter(fun number -> number % head <> 0L)
let newAccumulator = head::accumulator
Prime.SieveOfEratosthenesRecursive rest newAccumulator
This function is not really memory efficient so I tried to eliminate the variables "rest" and "newAccumulator". I came up with the following code
static member internal SieveOfEratosthenesRecursive sequence accumulator =
match sequence with
| [] -> accumulator
| head::tail -> tail |> List.filter(fun number -> number % head <> 0L)
|> Prime.SieveOfEratosthenesRecursive (head::accumulator)
As far as I understand the tutorials I've read Prime.SieveOfEratosthenesRecursive will be called with the filtered tail as first parameter and a list consisting of head::accumulator as second one. However when I try to run the code with the reduced variable usage, the program gets trappen in an infinite loop. Why is this happening and what did I do wrong?
As far as I understand the tutorials I've read Prime.SieveOfEratosthenesRecursive will be called with the filtered tail as first parameter and a list consisting of head::accumulator as second one.
You have this backwards.
In the first version, you're passing rest then newAccumulator; in the second version, you're effectively passing newAccumulator then rest. I.e., you've transposed the arguments.
Prime.SieveOfEratosthenesRecursive (head::accumulator) is a partial function application wherein you're applying (head::accumulator) as the first argument (sequence). This partial function application yields a unary function (expecting accumulator), to which you are passing (via |>) what is called rest in the first version of your code.
Changing SieveOfEratosthenesRecursive's argument order is the easiest solution, but I would consider something like the following idiomatic as well:
static member internal SieveOfEratosthenesRecursive sequence accumulator =
match sequence with
| [] -> accumulator
| head::tail ->
tail
|> List.filter(fun number -> number % head <> 0L)
|> Prime.SieveOfEratosthenesRecursive <| (head::accumulator)
or
static member internal SieveOfEratosthenesRecursive sequence accumulator =
let inline flipzip a b = b, a
match sequence with
| [] -> accumulator
| head::tail ->
tail
|> List.filter(fun number -> number % head <> 0L)
|> flipzip (head::accumulator)
||> Prime.SieveOfEratosthenesRecursive
FWIW, eliminating rest and newAccumulator as named variables here is not going to impact your memory usage in the slightest.
The last call in your second function is equivalent to:
Prime.SieveOfEratosthenesRecursive newAccumulator rest
where you switch positions of two params. Since newAccumulator grows bigger after each recursive call, you will never reach the base case of empty list.
The rule of thumb is putting the most frequently changing parameter at last:
let rec sieve acc xs =
match xs with
| [] -> acc
| x::xs' -> xs' |> List.filter (fun y -> y % x <> 0L)
|> sieve (x::acc)
The above function could be shortened using function keyword:
let rec sieve acc = function
| [] -> acc
| x::xs' -> xs' |> List.filter (fun y -> y % x <> 0L)
|> sieve (x::acc)
Using pipe (|>) operator only makes the function more readable, it doesn't affect memory usage at all.