I want to be able to store a map using :dets
Currently, that is the solution I am trying to implement:
# a list of strings
topics = GenServer.call(MessageBroker.TopicsProvider, {:get_topics})
# a map with each element of the list as key and an empty list as value
topics_map =
topics
|> Enum.chunk_every(1)
|> Map.new(fn [k] -> {k, []} end)
{:ok, table} = :dets.open_file(:messages, type: :set)
# trying to store the map
:dets.insert(table, [topics_map])
:dets.close(table)
However, I get
** (EXIT) an exception was raised:
** (ArgumentError) argument error
(stdlib 3.12) dets.erl:1259: :dets.insert(:messages, [%{"tweet" => [], "user" => []}])
How is it possible to accomplish this?
I have tested by erlang. You should convert the map to list first.
Following from dets:insert_new() doc
insert_new(Name, Objects) -> boolean() | {error, Reason}
Types
Name = tab_name()
Objects = object() | [object()]
Reason = term()
Inserts one or more objects into table Name. If there already exists some object with a key matching the key of any of the specified objects, the table is not updated and false is returned. Otherwise the objects are inserted and true returned.
test code
dets:open_file(dets_a,[{file,"/tmp/aab"}]).
Map = #{a => 2, b => 3, c=> 4, "a" => 1, "b" => 2, "c" => 4}.
List_a = maps:to_list(Map). %% <----- this line
dets:insert(dets_a,List_a).
Chen Yu's solution is good, but before getting it I already found another solution.
Basically, you can just add the map to a tuple
:dets.insert(table, {:map, topics_map})
Then, you can get this map by using
:dets.lookup(table, :map)
As I understood your intent, you want to store users and tweets under separate keys. For that, you need to construct a keyword list, not a map, in the first place.
topics = for topic <- topics, do: {topic, []}
# or topics = Enum.map(topics, &{&1, []})
# or topics = Enum.map(topics, fn topic -> {topic, []} end)
then you might use this keyword list to create dets.
{:ok, table} = :dets.open_file(:messages, type: :set)
:dets.insert(table, topics)
:dets.close(table)
I am new to functional programming and so can not imagen how to build the new dictionary based on two other dictionaries with similar set of keys. The new dictionary will have the entries with all keys but values will be selected/computed based on some condition.
For example, having two dictionaries:
D1: [(1,100);(2,50);(3,150)]
D2: [(1,20);(2,30);(3,0);(4,10)]
and condition to get the average of two values, the resulting dictionary will be
DR: [(1,60);(2,40);(3,75);(4,10)]
I need implementation in F#.
Please could you give me some advise.
View them as two (or more...) lists of tuples that we concat makes it easier. The below solves your specfic problem. To generalise the process aggeragting a list of values to something specific you would need to change averageBy to fold and provide a fold function instead of float. Assuming d1 and d2 mataches your exmaple.
Seq.concat [ d1 ; d2 ]
|> Seq.map (|KeyValue|)
|> Seq.groupBy fst
|> Seq.map (fun (k, c) -> k, Seq.averageBy (snd >> float) c |> int)
|> dict
If you wanted to use an external library, you could do this using Deedle series, which has various operations for working with (time) series of data.
Here, you have two data series that have different keys. Deedle lets you zip series based on keys and handle the cases where one of the values is missing using the opt type:
#r "nuget:Deedle"
open Deedle
let s1 = series [(1,100);(2,50);(3,150)]
let s2 = series [(1,20);(2,30);(3,0);(4,10)]
Series.zip s1 s2
|> Series.mapValues (fun (v1, v2) ->
( (OptionalValue.defaultArg 0 v1) +
(OptionalValue.defaultArg 0 v2) ) / 2)
This may not make sense if this is a thing that you need just in one or two places, but if you're working with key-value series of data more generally, it may be worth checking out.
Solution 1
From a functional perspective I would use a Map data-structure, instead of a dictionary. You can convert a dictionary to a Map like this
let d1 = dict [(1,100);(2,50);(3,150)]
let m1 = Map [for KeyValue (key,value) in d1 -> key, value]
But i wouldn't use a Dictionary and convert it, I would use a Map diretly.
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
Next, you need a way to get all keys from both Maps. You can get the keys of a map with Map.keys but you need all the keys from both. You could get them by using a Set.
let keys = Set (Map.keys m1) + Set (Map.keys m2)
By adding two Sets you get a Set.union of both sets. Once you have them, you can traverse the keys, and try to get both values from both keys. If you use Map.find then you get an optional. You can Pattern match on both cases at once.
let result = Map [
for key in keys do
match Map.tryFind key m1, Map.tryFind key m2 with
| Some x, Some y -> key, (x + y) / 2
| Some x, None -> key, x
| None , Some y -> key, y
| None , None -> failwith "Cannot happen"
]
This creates a new Map data-structure and saves it into result. If both cases are Some then you compute the average, otherwise you just keep the value. As you iterate the keys of both Maps the None,None case cannot happen. A Key always must be in either one or the other.
After all of this, result will be:
Map [(1, 60); (2, 40); (3, 75); (4, 10)]
Again, here is the whole code at once:
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
let keys = Set (Map.keys m1) + Set (Map.keys m2)
let result = Map [
for key in keys do
match Map.tryFind key m1, Map.tryFind key m2 with
| Some x, Some y -> key, (x + y) / 2
| Some x, None -> key, x
| None , Some y -> key, y
| None , None -> failwith "Cannot happen"
]
You also can inline the keys variable, if you want.
Solution 2
When you have a Map then you can make use of the fact that adding a value always to a map, always creates a new Map data-structure. This way you are able to use Map.fold that traverses a Map data-structure and uses one of the map as the starting state while you traverse the other Map.
With Map.change you then can read and change a value in one step. If a key is already available you calculate the average, otherwise just add the value.
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
let result =
(m1,m2) ||> Map.fold (fun state key y ->
state |> Map.change key (function
| Some x -> Some ((x + y) / 2)
| None -> Some y
)
)
Bonus: Adding Functions to Modules
It's sad sometimes that F# has so few functions on Map. But you need the a lot, you always can add a union function youself to the Module. For example:
module Map =
let union f map1 map2 =
let keys = Set (Map.keys map1) + Set (Map.keys map2)
Map [
for key in keys do
match Map.tryFind key map1, Map.tryFind key map2 with
| Some x, Some y -> key, (f x y)
| Some x, None -> key, x
| None , Some y -> key, y
| None , None -> failwith "Cannot happen"
]
let m1 = Map [(1,100);(2,50);(3,150)]
let m2 = Map [(1,20);(2,30);(3,0);(4,10)]
This way you get a Map.union and you can specify a lambda-function that is executed if both keys are present in both maps, otherwise the value is used unchanged.
There have been a couple of useful suggestions:
Group by keys with standard library functions from the Seq module, by user1981
Use a specialized library for dealing with data series, by Tomas Petricek
Use a map instead (a functional data structure based on comparison), by David Raab
To this I'd like to add
An imperative way, filling a combined dictionary by iterating through the keys of the source data structures, and finally
A query expression
An imperative way
The average calculation is hard-coded with the type int. You can still have generic keys, as their type does not figure in the function, except for the equality constraint required for dictionary keys. You could make the function generic for values too, by marking it inline, but that won't be a pretty sight as it will introduce a host of other constraints onto the type of values.
open System.Collections.Generic
let unionAverage (d1 : IDictionary<_,_>) (d2 : IDictionary<_,_>) =
let d = Dictionary<_,_>()
for k in Seq.append d1.Keys d2.Keys |> Seq.distinct do
match d1.TryGetValue k, d2.TryGetValue k with
| (true, v1), (true, v2) -> d.Add(k, (v1 + v2) / 2)
| (true, v), _ | _, (true, v) -> d.Add(k, v)
| _ -> failwith "Key not found"
d
let d1 = dict[1, 100; 2, 50; 3, 150]
let d2 = dict[1, 20; 2, 30; 3, 0; 4, 10]
unionAverage d1 d2
A query expression
It operates on the same principle as the answer from user1981, but for re-usability the average function has been factored out. It expects an arbitrary number of #seq<KeyValuePair<_,_>> elements, which is just another way to represent dictionaries that are accessed through their enumerators.
As the query expression uses System.Linq.IGrouping under the hood, this is upcast to a regular sequence to reduce confusion. Then there's the conversion to float for Seq.average to operate on, because the type int does not have the required member DivideByInt.
module Dict =
let unionByMany f src =
query{
for KeyValue(k, v) in Seq.concat src do
groupValBy v k into group
select (group.Key, f (group :> seq<_>)) }
|> dict
Dict.unionByMany (Seq.averageBy float >> int) [d1; d2]
Dict.unionByMany Seq.sum [d1; d2]
Dict.unionByMany Seq.min [d1; d2]
I have a relation that has a list of ids s_ids as a property of the relation. each id in the list correspond to another node that has a sentence corresponding to an id.I used:
MATCH (c: term)-[r: semrel]->(t: term), (b: Sentence)
Where r.source = "xyz" And b.sentence_id IN r.s_id
return r,b
to return all sentences corresponding to the relation,
the result looks like :
r b
w abc
w rty
w zxv
e nmx
e qrt
the relation r is repeated for every sentence how can I group the list of sentences corresponding to each relation to get
r b
w abc, rty, zxv
e nmx,qrt
Thanks
This should return each r and its collection of sentences:
MATCH (c: term)-[r: semrel]->(t: term), (b: Sentence)
WHERE r.source = "xyz" AND b.sentence_id IN r.s_i
RETURN r, COLLECT(b) AS sentences;
For better performance, if you create an index on :Sentence(sentence_id), like this:
CREATE INDEX ON :Sentence(sentence_id);
then this query (which adds a hint to use the index) should be faster (as the b nodes can be found using the index):
MATCH (c: term)-[r: semrel]->(t: term), (b: Sentence)
USING INDEX b:Sentence(sentence_id)
WHERE r.source = "xyz" AND b.sentence_id IN r.s_i
RETURN r, COLLECT(b) AS sentences;
Consider following nodes that are connected between each other with 2 type of edges: direct and intersect. The query needs to discover all possible paths between 2 nodes that satisfies all following rules:
0..N direct edges
0..1 intersect edge
intersect edge can be between direct edges
These paths are considered valid between nodeA and nodeZ:
(nodeA)-[:direct]->(nodeB)-[:direct]->(nodeC)->[:direct]->(nodeZ)
(nodeA)-[:intersect]->(nodeB)-[:direct]->(nodeC)->[:direct]->(nodeZ)
(nodeA)-[:direct]->(nodeB)-[:intersect]->(nodeC)->[:direct]->(nodeZ)
(nodeA)-[:direct]->(nodeB)->[:direct]->(nodeC)-[:intersect]->(nodeZ)
Basically intersect edge can happen anywhere in the path but only once.
My ideal cypher query in non-existing neo4j version would be this:
MATCH (from)-[:direct*0..N|:intersect*0..1]->(to)
But neo4j doesn't support multiple constraints for edges type :(.
UPDATE 23.04.16
There 6609 nodes (out of 550k total), 5184 edges of type direct (out of 440k total) and 34119 of type intersect (out of 37289 total). There are some circular references expected (which neo4j avoids, isn't it?)
The query that looked promising but failed to finish in a manner of seconds:
MATCH p = (from {from: 1})-[:direct|intersect*0..]->(to {to: 99})
WHERE
123 < from.departureTS < 123 + 86400 //next day
AND REDUCE(s = 0, x IN RELATIONSHIPS(p) | CASE TYPE(x) WHEN 'intersect' THEN s + 1 ELSE s END) <= 1
return p;
Here is a query that conforms to the stated requirements:
MATCH p = (from)-[:direct|intersect*0..]->(to)
WHERE REDUCE(s = 0, x IN RELATIONSHIPS(p) |
CASE WHEN TYPE(x) = 'intersect' THEN s + 1 ELSE s END) <= 1
return p;
It returns all paths with 0 or more direct relationships and 0 or 1 intersect relationships.
This will do what you want:
// Cybersam's correction:
MATCH p = ((from)-[:direct*0..]->(middle)-[:intersect*0..1]->(middle2)-[:direct*0..]->(to)) return DISTINCT p;
return p
Here's the test scenario I used:
create (a:nodeA {name: "A"})
create (b:nodeB {name: "B"})
create (c:nodeC {name: "C"})
create (z:nodeZ {name: "Z"})
merge (a)-[:direct {name: "D11"}]->(b)-[:direct {name: "D21"}]->(c)-[:direct {name: "D31"}]->(z)
merge (a)-[:intersect {name: "I12"}]->(b)-[:direct {name: "D22"}]->(c)-[:direct {name: "D32"}]->(z)
merge (a)-[:direct {name: "D13"}]->(b)-[:intersect {name: "I23"}]->(c)-[:direct {name: "D33"}]->(z)
merge (a)-[:direct {name: "D14"}]->(b)-[:direct {name: "D24"}]->(c)-[:intersect {name: "I34"}]->(z)
merge (a)-[:intersect {name: "I15"}]->(z)
// Cybersam's correction:
MATCH p = ((from)-[:direct*0..]->(middle)-[:intersect*0..1]->(middle2)-[:direct*0..]->(to)) return DISTINCT p;
return p
I made the mistake of thinking the graph on the browser reflected the data that was returned in "p" - it did not, you have to look at the "rows" part of the report to get all the details.
This query will also return single nodes- which fits the requirements.
Let say I have this two-dimensional array:
let a = Array2D.create 2 2 "*"
What is an idiomatic way to turn that into the following string?
**\n
**\n
My thought would be that I need to iterate over the rows and then map string.concat over the items in each row. However I can't seem to figure out how to iterate just the rows.
I think you'll have to iterate over the rows by hand (Array2D does not have any handy function for this),
but you can get a row using splicing syntax. To get the row at index row, you can write array.[row, *]:
let a = Array2D.create 3 2 "*"
[ for row in 0 .. a.GetLength(0)-1 ->
String.concat "" a.[row,*] ]
|> String.concat "\n"
This creates a list of rows (each turned into a string using the first String.concat) and then concatenates the rows using the second String.concat.
Alternatively, you may use StringBuilder and involve Array2D.iteri function:
let data' = [|[|"a"; "b"; "c"|]; [|"d"; "e"; "f"|]|]
let data = Array2D.init 2 3 (fun i j -> data'.[i].[j])
open System.Text
let concatenateArray2D (data:string[,]) =
let sb = new StringBuilder()
data
|> Array2D.iteri (fun row col value ->
(if col=0 && row<>0 then sb.Append "\n" else sb)
.Append value |> ignore
)
sb.ToString()
data |> concatenateArray2D |> printfn "%s"
This prints:
abc
def