Spring Neo4j #Query not populating related entities until a subsequent .findOne called - graph

I have the following Cypher which is working correctly in the Neo4j browser and returning all related entities as expected:
MATCH (cmp:Competition)-[:COMPETITION_COUNTRY]-(cc:Country)
WHERE ID(cmp)=16860
MATCH (cmp)-[:COMPETITION]-(s:Season)<-[:SEASON]-(f:Fixture)
WHERE s.yearStart=2019
MATCH (f)-[:HOME_TEAM]-(ht:Team)-[:TEAM_COUNTRY]-(htc:Country)
MATCH (f)-[:HOME_TEAM]-(at:Team)-[:TEAM_COUNTRY]-(atc:Country)
RETURN cmp, s, f, ht, at, htc, atc
ORDER BY f.matchDate DESC, ht.name DESC
And I have this as a Neo4jRepository function as follows:
#Query("MATCH (cmp:Competition)-[:COMPETITION_COUNTRY]-(cc:Country)\n" +
"WHERE ID(cmp)={0}\n" +
"\n" +
"MATCH (cmp)-[:COMPETITION]-(s:Season)<-[:SEASON]-(f:Fixture)\n" +
"WHERE s.yearStart={1}\n" +
"\n" +
"MATCH (f)-[:HOME_TEAM]-(ht:Team)-[:TEAM_COUNTRY]-(htc:Country)\n" +
"MATCH (f)-[:HOME_TEAM]-(at:Team)-[:TEAM_COUNTRY]-(atc:Country)\n" +
"\n" +
"RETURN cmp, s, f, ht, at, htc, atc\n" +
"ORDER BY f.matchDate DESC, ht.name DESC"
)
List<Fixture> getCompetitionYearFixtures(
Long competitionId, Integer yearStart
);
I call this method as follows:
List<Fixture> fixtures =
fixtureRepository
.getCompetitionYearFixtures(
competitionId, year);
Although all the expected Fixtures are returned, none of the related entities are populated:
However, I discovered that if I immediately run the following statement for any one (and only one) of the returned Fxtures, then every Fixture in fixtures is suddenly fully populated:
fixtureRepository.findOne(fixtures.get(0).getId(), 3);
As so:
So my question is, is there a way of returning the Fixtures with all related entities populated after the first trip to the database, without having to go back?
I specifically retrieved everything I needed in the Cypher, and the idea of using a depth of 3 in the findOne is something I'm a little uncomfortable with as in future I may add new relations which I won't necessarily want bloating the query.
EDIT
Solution, with thanks to František Hartman:
MATCH r1=(cmp:Competition)-[cmp_c:COMPETITION_COUNTRY]-(cmpc:Country)
WHERE ID(cmp)=16860
MATCH r2=(cmp)-[cmp_s:COMPETITION]-(s:Season)<-[s_f:SEASON]-(f:Fixture)
WHERE s.yearStart=2019
MATCH r3=(f)-[f_ht:HOME_TEAM]-(ht:Team)-[ht_c:TEAM_COUNTRY]-(htc:Country)
MATCH r4=(f)-[f_at:AWAY_TEAM]-(at:Team)-[at_c:TEAM_COUNTRY]-(atc:Country)
RETURN r1,r2,r3,r4
ORDER BY f.matchDate DESC,ht.name DESC

You need to return everything that you want to map, in your case you are expecting the relationships to be mapped as well (even if you don't have relationship entities, the relationship says e.g. the fixture and home team are related).
To see what is exactly returned in your Neo4j browser go to "Browser settings" (cog icon left bottom) -> Uncheck "Connect result nodes"
You query with relationships (you might be able to come up with better names then r1-r7):
MATCH (cmp:Competition)-[r1:COMPETITION_COUNTRY]-(cc:Country)
WHERE ID(cmp)=16860
MATCH (cmp)-[r2:COMPETITION]-(s:Season)<-[r3:SEASON]-(f:Fixture)
WHERE s.yearStart=2019
MATCH (f)-[r4:HOME_TEAM]-(ht:Team)-[r5:TEAM_COUNTRY]-(htc:Country)
MATCH (f)-[r6:HOME_TEAM]-(at:Team)-[r7:TEAM_COUNTRY]-(atc:Country)
RETURN cmp, s, f, ht, at, htc, atc, r1, r2, r3, r4, r5, r6, r7
ORDER BY f.matchDate DESC, ht.name DESC
The findOne method (with depth parameter 3) loads everything, including the relationships up to 3 steps, which will be populated into cached instances of your data.

Related

How to export edgelist from Grakn without using client APIs

I'm trying to export edges from grakn. I can do that with Python client like so:
edge_query = "match $c2c($c1, $c2) isa c2c; $c1 has id $id1; $c2 has id $id2;get $id1,$id2;"
with open(f"grakn.edgelist","w") as outfile:
with GraknClient(uri="localhost:48555") as client:
with client.session(keyspace=KEYSPACE) as session:
with session.transaction().read() as read_transaction:
answer_iterator = read_transaction.query(edge_query)
for answer in tqdm(answer_iterator):
id1 = answer.get("id1")
id2 = answer.get("id2")
outfile.write(f"{id1.value()} {id2.value()} \n")
Edit: For each Relation, I want to export entities pairwise. The output can be a pair of Grakn IDs.
I can ignore the attributes of relation or entities.
Exporting to edges seems like a common task. Is there a better way(more elegant, faster, more efficient) to do it in Grakn?
This works as long as the relation type c2c always has two roleplayers. However, this will produce two edges for every $c1, $c2, which is probably not what you want.
Let's take a pair of Things, with ids V123 and V456. If they satisfy $c2c($c1, $c2) isa c2c; with $c1 = V123 and $c2 = V456 then they will also satisfy the same pattern as $c1 = V456 and $c2 = V123. Grakn will return all combinations of $c1, $c2 that satisfy your query, so you'll get two answers back for this one c2c relation.
Assuming this isn't what you want, if $c1 and $c2 play different roles in the relation c2c (likely implying there is direction to the edge) then try changing the query, adding the roles, to:
edge_query = "match $c2c(role1: $c1, role2: $c2) isa c2c; $c1 has id $id1; $c2 has id $id2; get $id1,$id2;"
If they both play the same role (implying undirected edges), then we need to do something different in our logic. Either store edges as a set of sets of ids to remove duplicates without much effort, or perhaps consider using the Python ConceptAPI, something like this:
relation_query = "match $rc2c isa c2c;get;"
with open(f"grakn.edgelist","w") as outfile:
with GraknClient(uri="localhost:48555") as client:
with client.session(keyspace=KEYSPACE) as session:
with session.transaction().read() as read_transaction:
answer_iterator = read_transaction.query(relation_query)
for answer in answer_iterator:
relation_concept = answer.get("rc2c")
role_players_map = relation_concept.role_players_map()
role_player_ids = set()
for role, thing in role_players_map.items():
# Here you can do any logic regarding what things play which roles
for t in thing:
role_player_ids.add(t.id) # Note that you can retrieve a concept id from the concept object, you don't need to ask for it in the query
outfile.write(", ".join(role_player_ids) + "\n")
Of course, I have no idea what you're doing with the resulting edgelist, but for completeness, the more Grakn-esque way would be to treat the Relation as a first-class citizen since it represents a hyperedge in the Grakn knowledge model, in this case we would treat the Roles of the relation as edges. This means we aren't stuck when we have ternary or N-ary relations. We can do this by changing the query:
match $c2c($c) isa c2c; get;
Then in the result we get the id of the $c2c and of the $c.

Program recurs in different line than supposed to

I have some rules called sessions that have a Title(String) and Topics (a list of String). I'm creating a query where I input some keywords and find which session has a best match. Each keyword can have a weight (keyword is followed by -2 for example if weight of keyword is 2, if weight is 1 there is nothing next to the keyword).
My problem is in the recursion. For inputs without weights everything is ok but with weights the program recurs differently.
session('Rules; Semantic Technology; and Cross-Industry Standards',
['XBRL - Extensible Business Reporting Language',
'MISMO - Mortgage Industry Standards Maintenance Org',
'FIXatdl - FIX Algorithmic Trading Definition Language',
'FpML - Financial products Markup Language',
'HL7 - Health Level 7',
'Acord - Association for Cooperative Operations Research and Development (Insurance Industry)',
'Rules for Governance; Risk; and Compliance (GRC); eg; rules for internal audit; SOX compliance; enterprise risk management (ERM); operational risk; etc',
'Rules and Corporate Actions']).
session('Rule Transformation and Extraction',
['Transformation and extraction with rule standards; such as SBVR; RIF and OCL',
'Extraction of rules from code',
'Transformation and extraction in the context of frameworks such as KDM (Knowledge Discovery meta-model)',
'Extraction of rules from natural language',
'Transformation or rules from one dialect into another']).
accessList([H|T], H, T).
is_empty_string(String) :-
String == ''.
not_empty_string(String) :-
String \== ''.
get_weight_of_token(MyToken, WeightOfToken) :-
split_string(MyToken,"-","",TempList), % tokenize MyToken into a list ( [string_of_token, weight_of_token] )
accessList(TempList,_,TempWeight), %
accessList(TempWeight,TempWeight2,_), % accessing the tail of the list (weight_of_token)
number_string(WeightOfToken,TempWeight2) % convert string to number
;
WeightOfToken is 1.
find_relevance([], SessionTitle, SessionTopics).
find_relevance([H | T], SessionTitle, SessionTopics) :-
get_weight_of_token(H, WeightOfToken),
format('Token = ~w with weight = ~d ~n', [H,WeightOfToken]),
format('Title = ~w~nTopics = ~w ~n', [SessionTitle,SessionTopics]),
find_relevance(T, SessionTitle, SessionTopics).
query(ListOfKeywords) :-
is_empty_string(ListOfKeywords), write('List of keywords is empty!')
;
not_empty_string(ListOfKeywords),
split_string(ListOfKeywords," ","",KeywordsTokens), %tokenize string with space as separator
session(Title,Topics), find_relevance(KeywordsTokens, Title, Topics), fail.
With an input without weights like 'firword secword', this is the result:
https://imgur.com/a/Q2fU2IM
As you can see the program recurs fine and calls find_revelance for the next session.
With an input with weights like 'firword-2 secword', this is the result:
https://imgur.com/xKFpFvh
As you can see the program doesn't recur to (9) for the next session but recurs to (10) for some reason.. and specifically to the part after ;..
Why is this happening? Thanks in advance.
*I changed the titles and topics to something smaller so the images can be more clear

ArangoDB copy Vertex and Edges to neighbors

I'm trying to copy a vertex node and retain it's relationships in ArangoDB. I'm getting a "access after data-modification" error (1579). It doesn't like it when I iterate over the source node's edges and insert an edge copy within the loop. This makes sense but I'm struggling to figure out how to do what I'm wanting within a single transaction.
var query = arangojs.aqlQuery`
let tmpNode = (FOR v IN vertices FILTER v._id == ${nodeId} RETURN v)[0]
let nodeCopy = UNSET(tmpNode, '_id', '_key', '_rev')
let nodeCopyId = (INSERT nodeCopy IN 'vertices' RETURN NEW._id)[0]
FOR e IN GRAPH_EDGES('g', ${nodeId}, {'includeData': true, 'maxDepth': 1})
let tmpEdge = UNSET(e, '_id', '_key', '_rev')
let edgeCopy = MERGE(tmpEdge, {'_from': nodeCopyId})
INSERT edgeCopy IN 'edges'
`;
This quesion is somewhat similar to 'In AQL how to re-parent a vertex' - so let me explain this in a similar way.
One should use the ArangoDB 2.8 pattern matching traversals to solve this.
We will copy Alice to become Sally with similar relations:
let alice=DOCUMENT("persons/alice")
let newSally=UNSET(MERGE(alice, {_key: "sally", name: "Sally"}), '_id')
let r=(for v,e in 1..1 ANY alice GRAPH "knows_graph"
LET me = UNSET(e, "_id", "_key", "_rev")
LET newEdge = (me._to == "persons/alice") ?
MERGE(me, {_to: "persons/sally"}) :
MERGE(me, {_from: "persons/sally"})
INSERT newEdge IN knows RETURN newEdge)
INSERT newSally IN persons RETURN newSally
We therefore first load Alice. We UNSET the properties ArangoDB should set on its own. We change the properties that have to be uniq to be uniq for Alice so we have a Sally afterwards.
Then we open a subquery to traverse ANY first level relations of Alice. In this subequery we want to copy the edges - e. We need to UNSET once more the document attributes that have to be autogenerated by ArangoDB. We need to find out which side of _from and _to pointed to Alice and relocate it to Sally.
The final insert of Sally has to be outside of the subquery, else this statement will attempt to insert one Sally per edge we traverse. We can't insert Saly in front of the query as you already found out - no subsequent fetches are allowed after the insert.

How to access and mutate node property value by the property name string in Cypher?

My goal is to access and mutate a property of a node in a cypher query where the name of the property to be accessed and mutated is an unknown string value.
For example, consider a command:
Find all nodes containing a two properties such that the name of the first property is lower-case and the name of the latter is the upper-case representation of the former. Then, propagate the value of the property with the lower-case string name to the value of the property with the upper-case name.
The particular case is easy:
MATCH ( node )
WHERE has(node.age) AND has(node.AGE) AND node.age <> node.AGE
SET node.AGE = node.age
RETURN node;
But I can't seem to find a way to implement the general case in a single request.
Specifically, I am unable to:
Access the property of the node with a string and a value
Mutate the property of the node with a string and a value
For the sake of clarity, I'll include my attempt to handle the general case. Where I failed to modify the property of the node I was able to generate the cypher for a command that would accomplish my end goal if it were executed in a subsequent transaction.
MERGE ( justToMakeSureOneExists { age: 14, AGE : 140 } ) WITH justToMakeSureOneExists
MATCH (node)
WHERE ANY ( kx IN keys(node) WHERE kx = LOWER(kx) AND ANY ( ky in keys(node) WHERE ky = UPPER(kx) ) )
REMOVE node.name_conflicts // make sure results are current
FOREACH(kx in keys(node) |
SET node.name_conflicts
= COALESCE(node.name_conflicts,[])
+ CASE kx
WHEN lower(kx)
THEN []
+ CASE WHEN any ( ky in keys(node) WHERE ky = upper(kx) )
THEN ['match (node) where id(node) = ' + id(node)+ ' and node.' + upper(kx) + ' <> node.' + kx + ' set node.' + upper(kx) + ' = node.' + kx + ' return node;']
ELSE [] END
ELSE []
END )
RETURN node,keys(node)
Afterthought: It seems like the ability to mutate a node property by property name would be a pretty common requirement, but the lack of obvious support for the feature leads me to believe that the feature was omitted deliberately? If this feature is indeed unsupported is there any documentation to explain why and if there is some conflict between the approach and the recommended way of doing things in Neo/Cypher?
There is some discussion going on regarding improved support for dynamic property access in Cypher. I'm pretty confident that we will see support for this in the future, but I cannot comment on a target release nor on a date.
As a workaround I'd recommend implementing that into a unmanaged extension.
It appears that the desired language feature was added to Cypher in Neo4j 2.3.0 under the name "dynamic property". The Cypher docs from version 2.3.0-up declare the following syntax group as a valid cypher expression:
A dynamic property: n["prop"], rel[n.city + n.zip], map[coll[0]].
This feature is documented for 2.3.0 but is absent from the previous version (2.2.9).
Thank you Neo4j Team!

Correct parameter binding for SELECT WHERE .. LIKE using fmdb?

First time user of fmdb here, trying to start off doing things correctly. I have a simple single table that I wish to perform a SELECT WHERE .. LIKE query on and after trying several of the documented approaches, I can't get any to yield the correct results.
e.g.
// 'filter' is an NSString * containing a fragment of
// text that we want in the 'track' column
NSDictionary *params =
[NSDictionary dictionaryWithObjectsAndKeys:filter, #"filter", nil];
FMResultSet *results =
[db executeQuery:#"SELECT * FROM items WHERE track LIKE '%:filter%' ORDER BY linkNum;"
withParameterDictionary:params];
Or
results = [db executeQuery:#"SELECT * FROM items WHERE track LIKE '%?%' ORDER BY linkNum;", filter];
Or
results = [db executeQuery:#"SELECT * FROM items WHERE track LIKE '%?%' ORDER BY linkNum;" withArgumentsInArray:#[filter]];
I've stepped through and all methods converge in the fmdb method:
- (FMResultSet *)executeQuery:(NSString *)sql withArgumentsInArray:(NSArray*)arrayArgs orDictionary:(NSDictionary *)dictionaryArgs orVAList:(va_list)args
Depending on the approach, and therefore which params are nil, it then either calls sqlite3_bind_parameter_count(pStmt), which always returns zero, or, for the dictionary case, calls sqlite3_bind_parameter_index(..), which also returns zero, so the parameter doesn't get slotted into the LIKE and then the resultSet from the query is wrong.
I know that this is absolutely the wrong way to do it (SQL injection), but it's the only way I've managed to have my LIKE honoured:
NSString *queryString = [NSString stringWithFormat:#"SELECT * FROM items WHERE track LIKE '%%%#%%' ORDER BY linkNum;", filter];
results = [db executeQuery:queryString];
(I've also tried all permutations but with escaped double-quotes in place of the single quotes shown here)
Update:
I've also tried fmdb's own …WithFormat variant, which should provide convenience and protection from injection:
[db executeQueryWithFormat:#"SELECT * FROM items WHERE track LIKE '%%%#%%' ORDER BY linkNum;", filter];
Again, stepping into the debugger I can see that the LIKE gets transformed from this:
… LIKE '%%%#%%' ORDER BY linkNum;
To this:
… LIKE '%%?%' ORDER BY linkNum;
… which also goes on to return zero from sqlite3_bind_parameter_count(), where I would expect a positive value equal to "the index of the largest (rightmost) parameter." (from the sqlite docs)
The error was to include any quotes at all:
[db executeQuery:#"SELECT * FROM items WHERE track LIKE ? ORDER BY linkNum;", filter];
… and the % is now in the filter variable, rather than in the query.

Resources