I have a graph in Neo4j, where all nodes and edges have a property p. With a cypher query as follows, I get a subgraph of the whole graph that fulfills several conditions on p for vertices and edges in general.
Q1
MATCH (n)
WHERE n.p ... // some conditions for nodes
OPTIONAL MATCH (n)-[r]-()
WHERE r.p ... // some conditions for relationships
RETURN n,r
Now I want to execute a second query Q2 on the resulting subgraph of the first query Q1. For example
Q2
MATCH (a:Person)-[:knows*5]-(b:Person)-[s:studiedAt]-(u:University),
(b)-[:owns]-(i:Business)
WHERE i.city = "Boston"
RETURN DISTINCT a.name, i.name, u.name
I do not want to apply the general conditions of Q1 for each vertex and edge in Q2. That complicates the query, loses the generalization, and needs path variables for variable paths like [:knows*5].
Is there a smarter way to execute a query Q2 on the result of a query Q1? Or is this impossible in Cypher because of its missing composability and the fact, that the result is always a table and never a graph?
It's not present directly in Cypher as far as I know, but if you are using Neo4j Enterprise, then you can use Neo4j Graph Data Science Library, to create named subgraphs, and then you can query those subgraphs. The disadvantage that comes with this approach is your subgraph won't be updated as the original graph gets updated. Please go through the following documentation, on how to create a subgraph:
Projecting a subgraph
Querying your subgraph
Related
In Tinkerpop Gremlin,
I have two select query
g.V().has("id","foo").out().values("x").toList();
g.V().has("bar","foo").out().values("id").toList();
Now Can we club these query into one batched tinkerpop gremlin?
I tried
g.V().has("id","foo").out().values("x").union(__.V().has("bar","foo").out().values("id")).toList()
but this leads to a single list instead of two separate list. I want to extract the response of these two queries separately.
You could start your traversal with some dummy value and then union() the two traversals together:
gremlin> g.inject(0).union(V(1).out().fold(),V(2).in().fold())
==>[v[3],v[2],v[4]]
==>[v[1]]
I need to get vertices filtered by a specific predicate on the properties, and all the edges (with a particular label, and perhaps some predicate on the properties of an edge) existing between them.
This is for a Cosmos Azure Db graph, and the solution should be a single Gremlin query.
So far I am thinking of something along the lines of:
g.V().has('property1', value1).has('property2', value2).select('vertices')
.outE().as('edges').inV().has('property1', value1).has('property2', value2)
.select('vertices','edges')
Is there a better way to achieve this?
Given the description and your comment, this traversal should work for you:
g.V().has('property1', value1).has('property2', value2).
aggregate('v').
outE(). /* add edge filters here */
inV().where(within('v')).
path().
union(union(limit(local, 1),
tail (local, 1)).dedup().aggregate('vertices'),
range(local, 1, 2).aggregate('edges')).
cap('vertices','edges').next()
Using graph algorithms extension and the inbuilt shortest-path search in neo4j does not look at the direction of the relationships as long as 2 nodes are connected. Is there a way to query the graph db to include the directionality of the relationships connecting nodes, or would you have to code dijikstra's from scratch without leveraging neo4j and its graph algorithm library capabilities?
I'm currently using a query structured in this fashion:
MATCH (start:Db_Nodes{uid:"xxx"}),(end:Db_Nodes{uid:"yyy"}) CALL algo.shortestPath.stream(start, end, "weight") YIELD nodeId, cost MATCH (node) WHERE id(node) = nodeId RETURN node
The APOC procedure apoc.algo.dijkstra should work for you.
For instance, if you want the shortest weighted path with only outgoing relationships from start to end (regardless of relationship type):
MATCH (start:Db_Nodes{uid:"xxx"}), (end:Db_Nodes{uid:"yyy"})
CALL apoc.algo.dijkstra(start, end, '>', 'weight') YIELD path, weight AS totalWeight
RETURN path, totalWeight;
We have a multi-graph of email connections. In this graph there are many edges between 2 nodes. Is there any cypher query to aggregate all edges between each node pairs and then return a new graph?
ATTENTION: I know the query that aggregate edges, but it returns a table, not graph!
To visualize the relationships together you can use apoc's virtual relationships:
MATCH (a:Foo)-[r]->(b:Bar)
RETURN a,b, apoc.create.vRelationship(a,'COMBINED',{count:count(*)},b) as rel
Given we have the following Neo4j schema (simplified but it shows the important point). There are two types of nodes NODE and VERSION. VERSIONs are connected to NODEs via a VERSION_OF relationship. VERSION nodes do have two properties from and until that denote the validity timespan - either or both can be NULL (nonexistent in Neo4j terms) to denote unlimited. NODEs can be connected via a HAS_CHILD relationship. Again these relationships have two properties from and until that denote the validity timespan - either or both can be NULL (nonexistent in Neo4j terms) to denote unlimited.
EDIT: The validity dates on VERSION nodes and HAS_CHILD relations are independent (even though the example coincidentally shows them being aligned).
The example shows two NODEs A and B. A has two VERSIONs AV1 until 6/30/17 and AV2 starting from 7/1/17 while B only has one version BV1 that is unlimited. B is connected to A via a HAS_CHILD relationship until 6/30/17.
The challenge now is to query the graph for all nodes that aren't a child (that are root nodes) at one specific moment in time. Given the example above, the query should return just B if the query date is e.g. 6/1/17, but it should return B and A if the query date is e.g. 8/1/17 (because A isn't a child of B as of 7/1/17 any more).
The current query today is roughly similar to that one:
MATCH (n1:NODE)
OPTIONAL MATCH (n1)<-[c]-(n2:NODE), (n2)<-[:VERSION_OF]-(nv2:ITEM_VERSION)
WHERE (c.from <= {date} <= c.until)
AND (nv2.from <= {date} <= nv2.until)
WITH n1 WHERE c IS NULL
MATCH (n1)<-[:VERSION_OF]-(nv1:ITEM_VERSION)
WHERE nv1.from <= {date} <= nv1.until
RETURN n1, nv1
ORDER BY toLower(nv1.title) ASC
SKIP 0 LIMIT 15
This query works relatively fine in general but it starts getting slow as hell when used on large datasets (comparable to real production datasets). With 20-30k NODEs (and about twice the number of VERSIONs) the (real) query takes roughly 500-700 ms on a small docker container running on Mac OS X) which is acceptable. But with 1.5M NODEs (and about twice the number of VERSIONs) the (real) query takes a little more than 1 minute on a bare-metal server (running nothing else than Neo4j). This is not really acceptable.
Do we have any option to tune this query? Are there better ways to handle the versioning of NODEs (which I doubt is the performance problem here) or the validity of relationships? I know that relationship properties cannot be indexed, so there might be a better schema for handling the validity of these relationships.
Any help or even the slightest hint is greatly appreciated.
EDIT after answer from Michael Hunger:
Percentage of root nodes:
With the current example data set (1.5M nodes) the result set contains about 2k rows. That's less than 1%.
ITEM_VERSION node in first MATCH:
We're using the ITEM_VERSION nv2 to filter the result set to ITEM nodes that have no connection other ITEM nodes at the given date. That means that either no relationship must exist that is valid for the given date or the connected item must not have an ITEM_VERSION that is valid for the given date. I'm trying to illustrate this:
// date 6/1/17
// n1 returned because relationship not valid
(nv1 ...)->(n1)-[X_HAS_CHILD ...6/30/17]->(n2)<-(nv2 ...)
// n1 not returned because relationship and connected item n2 valid
(nv1 ...)->(n1)-[X_HAS_CHILD ...]->(n2)<-(nv2 ...)
// n1 returned because connected item n2 not valid even though relationship is valid
(nv1 ...)->(n1)-[X_HAS_CHILD ...]->(n2)<-(nv2 ...6/30/17)
No use of relationship-types:
The problem here is that the software features a user-defined schema and ITEM nodes are connected by custom relationship-types. As we can't have multiple types/labels on a relationship the only common characteristic for these kind of relationships is that they all start with X_. That's been left out of the simplified example here. Would searching with the predicate type(r) STARTS WITH 'X_' help here?
What Neo4j version are you using.
What percentage of your 1.5M nodes will be found as roots at your example date, and if you don't have the limit how much data comes back? Perhaps the issue is not in the match so much as in the sorting at the end?
I'm not sure why you had the VERSION nodes in your first part, at least you don't describe them as relevant for determining a root node.
You didn't use relationship-types.
MATCH (n1:NODE) // matches 1.5M nodes
// has to do 1.5M * degree optional matches
OPTIONAL MATCH (n1)<-[c:HAS_CHILD]-(n2) WHERE (c.from <= {date} <= c.until)
WITH n1 WHERE c IS NULL
// how many root nodes are left?
// # root nodes * version degree (1..2)
MATCH (n1)<-[:VERSION_OF]-(nv1:ITEM_VERSION)
WHERE nv1.from <= {date} <= nv1.until
// has to sort all those
WITH n1, nv1, toLower(nv1.title) as title
RETURN n1, nv1
ORDER BY title ASC
SKIP 0 LIMIT 15
I think a good start for improvement would be to match on nodes using an index so you can quickly get a smaller relevant subset of nodes to search. Your approach right now must inspect all your :NODEs and all their relationships and patterns off of them every single time, which, as you've found, won't scale with your data.
Right now the only nodes in your graph with date/time properties are your :ITEM_VERSION nodes, so let's start with those. You'll need an index on :ITEM_VERSION's from and until properties for fast lookup.
The nulls are going to be problematic for your lookups, as any inequality against a null value returns null, and most workarounds to working with nulls (using COALESCE() or several ANDs/ORs for null cases) seem to prevent usage of index lookups, which is the point of my particular suggestion.
I would encourage you to replace your nulls in from and until with min and max values, which should let you take advantage of finding nodes by index lookup:
MATCH (version:ITEM_VERSION)
WHERE version.from <= {date} <= version.until
MATCH (version)<-[:VERSION_OF]-(node:NODE)
...
That should at least provide quick access to a smaller subset of nodes at the start for continuing your query.