I want to traverse a tree and aggregate the parent and its immediate children only. How would I do this using Gremlin and aggregate this into a structure list arrayOf({parent1,child},{child, child1}...}
In this case I want to output [{0,1}, {0,2}, {1,8} {1,6}, {2,7},{2,9}, {8,16},{8,14},{8,15},{7,17}}
The order isnt important. Also, note I want to avoid any circular edges which can exist on the same node only (no circular loop possible from a child vertex to a parent)
Each vertex has a label city and each edge has a label highway
g.V().hasLabel("city").toList().map(x->x.id()+x.edges(Direction.OUT,"highway").collect(Collectors.toList())
My query is timing out and I was wondering if there is a faster way to do this. I have abt 5000 vertices and two vertices are connected with only one edge.
You can get close to what you are looking for using the Gremlin tree step while also avoiding Groovy closures. Assuming the following setup:
gremlin> g = traversal().withGraph(TinkerGraph.open())
==>graphtraversalsource[tinkergraph[vertices:0 edges:0], standard]
g.addV('0').as('0').
addV('1').as('1').
addV('2').as('2').
addV('6').as('6').
addV('7').as('7').
addV('8').as('8').
addV('9').as('9').
addV('14').as('14').
addV('15').as('15').
addV('16').as('16').
addV('17').as('17').
addE('route').from('0').to('1').
addE('route').from('0').to('2').
addE('route').from('1').to('6').
addE('route').from('1').to('8').
addE('route').from('2').to('2').
addE('route').from('2').to('9').
addE('route').from('2').to('7').
addE('route').from('7').to('17').
addE('route').from('8').to('14').
addE('route').from('8').to('15').
addE('route').from('8').to('16').iterate()
A query can be written to return the tree (minus cycles) as follows:
gremlin> g.V().hasLabel('0').
......1> repeat(out().simplePath()).
......2> until(__.not(out())).
......3> tree().
......4> by(label)
==>[0:[1:[6:[],8:[14:[],15:[],16:[]]],2:[7:[17:[]],9:[]]]]
An alternative approach, that also avoids using closures:
gremlin> g.V().local(union(label(),out().simplePath().label()).fold())
==>[17]
==>[0,1,2]
==>[1,6,8]
==>[2,9,7]
==>[6]
==>[7,17]
==>[8,14,15,16]
==>[9]
==>[14]
==>[15]
==>[16]
Which can be further refined to avoid leaf only nodes using:
gremlin> g.V().local(union(label(),out().simplePath().label()).fold()).where(count(local).is(gt(1)))
==>[0,1,2]
==>[1,6,8]
==>[2,9,7]
==>[7,17]
==>[8,14,15,16]
In your code you can then create the final pairs or perhaps extend the Gremlin to break up the result even more. Hopefully these approaches will prove more efficient than falling back onto closures (which are not going to be very portable to other TinkerPop implementations that do not support in-line code).
Say for example I have edges with the types A, B, C or D.
How do I traverse using AND and OR on these edge-types?
Say I want to traverse from node X by ( in(A) AND out(B) ) OR ( both(C) AND out(D) )
The result I expect are all the nodes that have atleast one A-edge that goes to X AND atleast one B-edge that goes from X, OR atleast one C-edge that either goes from or to X AND atleast one D-edge that goes from X.
How do I write a Gremlin query to do this in a general matter? Thank you so much for any help!
Using this simple graph
g.addV('1').as('1').
addV('2').as('2').
addV('3').as('3').
addV('4').as('4').
addV('5').as('5').
addE('B').from('2').to('3').
addE('A').from('3').to('2')
Your initial match step
gremlin> g.V().
......1> match(
......2> __.as('1').in('A').as('2'),
......3> __.as('1').out('B').as('2')).
......4> select('2').by(label)
==>3
can be reduced to simply
gremlin> g.V().as('1').out('B').filter(out('A').as('1')).label()
==>3
Notice only out steps are used. The first one takes you to any adjacent vertices. The second, inside the filter, looks to see if there are ways back to where you came from (which from the current vertex point of view is another out relationship).
You can use the same pattern to replace the other match. The query can be wrapped in an or step if you truly want or behavior or keep using union if you truly want all results that match your constraints.
I figured it out!
g.V().union(
// in(A) AND out(B)
match(
__.as('1').in('A').as('2'),
__.as('1').out('B').as('2'))
.select('2'),
// OR
// both(C) AND out(D)
match(
__.as('1').both('C').as('2'),
__.as('1').out('D').as('2'))
.select('2'))
// Delete duplicates
.dedup()
I'm trying to put together a Gremlin query that returns results for 1 to n depth of a certain edge type - without having to resort to using multiple queries stitched together with .union().
I have some test data that simulates the structure of sales offices and people that work in them, including who manages which offices and which offices "roll up" under the jurisdiction of which higher level offices. The following screen shot (from Neo4j, actually) shows a subset of the graph that I'm going to reference.
The graph can be created with the following:
g.
addV('Office').as('O_111').property('code','111').
addV('Office').as('O_356').property('code','356').
addV('Office').as('O_279').property('code','279').
addV('Office').as('O_KC5').property('code','KC5').
addE('MERGES_INTO').from('O_356').to('O_111').
addE('MERGES_INTO').from('O_279').to('O_356').
addE('MERGES_INTO').from('O_KC5').to('O_279').
addV('Person').as('Bob').property('name','Bob').
addE('MANAGES').from('Bob').to('O_111').addE('WORKS_WITH').from('Bob').to('O_111').
addV('Person').as('Michael').property('name','Michael').addE('WORKS_WITH').from('Michael').to('O_111').
addV('Person').as('John').property('name','John').addE('WORKS_WITH').from('John').to('O_111').
addV('Person').as('Rich').property('name','Rich').addE('WORKS_WITH').from('Rich').to('O_111').
addV('Person').as('Matt').property('name','Matt').
addE('WORKS_WITH').from('Matt').to('O_279').addE('MANAGES').from('Matt').to('O_279').
addV('Person').as('Judy').property('name','Judy').addE('WORKS_WITH').from('Judy').to('O_279').
addV('Person').as('Joe').property('name','Joe'). addE('WORKS_WITH').from('Joe').to('O_279').
addV('Person').as('Ben').property('name','Ben').addE('WORKS_WITH').from('Ben').to('O_279').
addV('Person').as('Ron').property('name','Ron').addE('WORKS_WITH').from('Ron').to('O_KC5').
If I want to see which people (orange) that work with an office (pink) that Bob directly or indirectly manages (because, for example, offices KC5, 279, and 356 roll up to Bob's 111 office), I can use .union() and something like the following to get the proper results:
gremlin> g.V().has('Person','name','Bob').
......1> out('MANAGES').
......2> union(
......3> __.in('WORKS_WITH'),
......4> __.in('MERGES_INTO').in('WORKS_WITH'),
......5> __.in('MERGES_INTO').in('MERGES_INTO').in('WORKS_WITH'),
......6> __.in('MERGES_INTO').in('MERGES_INTO').in('MERGES_INTO').in('WORKS_WITH')
......7> ).
......8> values('name').fold()
==>[Bob, Michael, John, Rich, Matt, Judy, Joe, Ben, Ron]
That seems super verbose and awkward. Is that my only choice? Is there a better way that doesn't seem so redundant like .union()?
Coming from a Neo4j world, I'd just do something with a ranged depth of "0 or more" using *0.., like this:
MATCH (manager:Person {name:'Bob'})
OPTIONAL MATCH (manager)-[:MANAGES]->(:Office)<-[:MERGES_INTO*0..]-(:Office)<-[:WORKS_WITH]-(p:Person)
RETURN p
How do I achieve the same sort of thing in Gremlin? Even if I can't do open ended, but could do 1 to some arbitrary limit (say, 1 to 10), that would work. It probably wouldn't matter, but I will be using AWS Neptune for the actual Graph database.
When asking questions about Gremlin, a picture of your graph is nice, but a script that provides some sample data is even better - like this:
g.addV('person').property('name','michael').as('mi').
addV('person').property('name','john').as('jo').
addV('person').property('name','rich').as('ri').
addV('person').property('name','bob').as('bo').
addV('person').property('name','matt').as('ma').
addV('person').property('name','ron').as('ro').
addV('person').property('name','joe').as('joe').
addV('person').property('name','ben').as('be').
addV('person').property('name','judy').as('ju').
addV('office').property('name','111').as('111').
addV('office').property('name','356').as('356').
addV('office').property('name','279').as('279').
addV('office').property('name','kc5').as('kc5').
addE('mergesInto').from('kc5').to('279').
addE('mergesInto').from('279').to('356').
addE('mergesInto').from('356').to('111').
addE('worksWith').from('mi').to('111').
addE('worksWith').from('jo').to('111').
addE('worksWith').from('ri').to('111').
addE('worksWith').from('bo').to('111').
addE('manages').from('bo').to('111').
addE('worksWith').from('ma').to('279').
addE('manages').from('ma').to('279').
addE('worksWith').from('joe').to('279').
addE('worksWith').from('be').to('279').
addE('worksWith').from('ju').to('279').
addE('worksWith').from('ro').to('kc5').iterate()
Your instincts are correct where union() isn't quite right for what you want to do. I would prefer repeat():
gremlin> g.V().has('person','name','bob').
......1> out('manages').
......2> repeat(__.in('worksWith','mergesInto')).
......3> emit(hasLabel('person')).
......4> values('name')
==>bob
==>michael
==>john
==>rich
==>matt
==>joe
==>ben
==>judy
==>ron
In this way it traverses to arbitrary depth (though we tend to recommend setting some kind of sensible limit to avoid problems if you run into some unexpected cycle somewhere) and is much more succinct. Note the use of emit() which controls which types of vertices are returned from the repeat() - if you do not include that filter you will also return "office" vertices.
I'm quite new to Gremlin, I've been practicing a bit with this guide, but when it comes to writing more complex queries I clearly haven't got the hang of it yet. To put you in context, I'm trying to answer a question that in SQL can easily be cracked with a self-join.
Imagine the following simplified graph:
As you can see, there are two types of entities in the graph: Routes and Legs. A Route is made of 1+ Legs following a particular order (specified in the edge), and a Leg can be in several Routes.
The question I want to answer is: which routes travel from one country to another, and then back to the previous country?
In the case of the graph above, Route 1 goes from ES to FR in the first Leg, and from FR to ES in the third Leg, so the output of the query would look like:
=> Route id: 1
=> Leg1 order: 1
=> Leg1 id: 1
=> Leg2 order: 3
=> Leg2 id: 3
If I had the following relational table:
route_id leg_id order source_country destination_country
1 1 1 ES FR
1 2 2 FR FR
1 3 3 FR ES
I could get the desired output with the following query:
SELECT
a.route_id
,a.leg_id
,a.order
,b.leg_id
,b.order
FROM Routes a
JOIN Routes b
ON a.route_id = b.route_id
AND a.source_country = b.destination_country
AND a.destination_country = b.source_country
WHERE a.source_country <> a.destination_country;
When it comes to writing it in Gremlin, I'm really not quite sure how to start. My inexperience makes me want to perform a self-join as well, but even then I didn't get very far:
g.V().hasLabel('Route').as('a').V().hasLabel('Route').as('b').where('a', eq('b')).and(join 'a' edges&legs with 'b' edges&legs)...
And that's about it, because I don't know how to reference a again as an object that can be traversed to look for the edges and legs connected to the routes.
Any help/guidance would be greatly appreciated, it could definitely happen that this problem can be solved in a simpler way as well :)
Thanks,
Béntor
With graphs you should try to think of terms of "navigating connected things" rather than "joining disparate things" because with a graph the things are already joined explicitly. It also helps to think in terms of streams of things being lazily evaluated (i.e. objects going from one Gremlin step to the next).
First of all, the picture is nice but it's always more helpful to provide some sample data in the form of a Gremlin script like this:
g = TinkerGraph.open().traversal()
g.addV('route').property('rid',1).as('r1').
addV('route').property('rid',2).as('r2').
addV('route').property('rid',3).as('r3').
addV('leg').property('lid',1).property('source','ES').property('dest','FR').as('l1').
addV('leg').property('lid',2).property('source','FR').property('dest','FR').as('l2').
addV('leg').property('lid',3).property('source','FR').property('dest','ES').as('l3').
addV('leg').property('lid',4).property('source','ES').property('dest','FR').as('l4').
addV('leg').property('lid',5).property('source','FR').property('dest','FR').as('l5').
addV('leg').property('lid',6).property('source','FR').property('dest','US').as('l6').
addE('has_leg').from('r1').to('l1').property('order',1).
addE('has_leg').from('r1').to('l2').property('order',2).
addE('has_leg').from('r1').to('l3').property('order',3).
addE('has_leg').from('r3').to('l4').property('order',1).
addE('has_leg').from('r3').to('l5').property('order',2).
addE('has_leg').from('r3').to('l6').property('order',3).
addE('has_leg').from('r2').to('l2').property('order',1).iterate()
Your question was:
which routes travel from one country to another, and then back to the previous country?
Note that I added some extra data that didn't meet the requirements of that question to be sure my traversal was working properly. I suppose I assumed that you were open to getting routes that just stayed in the country like a leg that just went from "FR" to FR" as it started in "FR" and ended in that "previous country". I guess I could revise this further to do that if you really needed me to, but for now I will stick with that assumption since you're just learning.
After considering the data and reading that question I immediately thought, let's find the routes which you did well enough and then let's just see what it takes to get the start leg of the trip and the end leg of the trip for that route:
gremlin> g.V().hasLabel('route').
......1> map(outE('has_leg').
......2> order().by('order').
......3> union(limit(1).inV().values('source'), tail().inV().values('dest')).
......4> fold())
==>[ES,ES]
==>[FR,FR]
==>[ES,US]
So, I find a "route" vertex with hasLabel('route') and then I convert each into a List of the start and end country (i.e. a pair where the first item is the "source" country and the second item is the "dest" country). To do that I traverse outgoing "has_leg" edges, order them. Once ordered I grab the first edge in the stream (i.e with limit(1)) and traverse to the incoming "leg" vertex and grab its "source" value and do the same for the last incoming vertex of the edge (i.e. with tail()) but this time grab its "dest" value. We then use fold() to push that two item stream from union() into a List. Again, because this all happens inside of map() we are effectively doing it for each "route" vertex so we get three pairs as a result.
With that output we just now need to compare the start/end values in the pairs to determine which represent a route starting and ending in the same country.
gremlin> g.V().hasLabel('route').
......1> filter(outE('has_leg').
......2> order().by('order').
......3> fold().
......4> project('start','end').
......5> by(unfold().limit(1).inV().values('source')).
......6> by(unfold().tail().inV().values('dest')).
......7> where('start', eq('end'))).
......8> elementMap()
==>[id:0,label:route,rid:1]
==>[id:2,label:route,rid:2]
At line 1, note that we changed map() to filter(). I only used map() initially so that I could see the results of what I was traversing before I worried about how to use those results to get rid of the data I didn't want. That's a common practice with Gremlin as you build more and more complexity in your traversals. So we are now ready to apply a filter() to each "route" vertex. I imagine that there are a number of ways to do this, but I chose to gather all the ordered edges into a List at line 3. I then project() that step at line 4 and transform the edge list for both "start" and "end" keys using the associated by() modulators. In both cases I must unfold() the edge list to a stream and then apply the same limit(1) and tail() sort of traversal that was explained earlier. The result is a Map with "start" and "end" keys which can be compared using where() step. As you can see from the result, the third route that started in "ES" and ended in "US" has been filtered away.
I'll expand my answer based on your comment - Since all of my previous data seems to align with your more general case of wanting to find any route that returns to a country in any sense:
g = TinkerGraph.open().traversal()
g.addV('route').property('rid',1).as('r1').
addV('route').property('rid',2).as('r2').
addV('route').property('rid',3).as('r3').
addV('route').property('rid',4).as('r4').
addV('leg').property('lid',1).property('source','ES').property('dest','FR').as('l1').
addV('leg').property('lid',2).property('source','FR').property('dest','FR').as('l2').
addV('leg').property('lid',3).property('source','FR').property('dest','ES').as('l3').
addV('leg').property('lid',4).property('source','ES').property('dest','FR').as('l4').
addV('leg').property('lid',5).property('source','FR').property('dest','FR').as('l5').
addV('leg').property('lid',6).property('source','FR').property('dest','US').as('l6').
addV('leg').property('lid',7).property('source','ES').property('dest','FR').as('l7').
addV('leg').property('lid',8).property('source','FR').property('dest','CA').as('l8').
addV('leg').property('lid',9).property('source','CA').property('dest','US').as('l9').
addE('has_leg').from('r1').to('l1').property('order',1).
addE('has_leg').from('r1').to('l2').property('order',2).
addE('has_leg').from('r1').to('l3').property('order',3).
addE('has_leg').from('r3').to('l4').property('order',1).
addE('has_leg').from('r3').to('l5').property('order',2).
addE('has_leg').from('r3').to('l6').property('order',3).
addE('has_leg').from('r4').to('l7').property('order',1).
addE('has_leg').from('r4').to('l8').property('order',2).
addE('has_leg').from('r4').to('l9').property('order',3).
addE('has_leg').from('r2').to('l2').property('order',1).iterate()
If I have this right the newly added "rid=4" route should be filtered as its route never revisits the same country. I think this bit of Gremlin is even easier than what I suggested previously because now we just need to look for unique routes which means that if we satisfy one of these two situations then we've found a route we care about:
There is one leg and it starts/ends in the same country
There are multiple legs and if the number of times that country appears in the route exceeds 2 (because we are taking into account "source" and "dest")
Here's the Gremlin:
gremlin> g.V().hasLabel('route').
......1> filter(out('has_leg').
......2> union(values('source'),
......3> values('dest')).
......4> groupCount().
......5> or(select(values).unfold().is(gt(2)),
......6> count(local).is(1))).
......7> elementMap()
==>[id:0,label:route,rid:1]
==>[id:2,label:route,rid:2]
==>[id:4,label:route,rid:3]
If you understood my earlier explanations of the code, then you likely follow everything up to line 5 where we take the Map produced by the groupCount() on country names and apply the two filter conditions I just described. At line 5, we apply the second condition which extracts the values from the Map (i.e. the counts of the number of times each country appears) and detects if any are greater than 2. On line 6, we count the entries in the Map which maps to the first condition. Note that we use local there because we aren't counting the Map-objects in the stream but the entries within the Map (i.e. local to the Map).
Just in case it's useful here is a similar example I was playing with before I saw Stephen had already answered. This uses the air-routes data set from the tutorial. The first example starts specifically at LHR. The second looks at all airports. I assumed a constant of 2 segments. You could change that by modifying the query, and, as Stephen mentioned, there are many ways you could approach this.
gremlin> g.V().has('code','LHR').as('a').
......1> out().
......2> where(neq('a')).by('country').
......3> repeat(out().simplePath()).times(1).
......4> where(eq('a')).by('country').
......5> path().
......6> by(values('country','code').fold()).
......7> limit(5)
==>[[UK,LHR],[MA,CMN],[UK,LGW]]
==>[[UK,LHR],[MA,CMN],[UK,MAN]]
==>[[UK,LHR],[MA,TNG],[UK,LGW]]
==>[[UK,LHR],[CN,CTU],[UK,LGW]]
==>[[UK,LHR],[PT,FAO],[UK,BHX]]
gremlin> g.V().hasLabel('airport').as('a').
......1> out().
......2> where(neq('a')).by('country').
......3> repeat(out().simplePath()).times(1).
......4> where(eq('a')).by('country').
......5> path().
......6> by(values('country','code').fold()).
......7> limit(5)
==>[[US,ATL],[CL,SCL],[US,DFW]]
==>[[US,ATL],[CL,SCL],[US,IAH]]
==>[[US,ATL],[CL,SCL],[US,JFK]]
==>[[US,ATL],[CL,SCL],[US,LAX]]
==>[[US,ATL],[CL,SCL],[US,MCO]]
For your specific example, the technique Stephen used taking advantage of segments having an order number is much nicer. The air-routes data set does not have a concept of a segment but thought this might be of some interest as you start exploring Gremlin more.
How do I use the times() Step on my repeat(..) to create multiple, identical edges at once?
g.V().has('Label1', 'id', '1234').repeat(addE('HAS').from(g.V().has('Label2', 'id', '5678'))).times(5)
I would think that it adds my edge 5 times to this vertex, in fact it returns nothing when times() is great than 1. Why is that and how would I use repeat() correctly?
I'm not sure what graph database you are using, but I'm somewhat surprised you don't get an error with that bit of Gremlin and that error should yield a hint as to what is wrong.
gremlin> g.V().has('person','name','marko').repeat(addE('knows').from(V().has('person','name','stephen'))).times(5)
org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerEdge cannot be cast to org.apache.tinkerpop.gremlin.structure.Vertex
Type ':help' or ':h' for help.
Display stack trace? [yN]
The repeat() step is not meant to simply execute the same child traversal with the same input for each iteration. It is meant to execute the same child traversal with the output of the previous iteration as the new input. That means on the first iteration we initialize that child traversal of:
addE('knows').from(V().has('person','name','stephen'))
with the "marko" vertex, but the output of that traversal is an Edge (because the output of addE() is an Edge). On the second iteration that edge becomes the input to addE() and therefore....error....as you can't call addE() on an edge.
If you want to use repeat() for this type of flow control you can though, but you need to arrange the child traversal so that the input is that same initial vertex on each iteration:
gremlin> g.addV('person').property('name','marko').addV('person').property('name','stephen').iterate()
gremlin> g.V().has('person','name','marko').as('m').
......1> V().has('person','name','stephen').as('s').
......2> repeat(select('m').addE('knows').to('s')).
......3> times(3).iterate()
gremlin> g.E()
==>e[4][0-knows->2]
==>e[5][0-knows->2]
==>e[6][0-knows->2]