Using Tinkerpop Gremlin (Neptune DB), is there a preferred/"faster" way to query?
For example, let's say I have a graph containing the node:
label: Student
id: 'student/12345'
studentId: '12345'
name: 'Bob'
Is there a preferred query? (for this example let's say we know the field 'studentId' value, which is also part of the id)
g.V().filter('studentId', '12345')
vs
g.V().filter(hasId(TextP.containing('12345'))
or using "has"/"hasId" vs "filter"?
g.V().has('studentId', '12345')
vs
g.V().hasId(TextP.containing('12345'))
So there seems to be two questions here, one about filter() vs has() and the other about using the vertex id versus a property.
The answer to the first question is going to depend on the underlying database implementation and what is has/has not optimized. In general, and in Neptune, I would suggest using the g.V().has('studentId', '12345') pattern to filter on a property as it is optimized and easier to read.
The answer to the second question also depends on the database implementaiton, as not all allow for setting of the vertex ids. Other databases may vary but in Neptune setting ids is allowed and a direct lookup by ID is the fastest (e.g. g.V('12345') or g.V().hasId('12345')) way to look something up as it is a single index lookup. One thing to note is that in Neptune vertex/edge id values need to be globally unique so you need to ensure that you will only have one vertex or edge with a specific id.
Related
On an Azure cosmosDB gremlin instance,
I have 2 vertices A and B linked by and edge E.
Both vertices has a 'name' property.
I'd like to run a query which will take A's name and put it in B
when I run
g.V("AId").as("a").oute().inv().hasLabel('B').property("name",select('a').values('name'))
I get the following error :
GraphRuntimeException ExceptionMessage : Gremlin Query Execution Error: Cannot create ValueField on non-primitive type GraphTraversal.
It looks like the select operator is not correctly used.
Thank you for your help
EDITED based on discussion in comments
You have oute and inv in lower case. In general, the steps use camelCase naming, such as outE and inV (outside of specific GLVs), but in the comments it was mentioned that CosmosDB will accept all lower case step names. Assuming therefore, that is not the issue here, the query as written looks fine in terms of generic Gremlin. The example below was run using TinkerGraph, and uses the same select mechanism to pick the property value.
gremlin> g.V(3).as("a").outE().inV().has('code','LHR').property("name",select('a').values('city'))
==>v[49]
gremlin> g.V(49).values('name')
==>Austin
What you are observing may be specific to CosmosDB and it's probably worth contacting their support folks to double check.
I am trying to figure out (at this point I think the answer is No) if it is possible to build a index on a List Attribute and query NOT_CONTAINS on that attribute.
Example table:
Tasks
Task_id: string
solved_by: List<String> # stores list of user_ids who previously solved this task.
My query would be:
Get me all the tasks not yet solved by current_user
select * from tasks where tasks.solved_by NOT_CONTAINS current_user_id
Is it possible to do this without full scans. I tried creating an attribute of type L but aws cli errors out saying Member must satisfy enum value set: [B, N, S]
If this is not possible with dynamodb, please suggest what datastore I can use.
Any help is highly appreciated. Thanks!
As you found out, and as the error you got suggests, this is NOT possible.
However, I'd argue if your design couldn't be improved. Storing a potentially unbound list of entries (users in your case) inside a single item, which is limited to 400kb seems dangerous.
If instead, you'd store for each task the information that a particular user resolved it as a separate item (partition key - task_id, sort key - user_id) than you could easily look up if a user solved a task or not. You could also store additional information about the particular solution or attempts.
If you haven't heard of DynamoDB single table design yet, or how to overload indexes, I can recommend looking at
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-modeling-nosql-B.html
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-gsi-overloading.html
https://www.dynamodbbook.com/
Update
I just realised, you care about a negation (NOT_CONTAINS) - for those, you can't use an index anyway. For the sort key you can only use positive comparison (=, <, >, <=, >=, between, begins_with): https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions
So you might have to rethink the whole approach, to better pre-process the data stored in DDB, so it's easier to fetch, or pick a different database.
In your original question, you defined your access pattern as
Get me all the tasks not yet solved by current_user
In a later comment, you clarified that the access pattern is
A solver should be shown a task that is not yet solved by them.
which is a slightly different access pattern.
Here's one way you could fetch a task not yet solved by a user.
In this data model, I chose to model Users and Tasks as separate items. Tasks have numerically increasing ID's. Each User item should start with the lastSolved attribute set to 1. Each time you fetch a new Task for a user, you fetch TASK#{last_solved+1} and increment the lastSolved attribute by 1.
You could probably take a similar approach by using timestamps instead of numbers... anything sortable, really.
I want to create 1000+ Edges in a single query.
Currently, I am using the AWS Neptune database and gremlin.net for creating it.
The issue I am facing is related to the speed. It took huge time because of HTTP requests.
So I am planning to combine all of my queries in a string and executing in a single shot.
_g.AddE("allow").From(_g.V().HasLabel('person').Has('name', 'name1')).To(_g.V().HasLabel('phone').Where(__.Out().Has('sensor', 'nfc'))).Next();
There are chances that the "To" (target) Vertex may not be available in the database. When it is the case this query fails as well. So I had to apply a check if that vertex exists before executing this query using hasNext().
So as of now its working fine, but when I am thinking of combining all 1000+ edge creation at once, is it possible to write a query which doesn't break if "To" (target) Vertex not found?
You should look at using the Element Existence pattern for each vertex as shown in the TinkerPop Recipes.
In your example you would replace this section of your query:
_g.V().HasLabel('person').Has('name', 'name1')
with something like this (I don't have a .NET environment to test the syntax):
__.V().Has('person', 'name', 'name1').Fold().
coalesce(__.Unfold(), __.AddV('person').Property('name', 'name1')
This will act as an Upsert and either return the existing vertex or add a new one with the name property. This same pattern can then be used on your To step to ensure that it exists before the edge is created as well.
I want to return a node where the node has a property as a specific uuid and I just want to return one of them (there could be several matches).
g.V().where('application_uuid', eq(application_uuid).next()
Would the above query return all the nodes? How do I just return 1?
I also want to get the property map of this node. How would I do this?
You would just do:
g.V().has('application_uuid', application_uuid).next()
but even better would be the signature that includes the vertex label (if you can):
g.V().has('vlabel', 'application_uuid', application_uuid).next()
Perhaps going a bit further if you explicitly need just one you could:
g.V().has('vlabel', 'application_uuid', application_uuid).limit(1).next()
so that both the graph provider and/or Gremlin Server know your intent is to only next() back one result. In that way, you may save some extra network traffic/processing.
This is a very basic query. You should read more about gremlin. I can suggest Practical Gremlin book.
As for your query, you can use has to filter by property, and limit to get specific number of results:
g.V().has('application_uuid', application_uuid).limit(1).next()
Running your query without the limit will also return a single result since the query result is an iterator. Using toList() will return all results in an array.
I am not clear on JanusGraph (0.1.1 or upcoming 0.2.0) index behavior.
All indexes (Composite-index, Mixed-index and Vertex-centric-index) are used only to lookup entry point elements ?
For example, in the following traversal, indexes are use only at has('Lang', 'name', 'python') ?
g.V().has('Lang', 'name', 'python').in('use').has('person', 'age', P.gt(28)).toList()
If so, when an intermediate vertex has so many edges (super-node), JanusGraph must read so many elements from backend storage in order to filter ?
After the initial lookup you can only leverage vertex centric indexes to improve the performance of traversals. You could, for example, duplicate the person's age property (also store it on the use edges and create a VCI over this property).
mgmt.buildEdgeIndex(uses, "usedByAge", Direction.IN, Order.incr, age);