OrientDB ETL edge lookup from query - how to access $input? - graph

Question

I'm trying my darnedest to do an ETL import from a large dataset that I've been keeping in MongoDB. I've successfully imported the vertices, and I feel like I'm one little syntax misunderstanding away from importing the edges too.
I am pretty sure that the error is in this transformer:
{"edge":{"class":"Friend", "joinFieldName":"id",
"lookup": "select from Character WHERE $input.id IN character_friends",
"unresolvedLinkAction":"CREATE"}},
So what I'm trying to do is to make an edge from a document with id = FOO to all other documents that contain FOO in their character_friends array.
If I execute
select from Character WHERE FOO IN character_friends
in the browser, I get a ton of documents, so my guess is that my problem is with $input.id either not returning the id I'm expecting, or maybe not being recognized as a variable at all.
Documents look like this:
{
id: FOO,
character_friends: [BAR, BAZ, QUX]
(and a bunch of other junk)
}

Answer 1

Seems you're inserting a property "id", but it's reserved in Blueprints standard. You can rename it (with "field" transformers) or set this in Orient Loader:
standardElementConstraints: false,
Then I've created the file /temp/datasets/charles.json with this content:
[
{
name: "Joe",
id: 1,
friends: [2,4,5],
enemies: [6]
},
{
name: "Suzie",
id: 2,
friends: [1,4,6],
enemies: [5,2]
}
]
And this pipeline:
{
config: {
log: "debug",
parallel: false
},
source : {
file: { path: "/temp/datasets/charles.json", lock : true }
},
extractor : {
json: {}
},
transformers : [
{ merge: { joinFieldName:"id", lookup:"Account.id" } },
{ vertex: { class: "Account"} },
{ edge: {
"class": "Friend",
"joinFieldName": "friends",
"lookup": "Account.id",
"unresolvedLinkAction": "CREATE"
} },
{ edge: {
"class": "Enemy",
"joinFieldName": "enemies",
"lookup": "Account.id",
"unresolvedLinkAction": "CREATE"
} }
],
loader : {
orientdb: {
dbURL: "plocal:/temp/databases/charles",
dbUser: "admin",
dbPassword: "admin",
dbAutoDropIfExists: true,
dbAutoCreate: true,
standardElementConstraints: false,
tx: false,
wal: false,
batchCommit: 1000,
dbType: "graph",
classes: [{name: 'Account', extends:"V"}, {name: 'Friend', extends:"E"}, {name: 'Enemy', extends:"E"}],
indexes: [{class:"Account", fields:["id:integer"], type:"UNIQUE_HASH_INDEX" }]
}
}
}
Assure to use last version of ETL jar (replace it in $ORIENTDB/lib) with default version. Last version is downloadable from:
https://oss.sonatype.org/content/repositories/snapshots/com/orientechnologies/orientdb-etl/2.0.2-SNAPSHOT/orientdb-etl-2.0.2-20150208.225903-1.jar
Or get OrientDB ETL 2.0.2 of major.