In eXist 4.7 I am trying to move masses of documents into seperate collections.
I build the new target collection dynamically from information in the document (#corresp). So this:
for $doc in collection($globalvar:URIdb)//tei:TEI[#type="exempla"]
let $source := concat("db/apps/thema/data/",
$doc/data(#xml:id),".xml")
let $target := concat("db/apps/thema/data/",
$doc/replace(#corresp,"#",""),
"/",
$doc/data(#xml:id),".xml")
return ($source, $target)
...returns sequences like the following, which are the correct from/to file paths:
db/apps/thema/data/TE001553.xml
db/apps/thema/data/TC0001/TE001553.xml
db/apps/thema/data/TE003403.xml
db/apps/thema/data/TC0001/TE003403.xml
db/apps/thema/data/TE003404.xml
db/apps/thema/data/TC0001/TE003404.xml
But using the function file:move($original as item(), $destination as item()) as xs:boolean:
for $doc in collection($globalvar:URIdb)//tei:TEI[#type="exempla"]
let $source := concat("db/apps/thema/data/",
$doc/data(#xml:id),".xml")
let $target := concat("db/apps/thema/data/",
$doc/replace(#corresp,"#",""),
"/",
$doc/data(#xml:id),".xml")
return file:move($source, $target)
...returns false i.e. no files are moved.
What am I missing in the from/to paths?
Many thanks.
Related
Update:
add my Cargo.toml in case needed:
serde = { version = "1.0.117", default-features = false }
serde_json = "1.0.66"
sql-builder = "3.1"
sqlite = "0.26.0"
Update:
i did #cdhowie says on comment, add println!("{:?}", row[2].kind());, it's print String.
it's the way i store blob wrong, shouldn't use serde_json::to_string or something?
OP:
i writed a demo to learn sqlite and rust, here the code:
use std::{fs::File, io::{Read, Write}};
use sql_builder::{quote, SqlBuilder};
use sqlite::Connection;
fn main() {
// create sqlite databases on ./tmp/sqlite.db
let conn = Connection::open("./tmp/sqlite.db").unwrap();
// create table
conn.execute(
"CREATE TABLE IF NOT EXISTS icon (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT,
content BLOB,
used STRING
)",
)
.unwrap();
// read image file from disk and store to sqlite as blob
let mut file = File::open("./tmp/in.jpg").unwrap();
let mut contents = Vec::new();
file.read_to_end(&mut contents).unwrap();
println!("{:?}", contents);
// build sql query
let sql = SqlBuilder::insert_into("icon")
.fields(&["name", "content", "used"])
.values(&[
quote(serde_json::to_string("in").unwrap()),
quote(serde_json::to_string(&contents).unwrap()),
quote(serde_json::to_string("1").unwrap()),
])
.sql().unwrap();
// execute query
conn.execute(&sql).unwrap();
// read image file from sqlite and store to disk
let mut builder = SqlBuilder::select_from("icon");
builder.field("id");
builder.field("name");
builder.field("content");
builder.field("used");
let stmt = conn.prepare(&builder.sql().unwrap()).unwrap();
let mut cursor = stmt.into_cursor();
let row = cursor.next().unwrap().unwrap();
let id = row[0].as_integer().unwrap();
let name = row[1].as_string().unwrap();
let content = row[2].as_binary().unwrap(); // src/main.rs:51:38
let used = row[3].as_string().unwrap();
println!("{} {} {}", id, name, used);
let mut file = File::create("./tmp/out.jpg").unwrap();
file.write_all(content).unwrap();
}
when i run this code, will got error:
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/main.rs:51:38
but where i check the sqlite.db file, it shows:
seems file stored success. but how can i fix the code make read form sqlite and store to disk work?
if you guys need more info, please let me know. :)
I can fix the issue now, but this may not be the best solution.
// ...
let content = row[2].as_string().unwrap();
// ...
// ...
let content: Vec<u8> = serde_json::from_str(&content).unwrap();
file.write_all(&content).unwrap();
// ...
also, I may have found a better one SQL generation lib, called diesel.
Maybe as I keep learning, I might find a better, more elegant way. Anyway, it's working now. :)
<MasterData>
<Name>AA</Name>
<EmpId>123</EmpId>
<AccountNo>111</AccountNo>
<IFSC>ABC</IFSC>
<AccountData>
<AccountNo>111</AccountNo>
<IFSC>ABC</IFSC>
</AccountData>
<AccountData>
<AccountNo>222</AccountNo>
<IFSC>DEF</IFSC>
</AccountData>
</MasterData>
I have an xml like this in my database,I have a requirement to check the combination of AccountNo+IFSC present in the MasterData(not under the AccountData section) and compare with all documents present in the collection and check whether its matching to the data present in the AccountData section,If its matching identify the URI of the document.
First identify the unique combination of AccountNo+IFSC from Masterdata section and then check whether this combination present under any of the AccountData section, there are more elements in this xml other than AccountNo and IFSC
If you had range indexes on the AccountNo and IFSC elements, then you could:
retrieve the set of values from AccountNo, IFSC, and a cts:uri-reference() with cts:value-tuples().
create a map using a composite key with the AccountNo and IFSC values and the URIs as the values for those map entries
prune any entry that only has one URI associated
return the map that will have the set of URIs corresponding to each combo of AccountNo and IFSC value
Something like this:
let $accountNumber-IFSC :=
cts:value-tuples(
(
cts:element-reference(xs:QName("AccountNo")),
cts:element-reference(xs:QName("IFSC")),
cts:uri-reference()
)
)
let $map := map:new()
let $_create_map_value_to_uris := for $co-occurrence in $accountNumber-IFSC
let $key := concat($co-occurrence[1], " ", $co-occurrence[2])
let $value := (map:get($map, $key), $co-occurrence[3])
return map:put($map, $key, $value)
let $_prune_single_uri :=
for $key in map:keys($map)
let $value := map:get($map, $key)
where not(tail($value))
return
map:put($map, $key, ())
return
$map
If you just wanted the list of URIs, you can invert the map: -$map and return it's keys: return map:keys(-$map)
If you had a range-index on the EmpId you could pivot on that instead of the document URIs.
Using the Optic API functions, you can do something similar with element-range indexes:
import module namespace op = "http://marklogic.com/optic" at "/MarkLogic/optic.xqy";
op:from-lexicons(
map:entry("AccountNo", cts:element-reference(xs:QName("AccountNo")))
=> map:with("IFSC", cts:element-reference(xs:QName("IFSC")))
=> map:with("URI", cts:uri-reference())
)
=> op:group-by(
("IFSC", "AccountNo"),
(
op:group-concat("URIs", "URI", map:entry("separator", ", ")),
op:count("count", op:col("URI"))
)
)
=> op:where(op:gt(op:col("count"), 1))
=> op:result()
I have a csv file. I've managed import these data into MarkLogic using mlcp which then created a xml file in MarkLogic.
Now in csv I have this format "6/29/2013 5:00:00 PM" random in one of the column. How do I use xquery and probably node-replace as a transform function to convert this date into a different format such as "2013-06-29" as MarkLogic default date format?
Any help is appreciated...
I have created transform.xqy and install it on Modules in MLogic. I'm
thinking about using "xdmp:node-replace" to replace the date with expected
format. Or should I go thorugh the csv column by column (How to do?) and
use "castable as xs:dateTime" to determine date value or not. Yet, even
just printing out the content value/uri, always giving me error.
xquery version "1.0-ml";
module namespace example = "http://test.com/example";
(: If the input document is XML, insert #NEWATTR, with the value
: specified in the input parameter. If the input document is not
: XML, leave it as-is.
:)
declare function example:transform(
$content as map:map,
$context as map:map
) as map:map*
{
let $the-doc-uri := map:get($content, "uri")
let $the-doc := map:get($content, "value")
return
trace($the-doc, 'The value of doc is: ')
};
The MarkLogic documentation contains a full example of an MLCP transform:
https://docs.marklogic.com/guide/mlcp/import#id_65640
It shows this example, which adds an attribute to the XML content:
declare function example:transform(
$content as map:map,
$context as map:map
) as map:map*
{
let $attr-value :=
(map:get($context, "transform_param"), "UNDEFINED")[1]
let $the-doc := map:get($content, "value")
return
if (fn:empty($the-doc/element()))
then $content
else
let $root := $the-doc/*
return (
map:put($content, "value",
document {
$root/preceding-sibling::node(),
element {fn:name($root)} {
attribute { fn:QName("", "NEWATTR") } {$attr-value},
$root/#*,
$root/node()
},
$root/following-sibling::node()
}
), $content
)
};
Keep in mind you are supposed to update the "value" property of the $content map:map, and return $content to get your transformation result added to the database. I suggest using a (potentially recursive) typeswitch to identify element nodes, and then adjusting their value accordingly..
HTH!
finally did it.
The thing is I must use mem:node-replace because it is on the fly, on memory. While xdmp:node-replace is when the data is already on MarkLogic.
The rest is as expected I must use format-date and xdmp:parse-dateTime to get date format as required.
Here is some snippets
xquery version "1.0-ml";
module namespace ns_transform = "this_is_my_namespace";
import module namespace mem = "http://xqdev.com/in-mem-update" at "/MarkLogic/appservices/utils/in-mem-update.xqy";
declare variable $ns := "this_is_my_namespace";
declare function ns_transform:transform(
$content as map:map,
$context as map:map
) as map:map*
{
let $doc := map:get($content, "value")
let $format_in := "[M]/[D]/[Y0001] [h01]:[m01]:[s01] [P]"
let $format_out := "[Y0001]-[M01]-[D01]"
let $old_date := $doc/*:root_doc/*:date/text()
let $new_date := format-date(xs:date(xdmp:parse-dateTime($format_in, $old_date)), $format_out)
let $new_doc := mem:node-replace($doc/*:root_doc/*:date,element {fn:QName($ns, "date")}{$new_date})
let $_ := map:put($content, "value", $new_doc)
return $content
};
I have tried the below mentioned xquery. If document is not managed I want to manage the document using a DLS query otherwise I want to checkout the document.
xquery version "1.0-ml";
import module namespace dls = "http://marklogic.com/xdmp/dls" at "/MarkLogic/dls.xqy";
let $uri :="/root/189966_01.xml"
let $i := dls:document-is-managed($uri)
return
if ($i = fn:false())
then
dls:document-manage($uri, fn:false(), "Baz is now a managed document")
(: dls:document-checkout($uri, fn:true(), "updating doc", 3600) :)
else if ($i = fn:true())
then
dls:document-checkout($uri, fn:true(), "updating doc", 3600)
else
"No action"
please correct me if anything is wrong on my side.
The xquery looks fine. Just a restructuring of the above query -
xquery version "1.0-ml";
import module namespace dls = "http://marklogic.com/xdmp/dls" at "/MarkLogic/dls.xqy";
let $uri :="/root/189966_01.xml"
let $i := dls:document-is-managed($uri)
return
if ($i = fn:false()) then
dls:document-manage($uri, fn:false(), "Baz is now a managed document")
else
if ($i = fn:true()) then
dls:document-checkout($uri, fn:true(), "updating doc", 3600)
else
()
you cannot checkout and manage in the same transaction. You can checkout in 2 independent transactions in the same request, managed by a parent transaction (as long as you do not hold any locks before or during.
I am using Swift in a project, and using SQLite.swift for database handling. I am trying to retrieve the most recent entry from my database like below:
func returnLatestEmailAddressFromEmailsTable() -> String{
let dbPath = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true).first as String
let db = Database("\(dbPath)/db.sqlite3")
let emails = db["emails"]
let email = Expression<String>("email")
let time = Expression<Int>("time")
var returnEmail:String = ""
for res in emails.limit(1).order(time.desc) {
returnEmail = res[email]
println("from inside: \(returnEmail)")
}
return returnEmail
}
I am trying to test the returned string from the above function like this:
println("from outside: \(returnLatestEmailAddressFromEmailsTable())")
Note how I print the value from both inside and outside of the function. Inside, it works every single time. I am struggling with the "from outside:" part.
Sometimes the function returns the correct email, but sometimes it returns "" (presumably, the value was not set in the for loop).
How can I add "blocking" functionality so calling returnLatestEmailAddressFromEmailsTable() will always first evaluate the for loop, and only after this return the value?