I need to pass an argument into an xdmp:eval with xquery:
let $uri :="/tmp/myDoc.csv"
let $doc:= xdmp:eval('fn:doc($uri)' , (), <options xmlns="xdmp:eval"><database>{xdmp:database("My-Database")}</database></options>)
return $doc
But i'm getting Undefined variable $uri
I need to do this with an xdmp:eval for many reasons, have any one any idea how to do this in xquery ?
When you eval that string, it has no context to know that the $uri value should be.
You can pass those context values in the second parameter when you invoke:
let $uri :="/tmp/myDoc.csv"
let $doc:= xdmp:eval('fn:doc($uri)',
(xs:QName("uri"), $uri),
<options xmlns="xdmp:eval">
<database>{xdmp:database("My-Database")}</database>
</options>)
return $doc
But you should consider using xdmp:invoke-function() instead, with the anonymous function:
let $uri := "/tmp/myDoc.csv"
xdmp:invoke-function(function(){ fn:doc($uri) },
<options xmlns="xdmp:eval">
<database>{xdmp:database("My-Database")}</database>
</options>
)
It is generally easier and safer to use xdmp:invoke-function.
Related
I am using below code to copy documents related to specified collection from one database to another database.
In my try block I am returning to delete all the documents after successful copy. And in the catch just to print the exception .
But it gives me a syntax error which I am not able to figure out. Can someone please help me in this -
xquery version "1.0-ml";
for $doc in cts:search(doc(), cts:collection-query('Test'))
let $docuri := $doc/base-uri()
let $collections := xdmp:document-get-collections($docuri)
let $permissions := xdmp:document-get-permissions($docuri)
let $document-insert-options :=
<options xmlns="xdmp:document-insert">
<permissions>{$permissions}</permissions>z
<collections>{
<collection>{$collections}</collection>
}
</collections>
</options>
let $db-name := "Final"
let $invoke-function-options :=
<options xmlns="xdmp:eval">
<database>{ xdmp:database($db-name) }</database>
<commit>auto</commit>
</options>
return
try {
xdmp:invoke-function(
function(){ xdmp:document-insert($docuri, $doc, $document-insert-options)},
$invoke-function-options
);
return xdmp:document-delete($doc)
}
catch ($exception) {
"Problem loading file, received the following exception: ",
$exception }
Michael and Rob both have good points, but there are a few more issues.
MarkLogic does complain about the extra 'z' in the $document-insert-options. Get rid of that.
MarkLogic will also complain about the fact you are passing in $doc instead of $docuri into xdmp:document-delete. So change that as well.
Last is the semi-colon bit. MarkLogic does support that, but only on top-level. Using Michael suggestion, insert and delete might happen in parallel. Using Rob's suggestion of an extra 'let $_ :=', and removing ';' will cause MarkLogic to first insert, and delete second. I think that is what you are after.
Running the following against a random database from QC ('Fab' will do), works fine on my end:
xquery version "1.0-ml";
xdmp:document-insert('/test.json', object-node{"foo":"bar"}, map:entry("collections", "Test"))
;
xquery version "1.0-ml";
for $doc in cts:search(doc(), cts:collection-query('Test'))
let $docuri := $doc/base-uri()
let $collections := xdmp:document-get-collections($docuri)
let $permissions := xdmp:document-get-permissions($docuri)
let $document-insert-options :=
<options xmlns="xdmp:document-insert">
<permissions>{$permissions}</permissions>
<collections>{
<collection>{$collections}</collection>
}
</collections>
</options>
let $db-name := "Documents"
let $invoke-function-options :=
<options xmlns="xdmp:eval">
<database>{ xdmp:database($db-name) }</database>
<commit>auto</commit>
</options>
return
try {
let $_ := xdmp:invoke-function(
function(){ xdmp:document-insert($docuri, $doc, $document-insert-options)},
$invoke-function-options
)
return xdmp:document-delete($docuri)
}
catch ($exception) {
"Problem loading file, received the following exception: ",
$exception }
HTH!
What is the error?
Offhand this block looks wrong:
let $document-insert-options :=
<options xmlns="xdmp:document-insert">
<permissions>{$permissions}</permissions>z
<collections>{
<collection>{$collections}</collection>
}
</collections>
</options>
You have a z floating out there after </permissions> which could be breaking schema validation of the options node.
This also could be wrong:
catch ($exception) {
"Problem loading file, received the following exception: ",
$exception }
You may need to explicitly return a Sequence like so:
catch ($exception) {
("Problem loading file, received the following exception: ",
$exception)}
This also seems odd to me:
try {
xdmp:invoke-function(
function(){ xdmp:document-insert($docuri, $doc, $document-insert-
options)},
$invoke-function-options
);
return xdmp:document-delete($doc)
}
This breaks FLWOR by trying to use a multi-statement transaction with the ; .
Instead try:
try {
let $_ := xdmp:invoke-function(
function(){ xdmp:document-insert($docuri, $doc, $document-insert-
options)},
$invoke-function-options
)
return xdmp:document-delete($doc)
}
XQuery doesn't have semicolon-delimited statements, nor does it have any instruction or expression that starts with the keyword "return".
xdmp:invoke-function(
function(){ xdmp:document-insert($docuri, $doc, $document-insert-options)},
$invoke-function-options
);
return xdmp:document-delete($doc)
}
catch ($exception) {
"Problem loading file, received the following exception: ",
$exception }
Replace the "; return" by a comma (",").
Marklogic may also have special rules for invoking functions with side-effects; I wouldn't know about that area.
i've function for dynamic parameter. Im using MicroStrategy tool for this.
let $uri := replace('http://url/suggest?searchTerm=DYNAMIC PROMPT','\s+','%20')
i want to replace 'ü' to 'u' for in this function. I already use ('\s+','%20') for space character.
Multiple replace functions can be nested:
let $string := 'http://url/suggest?searchTerm=DüNAMIC PROMPT'
let $uri := replace(replace($string, '\s+','%20'), 'ü', 'u')
return $uri
For replacing single characters, you can also use the translate function:
let $string := 'http://url/suggest?searchTerm=DüNAMIC PROMPT'
let $uri := translate($string, ' üÜöÖäÄ','+uUoOaA')
return $uri
Can i use "let $uri := replace(replace($string, '\s+','%20'), 'ü', 'u')" this for multiple characters.
For example: let $uri := replace(replace($string, '\s+','%20'),'üÜöÖäÄ','+uUoOaA')
Because i need two replace case.
Microstrategy doesn't support this case. For example i gave 'Müşteri' keyword that gaves me different answers :(
I agree with Leo's suggestion that fn:encode-for-uri() is the way to go for the stated use case - i.e., for handling spaces, special characters, and diacritics when constructing a URL.
But if you are interested in stripping diacritics from a string, here's how you'd do that:
xquery version "3.1";
"Müşteri"
=> normalize-unicode("NFD")
=> replace("\p{IsCombiningDiacriticalMarks}", "")
This returns Musteri.
For reference, this is based on the Gist I posted at https://gist.github.com/joewiz/af04074c28e0ae2a1b92.
I have to copy an entire project folder inside the MarkLogic server and instead of doing it manually I decided to do it with a recursive function, but is becoming the worst idea I have ever had. I'm having problems with the transactions and with the syntax but being new I don't find a true way to solve it. Here's my code, thank you for the help!
import module namespace dls = "http://marklogic.com/xdmp/dls" at "/MarkLogic/dls.xqy";
declare option xdmp:set-transaction-mode "update";
declare function local:recursive-copy($filesystem as xs:string, $uri as xs:string)
{
for $e in xdmp:filesystem-directory($filesystem)/dir:entry
return
if($e/dir:type/text() = "file")
then dls:document-insert-and-manage($e/dir:filename, fn:false(), $e/dir:pathname)
else
(
xdmp:directory-create(concat(concat($uri, data($e/dir:filename)), "/")),
local:recursive-copy($e/dir:pathname, $uri)
)
};
let $filesystemfolder := 'C:\Users\WB523152\Downloads\expath-ml-console-0.4.0\src'
let $uri := "/expath_console/"
return local:recursive-copy($filesystemfolder, $uri)
MLCP would have been nice to use. However, here is my version:
declare option xdmp:set-transaction-mode "update";
declare variable $prefix-replace := ('C:/', '/expath_console/');
declare function local:recursive-copy($filesystem as xs:string){
for $e in xdmp:filesystem-directory($filesystem)/dir:entry
return
if($e/dir:type/text() = "file")
then
let $source := $e/dir:pathname/text()
let $dest := fn:replace($source, $prefix-replace[1], $prefix-replace[2])
let $_ := xdmp:document-insert($source,
<options xmlns="xdmp:document-load">
<uri>{$dest}</uri>
</options>)
return <record>
<from>{$source}</from>
<to>{$dest}</to>
</record>
else
local:recursive-copy($e/dir:pathname)
};
let $filesystemfolder := 'C:\Temp'
return <results>{local:recursive-copy($filesystemfolder)}</results>
Please note the following:
I changed my sample to the C:\Temp dir
The output is XML only because by convention I try to do this in case I want to analyze results. It is actually how I found the error related to conflicting updates.
I chose to define a simple prefix replace on the URIs
I saw no need for DLS in your description
I saw no need for the explicit creation of directories in your use case
The reason you were getting conflicting updates because you were using just the filename as the URI. Across the whole directory structure, these names were not unique - hence the conflicting update on double inserts of same URI.
This is not solid code:
You would have to ensure that a URI is valid. Not all filesystem paths/names are OK for a URI, so you would want to test for this and escape chars if needed.
Large filesystems would time-out, so spawning in batches may be useful.
A an example, I might gather the list of docs as in my XML and then process that list by spawning a new task for every 100 documents. This could be accomplished by a simple loop over xdmp:spawn-function or using a library such as taskbot by #mblakele
i try to split my incoming documents using "Information Studio Flows" (MarkLogic v 8.0-1.1). The problem is in "Transform" section.
This is my importing documents. For simplicity i reduce it content to one stwtext-element
<docs>
<stwtext id="RD-10-00258" update="03.2011" seq="RQ-10-00001">
<head>
<ti>
<i>j</i>
</ti>
<ff-list>
<ff id="0103"/>
</ff-list>
</head><p>
Symbol für die
<vw idref="RD-19-04447">Stromdichte</vw>
.
</p>
</stwtext>
</docs>
This is my "xquery transform" content:
xquery version "1.0-ml";
(: Copyright 2002-2015 MarkLogic Corporation. All Rights Reserved. :)
(:
:: Custom action. It must be a CPF action module.
:: Replace this text completely, or use it as a template and
:: add imports, declarations,
:: and code between START and END comment tags.
:: Uses the external variables:
:: $cpf:document-uri: The document being processed
:: $cpf:transition: The transition being executed
:)
import module namespace cpf = "http://marklogic.com/cpf"
at "/MarkLogic/cpf/cpf.xqy";
(: START custom imports and declarations; imports must be in Modules/ on filesystem :)
(: END custom imports and declarations :)
declare option xdmp:mapping "false";
declare variable $cpf:document-uri as xs:string external;
declare variable $cpf:transition as node() external;
if ( cpf:check-transition($cpf:document-uri,$cpf:transition))
then
try {
(: START your custom XQuery here :)
let $doc := fn:doc($cpf:document-uri)
return
xdmp:eval(
for $wpt in fn:doc($doc)//stwtext
return
xdmp:document-insert(
fn:concat("/rom-data/", fn:concat($wpt/#id,".xml")),
$wpt
)
)
(: END your custom XQuery here :)
,
cpf:success( $cpf:document-uri, $cpf:transition, () )
}
catch ($e) {
cpf:failure( $cpf:document-uri, $cpf:transition, $e, () )
}
else ()
by running of snippet, i take the error:
Invalid URI format
and long description of it:
XDMP-URI: (err:FODC0005) fn:doc(fn:doc("/8122584828241226495/12835482492021535301/URI=/content/home/admin/Vorlagen/testing/v10.new-ML.xml")) -- Invalid URI format: "
j
Symbol für die
Stromdichte
"
In /18200382103958065126.xqy on line 37
In xdmp:invoke("/18200382103958065126.xqy", (xs:QName("trgr:uri"), "/8122584828241226495/12835482492021535301/URI=/content/home/admi...", xs:QName("trgr:trigger"), ...), <options xmlns="xdmp:eval"><isolation>different-transaction</isolation><prevent-deadlocks>t...</options>)
$doc = fn:doc("/8122584828241226495/12835482492021535301/URI=/content/home/admin/Vorlagen/testing/v10.new-ML.xml")
In /MarkLogic/cpf/triggers/internal-cpf.xqy on line 179
In execute-action("on-state-enter", "http://marklogic.com/states/initial", "/8122584828241226495/12835482492021535301/URI=/content/home/admi...", (xs:QName("trgr:uri"), "/8122584828241226495/12835482492021535301/URI=/content/home/admi...", xs:QName("trgr:trigger"), ...), <options xmlns="xdmp:eval"><isolation>different-transaction</isolation><prevent-deadlocks>t...</options>, (fn:doc("http://marklogic.com/cpf/pipelines/14379829270688061297.xml")/p:pipeline, fn:doc("http://marklogic.com/cpf/pipelines/15861601524191348323.xml")/p:pipeline), fn:doc("http://marklogic.com/cpf/pipelines/15861601524191348323.xml")/p:pipeline/p:state-transition[1]/p:default-action, fn:doc("http://marklogic.com/cpf/pipelines/15861601524191348323.xml")/p:pipeline/p:state-transition[1])
$caller = "on-state-enter"
$state-or-status = "http://marklogic.com/states/initial"
$uri = "/8122584828241226495/12835482492021535301/URI=/content/home/admi..."
$vars = (xs:QName("trgr:uri"), "/8122584828241226495/12835482492021535301/URI=/content/home/admi...", xs:QName("trgr:trigger"), ...)
$invoke-options = <options xmlns="xdmp:eval"><isolation>different-transaction</isolation><prevent-deadlocks>t...</options>
$pipelines = (fn:doc("http://marklogic.com/cpf/pipelines/14379829270688061297.xml")/p:pipeline, fn:doc("http://marklogic.com/cpf/pipelines/15861601524191348323.xml")/p:pipeline)
$action-to-execute = fn:doc("http://marklogic.com/cpf/pipelines/15861601524191348323.xml")/p:pipeline/p:state-transition[1]/p:default-action
$chosen-transition = fn:doc("http://marklogic.com/cpf/pipelines/15861601524191348323.xml")/p:pipeline/p:state-transition[1]
$raw-module-name = "/18200382103958065126.xqy"
$module-kind = "xquery"
$module-name = "/18200382103958065126.xqy"
In /MarkLogic/cpf/triggers/internal-cpf.xqy on line 320
i thought, it was a problem with "Document setting" in "load" section of "Flow editor"
URI=/content{$path}/{$filename}{$dot-ext}
but if i remove it, i recive the same error.
i have no idea what to do. i am really new. please help
First of all, Information Studio has been deprecated in MarkLogic 8. I would also recommend very much looking in to the aggregate_record feature of MarkLogic Content Pump:
http://docs.marklogic.com/guide/ingestion/content-pump#id_65814
Apart from that, there are several issues with your code. You are calling fn:doc twice, effectively trying to interpret the doc contents as a uri. There is an unnecessary xdmp:eval wrapping the FLWOR statement, which expects a string as first param. I think you can shorten it to (showing inner part of the action only):
(: START your custom XQuery here :)
let $doc := fn:doc($cpf:document-uri)
for $wpt in $doc//stwtext
return
xdmp:document-insert(
fn:concat("/roempp-data/", fn:concat($wpt/#id,".xml")),
$wpt
)
(: END your custom XQuery here :)
HTH!
very many thanks #grtjn and this is my approach. Practically it is the same solution
(: START your custom XQuery here :)
xdmp:log(fn:doc($cpf:document-uri), "debug"),
let $doc := fn:doc($cpf:document-uri)
return
xdmp:eval('
declare variable $doc external;
for $wpt in $doc//stwtext
return (
xdmp:document-insert(
fn:concat("/roempp-data/", fn:concat($wpt/#id,".xml")),
$wpt,
xdmp:default-permissions(),
"roempp-data"
)
)'
,
(xs:QName("doc"), $doc),
<options xmlns="xdmp:eval">
<database>{xdmp:database("roempp-tutorial")}</database>
</options>
)
(: END your custom XQuery here :)
Ok, now it works. It is fine, but i found, that after the loading is over, i see in MarkLogic two documents:
my splited document "/rom-data/RD-10-00258.xml" with one root element "stwtext" (as desired)
origin document "URI=/content/home/admin/Vorlagen/testing/v10.new-ML.xml" with root element "docs"
is it possible to prohibit insert of origin document ?
I have the following module
xquery version "1.0-ml";
declare variable $datasets as map:map external;
...
I want to call this so I do an xdmp:invoke like this
...
let $update := xdmp:invoke("/app/lib/my-module.xqy", (xs:QName("datasets"), $map), <options xmlns="xdmp:eval">
<modules>{xdmp:modules-database()}</modules>
</options>)
...
$map is of type map:map.
Running this gives me the following error
[1.0-ml] XDMP-ARG: xdmp:invoke("/app/lib/20140527-0916-copy-y-axis-labels-from-chart-to-dataset-...", fn:QName("", "datasets"), <options xmlns="xdmp:eval"><modules>0</modules></options>) -- Invalid argument
Why is that?
It's hard to know for sure from the limited code samples you've posted, but I think that your $map variable is bound to the empty sequence (the rough analogue of null in XPath/XQuery).
I've created the following main module, that simply returns the external variable $datasets:
xquery version "1.0-ml";
declare variable $datasets as map:map external;
$datasets
Invoking it as follows works correctly:
let $map := map:entry("key", "value")
return
xdmp:invoke("/test.xqy",
(xs:QName("datasets"), $map),
<options xmlns="xdmp:eval">
<modules>{xdmp:modules-database()}</modules>
</options>)
This results in the "invalid argument" error:
let $map := ()
return
xdmp:invoke("/test.xqy",
(xs:QName("datasets"), $map),
<options xmlns="xdmp:eval">
<modules>{xdmp:modules-database()}</modules>
</options>)
XQuery flattens sequences (they don't nest like s-expressions), so when $map is the empty sequence, the <options/> element becomes the value of the $datasets param, which is an invalid value for the external variable.
Update: Doh! #mblakele's comments below explain the error conditions ...
TL;DR: () is not an instance of map:map.