So my query was working fine and I now need to verify the resulting .xml with a .dtd validation. My .xq looked like this before:
< root>
...
...
< /root>
Now it looks like this:
< !DOCTYPE root SYSTEM 'validation.dtd'>
< root>
...
...
< /root>
Running the .xq now, however, throws the following error:
XPST0003 XQuery syntax error near #...as xs:integer external; < !D#:
Expected '--' or '[CDATA[' after '< !'
Static error(s) in query
I don't know what this error means, and I'm unable to find how to fix it
Thanks in advance
To serialize an XML document with a document type declaration, use the fn:serialize() function with the doctype-system parameter:
xquery version "3.1";
fn:serialize(<root/>, map { "doctype-system": "validation.dtd" })
This produces the following string:
<!DOCTYPE root SYSTEM "validation.dtd">
<root/>
For more on this technique, see the function documentation for fn:serialize() at https://www.w3.org/TR/xpath-functions-31/#func-serialize and description of the doctype-system and doctype-public parameters in the XSLT and XQuery Serialization 3.1 Specification at https://www.w3.org/TR/xslt-xquery-serialization-31/#XML_DOCTYPE.
For processors that only support XPath 3.0 or that have not yet implemented the map(*) method of specifying serialization parameters, you can use this form:
xquery version "3.0";
declare namespace output="http://www.w3.org/2010/xslt-xquery-serialization";
fn:serialize(
<root/>,
<output:serialization-parameters>
<output:doctype-system value="validation.dtd"/>
</output:serialization-parameters>
)
I like to write library modules that can be executed simply by calling the xq file. However, these also contain functions that I would like to test. Something like this some.xql:
xquery version "3.0";
import module namespace xmldb="http://exist-db.org/xquery/xmldb";
declare namespace no="http://none";
declare namespace test="http://exist-db.org/xquery/xqsuite";
declare
%test:arg('1')
%test:assertEquals('2')
function no:something ($num as xs:string?) as xs:string {
return
$num + 1
};
xmldb:store('/db/data/', 'two.xml',<root>{no:something(1)}</root>)
However I cannot test the whole module or the no:something function from within it. I have no problem accessing the function in other context using:
import module namespace no="http://none" at "some.xql";
Yet, when trying to run test suite from a wrapper function I keep getting xpty00004 errors:
xquery version "3.0";
import module namespace test="http://exist-db.org/xquery/xqsuite" at "resource:org/exist/xquery/lib/xqsuite/xqsuite.xql";
test:suite(
inspect:module-functions(xs:anyURI("some.xql"))
)
I have tried different variation of getting to the no:some function, but no lock. Am I just writing really bad queries, using the xqsuite wrong, or is this a bug?
Your some.xql is a main module, you can only import and test functions in library modules.
Consider instead refactoring to a library module like no.xqm:
xquery version "3.0";
module namespace no="http://none";
declare namespace test="http://exist-db.org/xquery/xqsuite";
declare
%test:arg('1')
%test:assertEquals('2')
function no:something ($num as xs:string?) as xs:string {
$num + 1
};
Your app main module some.xq:
xquery version "3.0";
import module namespace no="http://none" at "no.xqm";
xmldb:store('/db/data/', 'two.xml',<root>{no:something(1)}</root>
Your test runner main module tests.xq:
xquery version "3.0";
import module namespace test="http://exist-db.org/xquery/xqsuite"
at "resource:org/exist/xquery/lib/xqsuite/xqsuite.xql";
test:suite(
inspect:module-functions(xs:anyURI("no.xqm"))
)
I am using Roxy to manage my project. Also using MarkLogic 8.0-6.1
I am trying to submit a searchTerm, and return a custom formatted search:snippet
Here are the complete steps that I am taking:
./../roxy/ml new test-app --server-version=8 --app-type=rest
Configure my build.properties
cd test-app/
./ml local bootstrap
Now I have my project Structure.
Create File - test-app/rest-api/ext/show-search.xqy
xquery version "1.0-ml";
module namespace ss = "http://marklogic.com/rest-api/resource/show-search";
import module namespace search = "http://marklogic.com/appservices/search" at "/MarkLogic/appservices/search/search.xqy";
import module namespace json = "http://marklogic.com/xdmp/json" at "/MarkLogic/json/json.xqy";
declare
function ss:get(
$context as map:map,
$params as map:map
) as document-node()*
{
map:put($context, "output-types", "application/json"),
map:put($context, "output-status", (200, "OK")),
let $search-term := map:get($params, "searchTerm")
let $query := search:search($search-term,
<options xmlns="http://marklogic.com/appservices/search">
<transform-results apply="raw"/>
</options>
)
return document {$query}
};
(:
:)
declare
function ss:put(
$context as map:map,
$params as map:map,
$input as document-node()*
) as document-node()?
{
map:put($context, "output-types", "application/xml"),
map:put($context, "output-status", (201, "Created")),
document { "PUT called on the ext service extension" }
};
(:
:)
declare
function ss:post(
$context as map:map,
$params as map:map,
$input as document-node()*
) as document-node()*
{
map:put($context, "output-types", "application/xml"),
map:put($context, "output-status", (201, "Created")),
document { "POST called on the ext service extension" }
};
(:
:)
declare
function ss:delete(
$context as map:map,
$params as map:map
) as document-node()?
{
map:put($context, "output-types", "application/xml"),
map:put($context, "output-status", (200, "OK")),
document { "DELETE called on the ext service extension" }
};
The above GET request uses the transform-results apply=raw option, deploys, and functions properly (I have some test documents).
However I do not want to return the whole document, I want to return a whole section of the json that had a match, no matter where in that seciton the match happened (lower levels)
So I try to write my own snipper
Create File - test-app/rest-api/ext/show-search-snipper.xqy
xquery version "1.0-ml";
module namespace sss = "http://marklogic.com/rest-api/resource/show-search-snipper";
import module namespace search = "http://marklogic.com/appservices/search" at "/MarkLogic/appservices/search/search.xqy";
import module namespace json = "http://marklogic.com/xdmp/json" at "/MarkLogic/json/json.xqy";
declare
function sss:my-snippet(
$result as node(),
$ctsquery as schema-element(cts:query),
$options as element(search:transform-results)?
) as element(search:snippet)
{
<search:snippet>
</search:snippet>
};
I then update the search:search call to the following
let $query := search:search($search-term,
<options xmlns="http://marklogic.com/appservices/search">
<transform-results apply="my-snippet" ns="http://marklogic.com/rest-api/resource/show-search-snipper" at="show-search-snipper.xqy"/>
</options>
)
Now I should have everything I need (I think)
I run the deploy ./ml local deploy rest
and get the following
Minty-linux test-app # ./ml local deploy rest Loading REST properties
in /opt/this-is-a-test/test-app/rest-api/config/properties.xml Loading
REST options in /opt/this-is-a-test/test-app/rest-api/config/options
Loading REST extensions from /opt/this-is-a-test/test-app/rest-api/ext
ERROR: 400 "Bad Request" ERROR: {"errorResponse":{"statusCode":400,
"status":"Bad Request", "messageCode":"RESTAPI-INVALIDCONTENT",
"message":"RESTAPI-INVALIDCONTENT: (err:FOER0000) Invalid content:
invalid show-search-snipper extension: show-search-snipper either is
not a valid module or does not provide extension functions (delete,
get, put, post) in the
http://marklogic.com/rest-api/resource/show-search-snipper
namespace"}}
So I tried moving the show-search-snipper.xqy file up 1 level (to test-app/rest-api/show-search-snipper.xqy`
Run the deployment
Deployment Works No errors
Hit the URL and receive the following
500 Internal Server Error
INTERNAL ERROR
RESTAPI-INVALIDREQ: (err:FOER0000) Invalid request: reason: Extension
show-search does not exist. . See the MarkLogic server error log for
further detail.
Although I know the extension was created properly, since it worked fine before the introduction of the custom snip function. (with apply="raw")
Any thoughts how I can apply my custom snip function or what I am doing wrong in deployment?
Should you decide to stick with a custom snippeter:
It looks like Roxy is trying to treat it your snippeter module as a resource extension, which it is not. Your snippeter should be just a vanilla module in the modules db.
IDK how to configure Roxy, unfortunately, but what you're aiming for is to get Roxy to either install it using PUT /v1/ext/directories/asset or a straight up insert (`PUT /v1/documents) on your modules db. See http://docs.marklogic.com/REST/PUT/v1/ext/[directories]/[asset].
Assuming Roxy uses /ext, then the path to your snippeter would NOT be the unqualified path you have in your options. It would be an absolute path rooted at /ext/. See http://docs.marklogic.com/guide/rest-dev/search#id_72390.
There's a simpler way -- you can do this with the extract-document-data search option.
I set up some sample data to work with like this:
xquery version "1.0-ml";
for $i in (1 to 10)
return
xdmp:document-insert(
'/test-content-' || $i || '.json',
xdmp:unquote('{ "important": { "foo": 1, "bar": 2 }, "not-important": { "stuff": 3, "blah": 4 } }')
)
Having bootstrapped and deployed modules, I can get search results at http://localhost:8040/v1/search. However, I can start taking more control of the search results by using stored search options. Take a look in your project at rest-api/config/options/all.xml. They were already deployed for you when you run ml local deploy modules, so you now can search using http://localhost:8040/v1/search?options=all. Since you're using JSON data, I took it one step further and ran with: http://localhost:8040/v1/search?format=json&options=all.
I added this to rest-api/config/options/all.xml:
<extract-document-data selected="include">
<extract-path>/important</extract-path>
</extract-document-data>
This tells the search options to include the specified path with all search results. After deploying and running the search again, one result looks like this:
{
"index":1,
"uri":"/test-content-6.json",
"path":"fn:doc(\"/test-content-6.json\")",
"score":0,
"confidence":0,
"fitness":0,
"href":"/v1/documents?uri=%2Ftest-content-6.json",
"mimetype":"application/json",
"format":"json",
"matches":[{"path":"fn:doc(\"/test-content-6.json\")/object-node()", "match-text":[]}],
"extracted":{
"kind":"array",
"content":[
{"important":{"foo":1, "bar":2}}
]
}
},
Notice the "extracted" part at the end there -- I get the "important" JSON property, as specified in the options.
For a full list of options you can set to control search, see the Query Options Reference appendix.
How to set external environment variable as array?
If I have environment variable
SYMFONY__NSQLOOKUPD__HOSTS=["localhost:4161"]
and in config.yml:
socloz_nsq:
lookupd_hosts: %nsqlookupd.hosts%
Then I got an error:
Invalid type for path "socloz_nsq.lookupd_hosts". Expected array, but got string
I've found solution. Here it is:
in config.yml add to the imports section:
imports:
- { resource: parameters.php }
then create parameters.php file at the same directory where config.yml exists, and look at the following example:
<?php
$nsqlookupdhosts = getenv('SYMFONY__NSQLOOKUPD__HOSTS');
$nsqdhosts = getenv('SYMFONY__NSQD__HOSTS');
$container->setParameter('nsqlookupd.hosts.parsed', explode(',', $nsqlookupdhosts));
$container->setParameter('nsqd.hosts.parsed', explode(',', $nsqdhosts));
use comma as delimiter in environment variable (you are not restricted to comma, use any)
SYMFONY__NSQLOOKUPD__HOSTS=localhost:4161,some.server:2222
You can use a built-in "json" environment variable processor to decode a JSON string into an array:
SYMFONY__NSQLOOKUPD__HOSTS='["localhost:4161"]'
$nsqlookupdhosts: '%env(json:SYMFONY__NSQLOOKUPD__HOSTS)%'
I'm trying to use google closure compiler (I'll call it compiler for short) to validate JavaScript codebase like that:
java -jar compiler.jar --js='**.js' --jscomp_error newCheckTypes > NUL 2> gcc.log
The thing is we already have a lot of code in AMD and type annotations written for core modules like that:
define(function () {
/** #constructor */
MyClass () {}
/** #param {string} s */
MyClass.prototype.myMethod = function (s) {
alert(s);
}
return MyClass;
});
I'd like to reference the type /** #type {MyClass} */ in other modules, so if I call some method with wrong arguments, it will complaint.
Unfortunately, when I use such type annotations, compiler outputs:
Bad type annotation. Unknown type MyClass
/** #type {MyClass} */
If I strip out module-delcaring function, compiler "understands" my declaration, but I don't feel like rewriting our code in that way.
Is there a way to make compiler see my type declarations and apply in other modules?
The compiler has a --transform_amd_modules which will rewrite an AMD module to CommonJS. It only supports a limited number of define signatures and is not widely used or actively maintained.
With CommonJS modules, you can use the --process_common_js_modules flag to rewrite and rescope the code to a global type which can be checked by the compiler. The CommonJS modules functionality is actively maintained.