I have the following JSON Documents in a MarkLogic DB
I am attempting to write a custom search extension, so that I can return the full segment JSON, for which a match is found. The other Segments of the Array should not be returned, if the matches were not found within that segment.
With Help from a previous thread ( MarkLogic - Extending the Search, return specific object node for reference )
I learned about the <extract-document-data> option for the search:search module.
However, when I use that option, It returns me All Segments in the document, regardless of which segment actually had the matches.
Here is my code
declare
function ext:get(
$context as map:map,
$params as map:map
) as document-node()*
{
map:put($context, "output-types", "application/json"),
map:put($context, "output-status", (200, "OK")),
let $search-term := map:get($params, "searchTerm")
let $query := search:search($search-term,
<options xmlns="http://marklogic.com/appservices/search">
<extract-document-data>
<extract-path>/segments</extract-path>
</extract-document-data>
<return-facets>true</return-facets>
</options>
)
return document {$query}
};
Doing This However, extracts All Segments from the document, and not just the ones that have the matches. I have not been able to find subsequent options for "match only" type of modifier.
Is there anyway to only extract the "segment" of the match, and not all segments
Essentially I am trying to get the raw json of the highlighted part
Update: attempting custom snippet
declare
function ext:my-snippet(
$result as node(),
$ctsquery as schema-element(cts:query),
$options as element(search:transform-results)?
) as element(search:snippet)
{
element search:snippet {
$result
}
};
let $query := search:search($search-term,
<options xmlns="http://marklogic.com/appservices/search">
<transform-results apply="my-snippet" ns="ext" at="/show-search.xqy">
<max-snippet-chars>150</max-snippet-chars>
<per-match-tokens>20</per-match-tokens>
</transform-results>
</options>
)
Deployment for the above module works fine. All code is in same file called show-search.xqy However When I try to hit the URL, it returns
Invalid Request: reason: Extension show-search does not exist
If I go back to default snippet option, it works fine. Only fails with custom snippet.
ANy thoughts?
If you want to extract the properties that matched the query, you could take a look at snippeting:
http://docs.marklogic.com/guide/search-dev/search-api#id_65347
You may need to write a custom snippetter to do what you want:
http://docs.marklogic.com/guide/search-dev/query-options#id_61707
Hoping that helps,
I have personally often found it more straight-forward to use transforms instead of extract-document-data when my data model is complex or I want to have some specific logic to pare down my results to the only the data points I want.
A transform will handle your search result as a document and you can write some XQuery (or JavaScript) to navigate through, select the data points of interest, format a response, and return the parsed results.
http://docs.marklogic.com/guide/rest-dev/transforms
Related
I want to write the XQuery to print the specific keys in JSON and want to except if it has an array value.
Sample JSON:
{
"id":"743",
"transation":{
"101":"success",
"102":"rejected",
"301":"processing"
},
"groupName":"group1"
}
Expected Result:
id
groupName
Assuming that you are reading a JSON document from the database, you could iterate over the object property nodes and filter out the object/array properties by testing whether the property has child nodes:
for $property in doc("/test.json")/object-node()/node()
where not($property/node())
return $property/name()
Or you could exclude those that are instance of object-node() or array-node()
for $property in doc("/test.json")/object-node()/node()
where not($property instance of object-node() or $property instance of array-node())
return $property/name()
I'm not sure if marklogic follows the W3C spec here, but in XQuery 3.1 you can do
let $json := json-doc('my.json')
return map:keys($json)[not($json(.) instance of array(*))]
I've been struggling with this for a while but can't find a clean way to do it, so I'm seeking for some help.
I have custom filters (ApiPlatform 2.5 and Symfony 5.1) on database API outputs, and I need to filter on the current workflow place, or status as you like, of each output.
The Status has the below structure, which is a symfony workflow's place :
Status = { "OPEN": 1 }
My issue is that the status is stored as an array in the DB, and I can't find a way to have the querybuilder finding a match.
I've tried to build locally an array to do an = , a LIKE or an IN :
$status['OPEN'] = 1;
$queryBuilder->andWhere(sprintf('%s.Status = :st', $rootAlias))
->leftJoin(sprintf('%s.Objs', $rootAlias), 'o')
->andWhere('o.Profile = :p')
->setParameters(array(
'st' => $status,
'p' => $profile
));
But no way :(
I implemented a workaround that works but I don't like it as I'm using workflows a lot and need a clean way to filter outputs.
My workaround is fairly simple, when the status is writen as an array in the DB, I also store it as a string in another field called StatusText, then filtering on StatusText is easy and straight.
Status can have different contents obviously : OPEN, CLOSING, CLOSED, ...
Help appreciated !!
Thanks
EDIT & Solution
As proposed by Youssef, use scienta/doctrine-json-functions and use JSON_EXTRACT :
composer require scienta/doctrine-json-functions
Important, that was part of my issue, use the Doctrine type json_array an not array to store the status or the state, however you call it, in the Database.
Integrate the alias provided inside the ApiPlatform custom filter :
$rootAlias = $queryBuilder->getRootAliases()[0];
$json_extract_string = "JSON_EXTRACT(".$rootAlias.".Status, '$.OPEN') = 1";
$queryBuilder->andwhere($json_extract_string )
->leftJoin(sprintf('%s.Objs', $rootAlias), 'o')
->andWhere('o.Profile = :p')
->setParameter('p', $profile);
You need to ask Doctrine if the JSON array contains the status, but you can't do that with the QueryBuilder method.
When you hit the ORM limitations you can use a Native Query with ResultSetMapping. It allows you to write a pure SQL query using specific features of your DBMS but still get entity objects.
Or you can use scienta/doctrine-json-functions and use JSON_EXTRACT
My complete project will need to return the nearest-posts to the given geolocation.
I succeeded in doing this by building a custom callback function seeking the nearest posts but also building a custom array containing the data-fields of those nearest posts. If I define this function as the callback function of my route all works great!
All ok but I would like the return to be a copy of the default post-return, of course limited to the nearest posts. And not a set of custom defined fields.
I can't seem to figure out how to do this.
Before going into details I would like to know which way to go. My plan, that doesn't work, was :
extend WP_REST_Posts_Controller
register (and initialize) a new route
define a custom callback function in this new route
the custom callback function builds an array of postID's that are the nearest (lat&long was passed in the http request)
So far so good but now comes my problem.
How do I pass the array of postIDs to the get_items method of WP_REST_Posts_Controller in order to get a "default" return of specific(nearest) posts
Clearly a simple approach like this doesn't work
public function custom_callback_function($geoloc_array) {
// call the function get_nearest_items($geoloc_array)
// that returns an array of postIDs'
// something like : [163,49]
$nearest_poi_array = get_nearest_items($geoloc_array);
// put the array of postIDs to fetch in "include" of a new array
// hand over new array to the get_items method of WP_REST_posts_controller
$posts_to_return['include'] = $nearest_poi_array;
return get_items($posts_to_return);
}
If somebody could point me to the right direction I would be grateful!!
with meteor's IronRouter, I'm trying to use the this.params object elsewhere, but confused as to what it is. It seems to be a zero length array, that is actually an object with named methods after the path components.
# coffee
#route 'magnets',
path: '/magnets/lesson/:lessonCname'
data: ->
if #ready()
debugger;
console.log("route.params", #params)
with this code, in the debug console I will get:
this.params
[]
this.params.lessonCname
"despite-magnets-01"
typeof(this.params)
"object"
this.params.length
0
this.ready()
but in passing the params object to a server method, the methods (ie "lessonCname") disappear.
If my understanding is correct, then the near-term question is what is the best way to retrieve/convert these methods to {property:value} so they can be serialized and passed to server calls?
There are two easy ways of solving your problem, you can either set a global variable from within the data scope (but this is considered bad practice, at least IMO) or you can use the "data" function, which returns the data context for the current template:
data: ->
window._globalscopedata = #params.whatever #setting global variable
return someCollection.findOne #returns data context
_id: #params.whatever
when proccessing this route I will have the whatever param available in _globalscoredata and my document available in the template context.
Take a look at the source code for retrieving the parameters from a path. params is an array, but may have named properties. To iterate over everything, you can use the for in loop:
for(var x in myArray){
// Do something.
}
In this way, you can copy over everything to a new object (there may be a simpler way to create a copy).
The params property attached to a RouteController is an object with the following properties :
hash : the value of the URL hash.
query : an object consisting of key/value pairs representing the query string.
a list of URL fragments with their name and actual value.
Let's take an example, for this route definition :
// using iron:router#1.0.0-pre2 new route definition
Router.route("/posts/:slug");
And this URL typed in the browser address bar : /posts/first-post#comments?lang=en
We can use the console to find out precisely what params will actually contain :
> Router.current().params
Which will display this result :
Object {
hash: "comments",
slug: "first-post",
query: {
lang: "en"
}
}
Here slug is already a property of the params object whose value is "first-post", this is not a method.
If you want to extract from params these URL fragments as an object of key/value pairs, you can use underscore omit :
// getting rid of the hash and the query string
var parameters=_.omit(this.params,["hash","query"]);
I am trying to return all documents in a collection, to use it with an {{#each}} in my template. My code looks like this:
return Answers.find({}, {sort: {$natural:-1}})
But the documents are returned in natural order (not reverse). Does anyone know why? I got the $natural selector from the MongoDB documentation, so I don't see what's wrong.
Can't tell why don't it returns in reverse order.
But you can create an array in the template helper method and return reverse of an array using array.sort() or array.reverse() functions.
For ex: Say you Answers collection looks like this:
Answers({ansNo: 1, ansBody: "body1"},
{ansNo: 2, ansBody: "body2"},
{ansNo: 3, ansBody: "body3"});
And the array to be returned is:
var AnswersArr = new Array();
then in your template helper :->
var tempCollection = Answers.find({});
tempCollection.forEach(function(data){
var obj = {ansNo: data.asnNo, ansBody: data.ansBody};
AnswersArr.push(abj);
});
AnswersArr.sort(function(a, b){return b.ansNo - a.ansNo;}); //sort in reverse order
return AnswersArr;
Sort isn't a parameter but a separate function to be called after find() on the resulting Cursor object. This is the method that the MongoDB documentation is referring to and works with drivers such as MongoJS:
return Answers.find().sort({$natural: -1});
It seems Meteor hasn't added the sort() function to their implementation of Cursor, so an alternate solution would be to sort by the _id field which is generated based on date (and hence insertion order):
return Answers.find({}, {sort: {'_id': -1}});
As a workaround you could do this:
return Answers.find().fetch().reverse();
I know it would be nicer to do it via the sort parameter, but I don't think it's possible right now.
I think you might be confusing definitions of 'natural order' here. One the one hand there is a natural sort order for letters/strings (A,B,C...) and numbers (1,2,3...).
But in the case of mongo, 'natural' refers to the order the data was written to disk. '{$natural:1}' returns 'documents in the order they exist on disk...' and, so '{$natural:-1}' reverses that (http://docs.mongodb.org/manual/reference/operator/meta/natural/).
So without the code that writes the data and some insight into how it was written to disk, we cannot test your hypothesis that it is not working correctly.