Getting the JSON output in an object format for a single record instead of an array using XQuery - xquery

I am getting JSON output from an API in an array format(for multiple records) and in object format for a single record. Instead consumer wants to maintain one format - as an array for single record also. Please let me know if there is a way to display the JSON output in array format irrespective of single/multiple records using XQuery
I have tried the below XQuery:
<Response>
<totalSize>{totalSize/number()}</totalSize>
<done>{done/text()}</done>
<nextRecordsUrl>{nextRecordsUrl/text()}</nextRecordsUrl>
{
let $input:=/root/records
for $i in $input
return
<records>
<Product1>{$i/Product_Lookup_BI__c/text()}</Product1>
<EventLastModifiedDate>{$i/LastModifiedDate/text()}</EventLastModifiedDate>
<Venue>{$i/Venue_vod__r/(fn:concat(fn:concat(fn:concat(fn:concat(fn:concat(fn:concat(BI_Country_Code__c/text(),'-'),State_Province_vod__c/text()),'-'),City_vod__c/text()),'-'),Address_vod__c/text()))}</Venue>
{
let $a:=$i/EM_Event_Team_Member_vod__r/records
for $y in $a
return
<User_records>
<AttendeeLastModifiedDate>{$y/LastModifiedDate/text()}</AttendeeLastModifiedDate>
<EmployeeName>{$y/Team_Member_vod__r/Name/text()}</EmployeeName>
<EmployeeID>{$y/Team_Member_vod__r/BIDS_ID_BI__c/text()}</EmployeeID>
</User_records>
}
</records>
}
</Response>
Actual Output from the above XQuery:
{
"Response": {
"totalSize": 1,
"done": true,
"nextRecordsUrl": "",
"records": {
"Product1": "12345",
"EventLastModifiedDate": "2021-11-10T01:30:55.000+0000",
"Venue": "UK",
"User_records": {
"AttendeeLastModifiedDate": "2021-11-08T02:55:03.000+0000",
"EmployeeName": "Ish",
"EmployeeID": "00002113152"
}
}
}
}
Expected Output:
The Output should be in an array format for "records" & "user_records"
{
"Response":{
"totalSize":1,
"done":true,
"nextRecordsUrl":"",
"records":[
{
"Product1":"12345",
"EventLastModifiedDate":"2021-11-10T01:30:55.000+0000",
"Venue":"UK",
"User_records":[
{
"AttendeeLastModifiedDate":"2021-11-08T02:55:03.000+0000",
"EmployeeName":"Ish",
"EmployeeID":"00002113152"
}
]
}
]
}
}

Try:
<User_records xmlns:json="http://www.json.org" json:array="true">
<AttendeeLastModifiedDate>{$y/LastModifiedDate/text()}</AttendeeLastModifiedDate>
<EmployeeName>{$y/Team_Member_vod__r/Name/text()}</EmployeeName>
<EmployeeID>{$y/Team_Member_vod__r/BIDS_ID_BI__c/text()}</EmployeeID>
</User_records>
I would do the same for <records> as well. This example works in eXist-db. The JSON namespace may be different in your environment.
Here is what I ran in eXide:
xquery version "3.1";
declare option exist:serialize "method=json indent=yes media-type=application/json";
<Response>
<totalSize>5</totalSize>
<done>yes</done>
<nextRecordsUrl>abc</nextRecordsUrl>
<User_records xmlns:json="http://www.json.org" json:array="true">
<AttendeeLastModifiedDate>123</AttendeeLastModifiedDate>
<EmployeeName>456</EmployeeName>
<EmployeeID>789</EmployeeID>
</User_records>
</Response>

Related

Need to compare array in Marklogic with xquery

I need to compare array in MarkLogic with Xquery .
Query parameters:
{
"list": {
"bookNo": 13,
"BookArray":[20,21,22,23,24,25]
}
}
Sample Data:
{
"no":01'
"arrayList"[20,25]
}
{
"no":02'
"arrayList"[20,27]
}
{
"no":03'
"arrayList"[20,23,25]
}
Output:
"no":01
"no":03
I need to return "no" where all values from arraylist should be match with bookArray.
Ok. You do not explain if the actual data is in the system or not. So I did an example as if it is all in memory.
I chose to keep the sample in the MarkLogic JSON representation which has some oddities like number-nodes and array-nodes under the hood. To make it more readable if you dig into it, i used fn:data() to get less verbose. In all reality, if this was an in-memory operation and I could not use Javascript, then I would have converted the JSON structures to maps.
Here is a sample to help you explore. I cleaned up the JSON to be valid and for my sample wrapped the three samples in a single array.
xquery version "1.0-ml";
let $param-as-json := xdmp:unquote('{
"list": {
"bookNo": 13,
"BookArray":[20,21,22,23,24,25]
}
}')
let $list-as-json := xdmp:unquote('[
{
"no":"01",
"arrayList":[20,25]
},
{
"no":"02",
"arrayList":[20,27]
},
{
"no":"03",
"arrayList":[20,23,25]
}
]')
let $my-list := fn:data($param-as-json//BookArray)
return for $item in $list-as-json/*
let $local-list := fn:data($item//arrayList)
let $intersection := fn:data($item//arrayList)[.=$my-list]
where fn:deep-equal($intersection, $local-list)
return $item/no
Result:
01
03

RethinkDB filtering Object Array

I'm new to rethinkdb and I wanted to filter something like... get all with Kiwi or Strawberry as preferred fruit
{
"id": "65dbaa34-f7d5-4a25-b01f-682032fc6e05" ,
"fruits": {
"favorite": "Mango" ,
"preferred": [
"Kiwi" ,
"Watermelon"
]
}
}
I tried something like this after reading contains doc:
r.db('appname').table('food')
.filter(r.row('fruits').contains(function(doc) {
return doc('preferred').contains('Kiwi');
}))
And I'm getting a e: Cannot convert OBJECT to SEQUENCE in: error.
This is what you're looking for:
r.db('appname').table('food')
.filter((row) => {
r.or( // Returns true if any of the following are true
row('fruits')('preferred').contains('Kiwi'),
row('fruits')('preferred').contains('Strawberry')
)
});
You should know as well, that you can create your own index that calculates this for you, then you'd be able to do a .getAll query using your custom index and return all documents that fit this constraint very quickly.
Lastly, for something that would also work but is probably less efficient on large arrays:
r.db("appname").table('food')
.filter((row) => {
return row('fruits')('preferred').setIntersection(['Kiwi', 'Strawberry']).count().gt(0)
})

Query to get exact matches of Elastic Field with multile values in Array

I want to write a query in Elastic that applies a filter based on values i have in an array (in my R program). Essentially the query:
Matches a time range (time field in Elastic)
Matches "trackId" field in Elastic to any value in array oth_usr
Return 2 fields - "trackId", "propertyId"
I have the following primitive version of the query but do not know how to use the oth_usr array in a query (part 2 above).
query <- sprintf('{"query":{"range":{"time":{"gte":"%s","lte":"%s"}}}}',start_date,end_date)
view_list <- elastic::Search(index = "organised_recent",type = "PROPERTY_VIEW",size = 10000000,
body=query, fields = c("trackId", "propertyId"))$hits$hits
You need to add a terms query and embed it as well as the range one into a bool/must query. Try updating your query like this:
terms <- paste(sprintf("\"%s\"", oth_usr), collapse=", ")
query <- sprintf('{"query":{"bool":{"must":[{"terms": {"trackId": [%s]}},{"range": {"time": {"gte": "%s","lte": "%s"}}}]}}}',terms,start_date,end_date)
I'm not fluent in R syntax, but this is raw JSON query that works.
It checks whether your time field matches given range (start_time and end_time) and whether one of your terms exact matches trackId.
It returns only trackId, propertyId fields, as per your request:
POST /indice/_search
{
"_source": {
"include": [
"trackId",
"propertyId"
]
},
"query": {
"bool": {
"must": [
{
"range": {
"time": {
"gte": "start_time",
"lte": "end_time"
}
}
},
{
"terms": {
"trackId": [
"terms"
]
}
}
]
}
}
}

ElasticSearch - Range filter by date is not working,

I'm trying to filter a query using range by date but it's not working. If i use gt, gte, lt, lte it returns zero results. If i use only gt or lt, it returns some results but the filter is not working.
I've checked datatype on uri http://mydomain.local:9200/logstash-2014.09.09/_mapping?pretty=true the field type is correct:
"original" : {
"type" : "date",
"format" : "dateOptionalTime"
}
Here is an example of a result that i have indexed in ElasticSearch:
{
"_index" : "logstash-2014.09.08",
"_type" : "iis",
"_id" : "wxtnfpyjR4u7dhwlEAWevw",
"_score" : 1.0,
"_source":{"#version":"1","#timestamp":"2014-09-08T20:55:46.460Z",
"type":"iis","original":"14-09-08 17:39:58"}
}
And here is how i'm trying to perform a query:
{
"query" : {
"filtered" : {
"filter" : {
"range" : {
"original" : {
"gt" : "14-09-10"
}
}
}
}
}
}
Anyone knows what is wrong on my query? Why it returns some results if i don't have any date greater than today ( 2014-09-09 )?
I created an index with the same mapping and tried to put a record in
curl -XPOST localhost:9200/test-1/x/_index -d '{"original": "14-09-08 17:39:58"}'
and got an error:
{"error":"MapperParsingException[failed to parse [original]]; nested: MapperParsingException[failed to parse date field [14-09-08 17:39:58], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: \"14-09-08 17:39:58\" is malformed at \" 17:39:58\"]; ","status":400}`
So I believe that you are in a a locale where that ##-##-## is being interpreted as something other than yy-mm-dd.
You are using Logstash so you can fix the original field before it goes in with a mutate to make it unambiguous.
mutate {
replace => { original => "20%{original}" }
}

Using jsonPath looking for a string

I'm trying to use jsonPath and the pick function to determine if a rule needs to run or not based on the current domain. A simplified version of what I'm doing is here:
global
{
dataset shopscotchMerchants <- "https://s3.amazonaws.com/app-files/dev/merchantJson.json" cachable for 2 seconds
}
rule checkdataset is active
{
select when pageview ".*" setting ()
pre
{
merchantData = shopscotchMerchants.pick("$.merchants[?(#.merchant=='Telefora')]");
}
emit
<|
console.log(merchantData);
|>
}
The console output I expect is the telefora object, instead I get all three objects from the json file.
If instead of merchant=='Telefora' I use merchantID==16 then it works great. I thought jsonPath could do matches to strings as well. Although the example above isn't searching against the merchantDomain part of the json, I'm experiencing the same problem with that.
Your problem comes from the fact that, as stated in the documentation, the string equality operators are eq, neq, and like. == is only for numbers. In your case, you want to test if one string is equal to another string, which is the job of the eq string equality operator.
Simply swap == for eq in you JSONpath filter expression and you will be good to go:
global
{
dataset shopscotchMerchants <- "https://s3.amazonaws.com/app-files/dev/merchantJson.json" cachable for 2 seconds
}
rule checkdataset is active
{
select when pageview ".*" setting ()
pre
{
merchantData = shopscotchMerchants.pick("$.merchants[?(#.merchant eq 'Telefora')]"); // replace == with eq
}
emit
<|
console.log(merchantData);
|>
}
I put this to the test in my own test ruleset, the source for which is below:
ruleset a369x175 {
meta {
name "test-json-filtering"
description <<
>>
author "AKO"
logging on
}
dispatch {
domain "exampley.com"
}
global {
dataset merchant_dataset <- "https://s3.amazonaws.com/app-files/dev/merchantJson.json" cachable for 2 seconds
}
rule filter_some_delicous_json {
select when pageview "exampley.com"
pre {
merchant_data = merchant_dataset.pick("$.merchants[?(#.merchant eq 'Telefora')]");
}
{
emit <|
try { console.log(merchant_data); } catch(e) { }
|>;
}
}
}

Resources