I want to know the value of 'pageName' where the value of 'createdTs' is '333' in JSON like below, how should I use Jsonpath?
[
{
"Test1":{
"pageName":"Apple"
},
"Test1-1":{
"createdTs":"333"
}
},
{
"Test2":{
"pageName":"Tomato"
},
"Test2-1":{
"createdTs":"555"
}
}
]
.Test1-1[?(#.createdTs == 333)] would look like this:
[
{
"createdTs" : "333"
}
]
The parsing result I want is this.
[
{
"Test1":{
"pageName":"Apple"
},
"Test1-1":{
"createdTs":"333"
}
}
]
if you are using Jayway JSONPath a Java DSL for reading JSON documents, then you can use the these jsonpath expressions.
$[?(#.['Test1-1'].createdTs == 333)]
OR using in filter operator
$[?(333 in #..createdTs)]
OR using contains filter operator
$[?(#..createdTs contains 333)]
Related
I wonder is there is a way of querying inverted index by using complex logical operations, like:
find "some_text" from property 'property1' OR 'property2' AND created_at before 'date'
where filter only accepts one operator according to documentation.
Absolutely, it's explained here
An example based on your question would be:
{
Get {
ClassName(
where: {
operator: And
operands: [{
operator: Or
operands: [{
path: ["property"]
operator: Like
valueString: "property1"
}, {
path: ["property"]
operator: Like
valueString: "property2"
}]
}, {
path: ["_creationTimeUnix"]
operator: LessThan
valueString: "1664505080008"
}]
}
) {
property
_additional {
creationTimeUnix
}
}
}
}
PS:
This indeed becomes -as you write- a complex filter ;)
I have an index in ElasticSearch with two fields of date type (metricsTime & arrivalTime). A sample document is quoted below. In Kibana, I created a scripted field delay for the difference between those two fields. My painless script is:
doc['arrivalTime'].value - doc['metricsTime'].value
However, I got the following error message when navigating to Kibana's Discover tab: class_cast_exception: Cannot apply [-] operation to types [org.joda.time.MutableDateTime] and [org.joda.time.MutableDateTime].
This looks same as the error mentioned in https://discuss.elastic.co/t/problem-in-difference-between-two-dates/121655. But the answer in that page suggests that my script is correct. Could you please help?
Thanks!
{
"_index": "events",
"_type": "_doc",
"_id": "HLV274_1537682400000",
"_version": 1,
"_score": null,
"_source": {
"metricsTime": 1537682400000,
"box": "HLV274",
"arrivalTime": 1539930920347
},
"fields": {
"metricsTime": [
"2018-09-23T06:00:00.000Z"
],
"arrivalTime": [
"2018-10-19T06:35:20.347Z"
]
},
"sort": [
1539930920347
]
}
Check the list of Lucene Expressions to check what expressions are available for date field and how you could use them
Just for sake of simplicity, check the below query. I have created two fields metricsTime and arrivalTime in a sample index I've created.
Sample Document
POST mydateindex/mydocs/1
{
"metricsTime": "2018-09-23T06:00:00.000Z",
"arrivalTime": "2018-10-19T06:35:20.347Z"
}
Query using painless script
POST mydateindex/_search
{ "query": {
"bool": {
"must": {
"match_all": {
}
},
"filter": {
"bool" : {
"must" : {
"script" : {
"script" : {
"inline" : "doc['arrivalTime'].date.dayOfYear - doc['metricsTime'].date.dayOfYear > params.difference",
"lang" : "painless",
"params": {
"difference": 2
}
}
}
}
}
}
}
}
}
Note the below line in the query
"inline" : "doc['arrivalTime'].date.dayOfYear - doc['metricsTime'].date.dayOfYear > params.difference"
Now if you change the value of difference from 2 to 26 (which is one more than the difference in the dates) then you see that the above query would not return the document.
But nevertheless, I have mentioned the query as an example as how using scripting you can compare two different and please do refer to the link I've shared.
I need help with writing my consumer Pacts using pact-jvm (https://github.com/DiUS/pact-jvm).
My problem is I have a field which is a list (an array) of maps. Each map can have elements of different types (strings or sub-maps), eg.
"validatedAnswers": [
{
"type": "typeA",
"answers": {
"favourite_colour": "Blue",
"correspondence_address": {
"line_1": "Main St",
"postcode": "1A 2BC",
"town": "London"
}
}
},
{
"type": "typeB",
"answers": {
"first_name": "Firstname",
"last_name": "Lastname",
}
}
]
but we're only interested in some of those answers.
NOTE: The above is only an example showing the structure of validatedAnswers. Each answers map has dozens of elements.
What we really need is this: https://github.com/pact-foundation/pact-specification/issues/38, but it's planned for v.4. In the meantime we're trying a different approach. What I'm attempting to do now is to specify that each element of the list is a non-empty map. Another approach is to specify that each element of the list is not null. Can any of this be done using Groovy DSL?
This:
new PactBuilder().serviceConsumer('A').hasPactWith('B')
.port(findAvailablePort()).uponReceiving(...)
.willRespondWith(status: 200, headers: ['Content-Type': 'application/json'])
.withBody {
validatedAnswers minLike(1) {
type string()
answers {
}
}
}
doesn't work because it mean answers is expected to be empty ("Expected an empty Map but received Map( [...] )", see also https://github.com/DiUS/pact-jvm/issues/298).
So what I would like to do is something like this:
.withBody {
validatedAnswers minLike(1) {
type string()
answers Matchers.map()
}
}
or:
validatedAnswers minLike(1) {
type string()
answers {
keyLike 'title', notNull()
}
}
or:
validatedAnswers minLike(1) {
type string()
answers notNull()
}
Can it be done?
I would create two separate tests for this, one test for each of the different response shapes and have a provider state for each e.g. given there are type b answers.
This way when you verify on provider side, it will only send those two field types.
The union of the two examples gives a contract that allows both.
You can do it without DSL, sample Groovy script:
class ValidateAnswers {
static main(args) {
/* Array with some samples */
List<Map> answersList = [
[
type: 'typeA',
answers: [
favourite_colour: 'Blue',
correspondence_address: [
line_1: 'Main St',
postcode: '1A 2BC',
town: 'London'
]
]
],
[
type: 'typeB',
answers: [
first_name: 'Firstname',
last_name: "Lastname"
]
],
[
type: 'typeC',
answers: null
],
[
type: 'typeD'
],
[
type: 'typeE',
answers: [:]
]
]
/* Iterating through all elements in list above */
for (answer in answersList) {
/* Print result of checking */
println "$answer.type is ${validAnswer(answer) ? 'valid' : 'not valid'}"
}
}
/**
* Method to recursive iterate through Map's.
* return true only if value is not an empty Map and it key is 'answer'.
*/
static Boolean validAnswer(Map map, Boolean result = false) {
map.each { key, value ->
if (key == 'answers') {
result = value instanceof Map && value.size() > 0
} else if (value instanceof Map) {
validAnswer(value as Map, false)
}
}
return result
}
}
Output is:
typeA is valid
typeB is valid
typeC is not valid
typeD is not valid
typeE is not valid
I would like to turn this resultset
[
{
"Document": {
"JsonData": "{\"key\":\"value1\"}"
}
},
{
"Document": {
"JsonData": "{\"key\":\"value2\"}"
}
}
]
into this
[
{
"key": "value1"
},
{
"key": "value2"
}
]
I can get close by using a query like
select value c.Document.JsonData from c
however, I end up with
[
"{\"key\":\"value1\"}",
"{\"key\":\"value2\"}"
]
How can I cast each value to an individual JSON fragment using the SQL API?
As David Makogon said above, we need to transform such data within our app. We can do as below:
string data = "[{\"key\":\"value1\"},{\"key\":\"value2\"}]";
List<Object> t = JsonConvert.DeserializeObject<List<Object>>(data);
string jsonData = JsonConvert.SerializeObject(t);
Screenshot of result:
I have a set of timestamps formatted as seconds since the epoch. I'd like to insert to ElasticSearch as epoch_seconds but when querying would like to see the output as a pretty date, e.g. strict_date_optional_time.
My below mapping preserves the format that the input came in - is there any way to normalize the output to just one format via the mapping api?
Current Mapping:
PUT example
{
"mappings": {
"time": {
"properties": {
"time_stamp": {
"type": "date",
"format": "strict_date_optional_time||epoch_second"
}
}
}
}
}
Example docs
POST example/time
{
"time_stamp": "2018-03-18T00:00:00.000Z"
}
POST example/time
{
"time_stamp": "1521389162" // Would like this to output as: 2018-03-18T16:05:50.000Z
}
GET example/_search output:
{
"total": 2,
"max_score": 1,
"hits": [
{
"_source": {
"time_stamp": "1521389162", // Stayed as epoch_second
}
},
{
"_source": {
"time_stamp": "2018-03-18T00:00:00.000Z"
}
}
]
}
Elasticsearch differentiates between the _source and the so called stored fields. The first one is supposed to represent your input.
If you actually use stored fields (by specifying store=true in your mapping) then specify multiple date formats this is easy: (emphasis mine)
Multiple formats can be specified by separating them with || as a separator. Each format will be tried in turn until a matching format is found. The first format will be used to convert the milliseconds-since-the-epoch value back into a string.
I have tested this with elasticsearch 5.6.4 and it works fine:
PUT /test -d '{ "mappings": {"doc": { "properties": {"post_date": {
"type":"date",
"format":"basic_date_time||epoch_millis",
"store":true
} } } } }'
PUT /test/doc/2 -d '{
"user" : "test1",
"post_date" : "20150101T121030.000+01:00"
}'
PUT /test/doc/1 -d '{
"user" : "test2",
"post_date" : 1525167490500
}'
Note how two different input-formats will result in the same format when using GET /test/_search?stored_fields=post_date&pretty=1
{
"hits" : [
{
"_index" : "test",
"_type" : "doc",
"_id" : "2",
"_score" : 1.0,
"fields" : {
"post_date" : [
"20150101T111030.000Z"
]
}
},
{
"_index" : "test",
"_type" : "doc",
"_id" : "1",
"_score" : 1.0,
"fields" : {
"post_date" : [
"20180501T093810.500Z"
]
}
}
]
}
If you want to change the input (in _source) you're not so lucky, the mapping-transform feature has been removed:
This was deprecated in 2.0.0 because it made debugging very difficult. As of now there really isn’t a feature to use in its place other than transforming the document in the client application.
If, instead of changing the stored data you are interested in formatting the output, have a look at this answer to Format date in elasticsearch query (during retrieval)