ElasticSearch - Range filter by date is not working, - datetime

I'm trying to filter a query using range by date but it's not working. If i use gt, gte, lt, lte it returns zero results. If i use only gt or lt, it returns some results but the filter is not working.
I've checked datatype on uri http://mydomain.local:9200/logstash-2014.09.09/_mapping?pretty=true the field type is correct:
"original" : {
"type" : "date",
"format" : "dateOptionalTime"
}
Here is an example of a result that i have indexed in ElasticSearch:
{
"_index" : "logstash-2014.09.08",
"_type" : "iis",
"_id" : "wxtnfpyjR4u7dhwlEAWevw",
"_score" : 1.0,
"_source":{"#version":"1","#timestamp":"2014-09-08T20:55:46.460Z",
"type":"iis","original":"14-09-08 17:39:58"}
}
And here is how i'm trying to perform a query:
{
"query" : {
"filtered" : {
"filter" : {
"range" : {
"original" : {
"gt" : "14-09-10"
}
}
}
}
}
}
Anyone knows what is wrong on my query? Why it returns some results if i don't have any date greater than today ( 2014-09-09 )?

I created an index with the same mapping and tried to put a record in
curl -XPOST localhost:9200/test-1/x/_index -d '{"original": "14-09-08 17:39:58"}'
and got an error:
{"error":"MapperParsingException[failed to parse [original]]; nested: MapperParsingException[failed to parse date field [14-09-08 17:39:58], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: \"14-09-08 17:39:58\" is malformed at \" 17:39:58\"]; ","status":400}`
So I believe that you are in a a locale where that ##-##-## is being interpreted as something other than yy-mm-dd.
You are using Logstash so you can fix the original field before it goes in with a mutate to make it unambiguous.
mutate {
replace => { original => "20%{original}" }
}

Related

Cannot deserialize instance with R POST()

I'm trying to get a queryID using the POST() funtion in R. It works well as long as I only use a simple JSON
library(httr)
library(jsonlite)
base_json <- paste('
{
"segment" : "WHG_M"
}
')
id <- POST("url",
body = fromJSON(base_json),
encode = "json",
authenticate(username,password, type = "basic"))
However, when I try to incorporate further conditions, i.e.:
base_json <- paste('
{
"segment" : "WHG_M",
"administrativeSpatialFilter" : {
"municipalityCodes" : [ 11000000 ]
}
}
')
I get the following error for POST():
Cannot deserialize instance of `java.util.ArrayList`
out of VALUE_NUMBER_INT token
with
fromJSON(base_json)
$segment
[1] "WHG_M"
$administrativeSpatialFilter
$administrativeSpatialFilter$municipalityCodes
[1] 11000000
Does anyone know how to solve the issue?
The exception contains 'java.util.', which clearly indicated that is is thrown by the REST-service written in Java.
I guess this JSON
{
"segment" : "WHG_M",
"administrativeSpatialFilter" : {
"municipalityCodes" : [ 11000000 ]
}
}
is transformed by you R-Client methods to
{
"segment" : "WHG_M",
"administrativeSpatialFilter" : {
"municipalityCodes" : 11000000
}
}
which is not a list anymore and breaks the JSON-Parser on the server.
You have to foce your JSON-Encoder to keep the JSON-list-structure, even though it has only one element.

How to access final element in Handlebars.js array?

Given the following data structure:
{
"name" : "test",
"assetType" : "Home",
"value" : [
{
"value" : "999",
"date" : "10/03/2018"
},
{
"value" : "1234.56",
"date" : "10/04/2018"
}
]
}
How can I directly access the first and last element of value in Handlebars? My code is:
<td>{{value.[0].value}}</td>
<td>{{value.[value.length - 1].value}}</td>
The first statement works fine, but I cannot figure out how to access the last element. value.length inside the []'s does not return any value, but rendering value.length outside of the array works fine.
I tried using #first and #last in the array but those don't seem to work either.
As already pointed out, you can easily solve this with a template helper:
Template.myTemplate.helpers({
first (arr) {
return arr && arr[0]
},
last (arr) {
return arr && arr[arr.length - 1]
},
atIndex (arr, index) {
return arr && index >= 0 && index <= arr.length - 1 && arr[index]
}
})
You can then create a context using #with where the context object is the result of the helpers. Inside this context you can access properties using this.
Accessing values with the name value would look like the following:
<td>{{#with first value}}{{this.value}}{{/with}}</td>
<td>{{#with last value}}{{this.value}}{{/with}}</td>
<td>{{#with atIndex value 5}}{{this.value}}{{/with}}</td>

firebase realtime database filter data by date range [duplicate]

This question already has answers here:
Firebase query by date string
(2 answers)
Closed 4 years ago.
We are using firebase realtime db and have orders data. the date is stored as a string value and we are looking to query like orders in last one week. The data look like
"orders" : {
"-LM_h6nKUKQY0wyqSCz4" : {
"order" : {
"_couponDiscount" : "382.5",
"_date" : "17-9-2018",
"_orderNumber" : "MannuSharma-21581",
"_orderStatus" : "Delivered",
"_orderTotal" : 450,
}
},
"-LMatbq6prD4kNtTqzmx" : {
"order" : {
"_date" : "19-9-2018",
"_orderNumber" : "AnkushVerma-22991",
"_orderStatus" : "Delivered",
"_orderTotal" : 600,
}
},
"-LMaxcGocVxe9nnfFnT2" : {
"order" : {
"_date" : "18-9-2018",
"_orderNumber" : "MannuSharma-40284",
"_orderStatus" : "Delivered",
"_orderTotal" : 450,
}
}
The _date is the field and in this case lets say i need to get data fro date 17 and 18th.
the startAt and endAt will do it lexographically it seems, so how do i get this data treated as date?
If you want to sort your dates lexicographically in such a way you can query from startAt to endAt you should store them as YYYY-MM-DD instead of DD-MM-YYYY.
Note that changing 19-9-2018 to 2018-9-19 would not work, you must store it as 2018-09-19.

Why is this push() overwriting instead of adding?

I have a line of code inside a http callable Firebase Function that is supposed to add a new record to a spot in the Firebase Database.
return admin.database().ref('games/' + gameid + '/' + uid + '/logs/').push().set({guess: guess, result: pigs.toString() + 'P ' + bulls.toString() + 'B', score: score});
Here is an example entry in my database.
"games" : {
"39dba69c-f5a8-4e94-9553-d19372d43a92" : {
"1P0mWbp1jYOayaMEyQcnKvX4YB63" : {
"logs" : {
"-LDshYx2wJK6eT47_OVQ" : {
"guess" : "5647",
"result" : "1P 1B",
"score" : 450
}
},
"maxbulls" : 3,
"maxpigs" : 3,
"moves" : 6,
"score" : 450
},
"answer" : "2739dba69c41",
"id" : "39dba69c-f5a8-4e94-9553-d19372d43a92",
"requestor" : "1P0mWbp1jYOayaMEyQcnKvX4YB63",
"status" : "pending",
"turn" : ""
}
}
For some reason the set() is overwriting the record under logs (id of -LDshYx2wJK6eT47_OVQ). I want it to add a new record under logs with the auto-generated key from push(). I have used a very similar method before (with basically no differences) that worked as intended.
I have also tried changing set() to update() but it still overwrote. Is there something wrong with my path or database structure that I am overlooking? Thanks.

Filtering / Querying by the Contents of a List in DynamoDB

I am attempting to filter a DynamoDB query by the contents of a Map contained within a List. Here's an example of the structure I'm dealing with.
{
'EventType': 'git/push'
'EventTime': 1416251010,
'Commits': [
{
'id': '29d02aff...',
'subject': 'Add the thing to the place'
},
{
'id': '9d888fec...',
'subject': 'Spelling errors'
},
...
]
}
The hash key is EventType and range key EventTime. I am trying to write a filter that filters the result of a query to a specific id. Is is possible to create a DynamoDB filter expression that correctly filters the query like this? (My first thought was to use contains (a, a), but I don't think that will work on a List of Maps.)
This isn't currently supported by DynamoDB's API's expression language (as of November 2014) but there are some workarounds mentioned here.
I found a solution and I tested it, and it is working fine.
Here's what I did,
// created a schema like that:
var Movie = dynamo.define('example-nested-attribute', {
hashKey : 'title',
timestamps : true,
schema : {
title : Joi.string(),
releaseYear : Joi.number(),
tags : dynamo.types.stringSet(),
director : Joi.object().keys({
firstName : Joi.string(),
lastName : Joi.string(),
titles : Joi.array()
}),
actors : Joi.array().items(Joi.object().keys({
firstName : Joi.string(),
lastName : Joi.string(),
titles : Joi.array()
}))
}
});
and then you can query of a nested array of objects:
var params = {};
params.UpdateExpression = 'SET #year = #year + :inc, #dir.titles = list_append(#dir.titles, :title), #act[0].firstName = :firstName ADD tags :tag';
params.ConditionExpression = '#year = :current';
params.ExpressionAttributeNames = {
'#year' : 'releaseYear',
'#dir' : 'director',
'#act' : 'actors'
};
params.ExpressionAttributeValues = {
':inc' : 1,
':current' : 2001,
':title' : ['The Man'],
':firstName' : 'Rob',
':tag' : dynamo.Set(['Sports', 'Horror'], 'S')
};

Resources