Given documents such as:
{
"id": 1,
"types":
{
"type1": { "visible": true },
"type2": { "visible": false }
}
},
{
"id": 2,
"types":
{
"type3": { "visible": true }
}
},
{
"id": 3,
"types":
{
"type4": { "visible": false }
}
}
What SQL query should I write to return documents which have at least one type with "visible" == false?
Notice that "types" is an object, not an array. Actually it is a dictionary where property name is key. The property names of "types" object are not predictable, so I can't hardcode them in a query.
As far as I am known, there are no way to deal with it by using SQL directly.
There are two workaround. First one, change your schema of your document and using known key. Another one, create UDF to deal with this.
you can refer to Querying complex nested object in cosmosdb using sql Api
Related
Question:
I am trying SQL query as the image showed below，I want it to be grouped by the same timestamp
expected output:
[
{
"tag1": {
"TagName": "PV1-input-power-L(10W)",
"Value": 0
},
"tag2": {
"TagName": "Sunshine-Display-Value",
"Value": 0
},
"tag3": {
"TagName": "TotalEnergy-(100kWh)_1",
"Value": 0
},
"timestamp": "2020-03-27T02:40:18Z"
}
]
sample document:
You can use User Defined Functions.
Here is the data from my containers
Here is the function I have created. I named it CustomArray.
function userDefinedFunction(input){
var obj={};
input.forEach(function(element,index){
obj["tag"+index] = {
TagName :element.TagName,
Value: element.Value
};
}); return obj;}
Here, I run the UDF with my select statement
It returns the following data.
Schema is very close to what you are looking for. I think you can make it better by changing some jscript in UDF.
I hope this helps!
I have an index in ElasticSearch with two fields of date type (metricsTime & arrivalTime). A sample document is quoted below. In Kibana, I created a scripted field delay for the difference between those two fields. My painless script is:
doc['arrivalTime'].value - doc['metricsTime'].value
However, I got the following error message when navigating to Kibana's Discover tab: class_cast_exception: Cannot apply [-] operation to types [org.joda.time.MutableDateTime] and [org.joda.time.MutableDateTime].
This looks same as the error mentioned in https://discuss.elastic.co/t/problem-in-difference-between-two-dates/121655. But the answer in that page suggests that my script is correct. Could you please help?
Thanks!
{
"_index": "events",
"_type": "_doc",
"_id": "HLV274_1537682400000",
"_version": 1,
"_score": null,
"_source": {
"metricsTime": 1537682400000,
"box": "HLV274",
"arrivalTime": 1539930920347
},
"fields": {
"metricsTime": [
"2018-09-23T06:00:00.000Z"
],
"arrivalTime": [
"2018-10-19T06:35:20.347Z"
]
},
"sort": [
1539930920347
]
}
Check the list of Lucene Expressions to check what expressions are available for date field and how you could use them
Just for sake of simplicity, check the below query. I have created two fields metricsTime and arrivalTime in a sample index I've created.
Sample Document
POST mydateindex/mydocs/1
{
"metricsTime": "2018-09-23T06:00:00.000Z",
"arrivalTime": "2018-10-19T06:35:20.347Z"
}
Query using painless script
POST mydateindex/_search
{ "query": {
"bool": {
"must": {
"match_all": {
}
},
"filter": {
"bool" : {
"must" : {
"script" : {
"script" : {
"inline" : "doc['arrivalTime'].date.dayOfYear - doc['metricsTime'].date.dayOfYear > params.difference",
"lang" : "painless",
"params": {
"difference": 2
}
}
}
}
}
}
}
}
}
Note the below line in the query
"inline" : "doc['arrivalTime'].date.dayOfYear - doc['metricsTime'].date.dayOfYear > params.difference"
Now if you change the value of difference from 2 to 26 (which is one more than the difference in the dates) then you see that the above query would not return the document.
But nevertheless, I have mentioned the query as an example as how using scripting you can compare two different and please do refer to the link I've shared.
I need help with writing my consumer Pacts using pact-jvm (https://github.com/DiUS/pact-jvm).
My problem is I have a field which is a list (an array) of maps. Each map can have elements of different types (strings or sub-maps), eg.
"validatedAnswers": [
{
"type": "typeA",
"answers": {
"favourite_colour": "Blue",
"correspondence_address": {
"line_1": "Main St",
"postcode": "1A 2BC",
"town": "London"
}
}
},
{
"type": "typeB",
"answers": {
"first_name": "Firstname",
"last_name": "Lastname",
}
}
]
but we're only interested in some of those answers.
NOTE: The above is only an example showing the structure of validatedAnswers. Each answers map has dozens of elements.
What we really need is this: https://github.com/pact-foundation/pact-specification/issues/38, but it's planned for v.4. In the meantime we're trying a different approach. What I'm attempting to do now is to specify that each element of the list is a non-empty map. Another approach is to specify that each element of the list is not null. Can any of this be done using Groovy DSL?
This:
new PactBuilder().serviceConsumer('A').hasPactWith('B')
.port(findAvailablePort()).uponReceiving(...)
.willRespondWith(status: 200, headers: ['Content-Type': 'application/json'])
.withBody {
validatedAnswers minLike(1) {
type string()
answers {
}
}
}
doesn't work because it mean answers is expected to be empty ("Expected an empty Map but received Map( [...] )", see also https://github.com/DiUS/pact-jvm/issues/298).
So what I would like to do is something like this:
.withBody {
validatedAnswers minLike(1) {
type string()
answers Matchers.map()
}
}
or:
validatedAnswers minLike(1) {
type string()
answers {
keyLike 'title', notNull()
}
}
or:
validatedAnswers minLike(1) {
type string()
answers notNull()
}
Can it be done?
I would create two separate tests for this, one test for each of the different response shapes and have a provider state for each e.g. given there are type b answers.
This way when you verify on provider side, it will only send those two field types.
The union of the two examples gives a contract that allows both.
You can do it without DSL, sample Groovy script:
class ValidateAnswers {
static main(args) {
/* Array with some samples */
List<Map> answersList = [
[
type: 'typeA',
answers: [
favourite_colour: 'Blue',
correspondence_address: [
line_1: 'Main St',
postcode: '1A 2BC',
town: 'London'
]
]
],
[
type: 'typeB',
answers: [
first_name: 'Firstname',
last_name: "Lastname"
]
],
[
type: 'typeC',
answers: null
],
[
type: 'typeD'
],
[
type: 'typeE',
answers: [:]
]
]
/* Iterating through all elements in list above */
for (answer in answersList) {
/* Print result of checking */
println "$answer.type is ${validAnswer(answer) ? 'valid' : 'not valid'}"
}
}
/**
* Method to recursive iterate through Map's.
* return true only if value is not an empty Map and it key is 'answer'.
*/
static Boolean validAnswer(Map map, Boolean result = false) {
map.each { key, value ->
if (key == 'answers') {
result = value instanceof Map && value.size() > 0
} else if (value instanceof Map) {
validAnswer(value as Map, false)
}
}
return result
}
}
Output is:
typeA is valid
typeB is valid
typeC is not valid
typeD is not valid
typeE is not valid
I have a database containing a structure similar to the one shown below. Since I am not allowed to make it public I am using the following keys:
i = {0,1,2,3....}
All a(i) represent single key/value pairings e.g. userName: "awesome"
With the below structure, every time a User wants to create new Stuff these are the steps I currently take
Store images to FIRStorage and retrieve their respective downloadURL
Then I add the Stuff to FIRDatabase. At this point I add all of the info related to a(i) because they're all under a single child; Stuff.UUID, hence I send one huge dictionary consisting of the data.
The issue arises in adding data to the mini-dictionaries. Because they are in different paths, I have to individually make requests to all of them as shown below
I then add location info
Followed by the respective child of timeStamp
The images information is next to be updated
Subscribe the User to respective Stuff.UUID
Lastly add the User.UUID to the members portion of Stuff
Is it possible to reduce steps 3-7?
As a follow up, is it possible to add values into different paths with one call?
PS: Link to what the code of the above demo might look like. Due to confidentiality stuff I am not allowed to post the actual code.
{
"Users":
{
"JglJnGDXcqLq6m844pZ":
{
a(0),
a(1),
a(2),
a(3),
a(4),
a(5),
"Stuff":
{
"fcWpzduhpPn8XR6Zqca": true,
"gfntTr6TkDwZ439jkW8": true
}
}
},
"Stuff":
{
"fcWpzduhpPn8XR6Zqca":
{
a(0),
a(1),
a(2),
a(3),
a(4),
a(5),
a(6),
a(7),
"location":
{
"latitude":"-17.41439",
"longitude":"-5.85170"
},
"timestamp":
{
"created":
{
a(0),
a(1),
a(2),
a(3),
a(4),
a(5),
},
"lastModified":
{
a(0),
a(1),
a(2),
a(3),
a(4),
a(5),
}
},
"images":
{
"B4FaR6wfJAeXqJ29T33":
{
"imageURL": "https://google.com"
}
},
"members":
{
"JglJnGDXcqLq6m844pZ": true,
"DpHAfrqL4eqbR8QNgHg": true
}
}
}
}
I want to have definitions section better generated and organized and for me this would mean to not allow depth building up in definitions, but, each class involved in the structure tree should have its own entry in the definitions section and referenced via $ref. This means that for each definition I would only have a list of properties that would either be of primitive types (string, boolean, etc.) or would be a $ref to another definition entry for another custom class. You can also see this as depth 1 definition, close to how classes are originally defined in C#.
To illustrate this via a trivial example:
JSchemaGenerator schemaGenerator = new JSchemaGenerator();
schemaGenerator = new JSchemaGenerator()
{
DefaultRequired = Newtonsoft.Json.Required.DisallowNull,
SchemaIdGenerationHandling = SchemaIdGenerationHandling.TypeName,
SchemaLocationHandling = SchemaLocationHandling.Definitions,
SchemaReferenceHandling = SchemaReferenceHandling.Objects,
};
JSchema schema = schemaGenerator.Generate(typeof(Setting));
Renders:
{
"id": "Setting",
"definitions": {
"SubSetting": {
"id": "SubSetting",
"type": "object",
"properties": {
"SubSubSetting": {
"id": "SubSubSetting",
"type": "object",
"properties": {
"String": {
"type": "string"
}
}
}
}
},
"SubSubSetting": {
"$ref": "SubSubSetting"
}
},
"type": "object",
"properties": {
"SubSetting": {
"$ref": "SubSetting"
},
"SubSubSetting": {
"$ref": "SubSubSetting"
}
}
}
Thus, SubSubSetting definition is placed inline SubSetting definition and later we have SubSubSetting defined as reference to that inline definition. That's what I want to avoid as for complex data structures it becomes really obscure and I want to provide the schema even as part of a living, auto-generated documentation based on data annotations and JsonProperty.
How can I accomplish this using JSchemaGenerator?
Maybe I shouldn't do this, but as a second very short question: Are those $ref syntactically correct? Shouldn't they look like "#/definitions/SubSetting"?
The latest version of Json.NET Schema (3.0.3) has been updated to fix this issue. SubSubSetting will contain the full definition and not just a $ref.
https://github.com/JamesNK/Newtonsoft.Json.Schema/releases/tag/3.0.3