I'm hoping to be able to analyze structured data stored in a custom dimension of a custom telemetry event emitted to application insights, and getting some weird behavior. It seems like the JSON can't be parsed normally, but if I pass it through strcat it is able to parse the json just fine.
customEvents
| where name == "PbConfigFilterComponentSaved"
| take 1
| project
jsonType=gettype(customDimensions.Json),
parsedType=gettype(parse_json(customDimensions.Json)),
strcatType=gettype(strcat('', customDimensions.Json)),
strcatParsedType=gettype(parse_json(strcat('', customDimensions.Json)))
Result:
jsonType: string
parsedType: string
strcatType: string
strcatParsedType: dictionary
Is there a better approach to getting parse_json to work on this kind of value?
Update
In case it's in any way relevant, here's the value of customDimensions.Json:
{"filterComponentKey":"CatalystAgeRange","typeKey":"TemporalConstraint","uiConfig":{"name":"Age","displayMode":"Age"},"config":{"dateSelector":"pat.BirthDTS"},"disabledForScenes":false,"disabledForFilters":false}
Could you please demonstrate a sample record that isn't parsed correctly?
Speculating (before seeing the data): Have you verified the final paragraph here doesn't apply to your case?
It is somewhat common to have a JSON string describing a property bag in which one of the "slots" is another JSON string. […] In such cases, it is not only necessary to invoke parse_json twice, but also to make sure that in the second call, tostring will be used. Otherwise, the second call to parse_json will simply pass-on the input to the output as-is, because its declared type is dynamic.
The type of customDimensions is dynamic and so accessing a property like customDimensions.json from it will return a string typed as dynamic.
You have to explicitly cast it as string and then parse it:
todynamic(tostring(customDimensions.json)).property
I think the "Notes" section in the documentation is exactly the issue, as mentioned by Yoni L. in the previous answer.
Related
I have a JSON list where one of the attributes of each element happens to be a JSON itself. It comes from a poor design upfront, but here it is.
I want to query the distinct attributes inside the JSON string contained in the elements.
Here is an example, just one item. I hand-wrote the code, but believe me that is valid JSON in production by the way it's generated
[{
"extraData": "{\"foo\":\"bar\"}"
}]
I would like to query for something like $.*.extraData.foo, but obviously such syntax does not work.
I am using IntelliJ IDEA Jsonpath evaluator.
The syntax should be something like parse($.*.extraData).*.foo
This article suggests me that no such operator is available to parse a JSON inside a JSON
It has to be JSONPath only for data analysis purposes. In Java, I use Jackson to parse the extraData object as a JsonNode, but my goal is to explore the large data set, and perhaps obtain some distinct values I want to use for enumeration purposes.
To JSON Path, the embedded JSON is just a string. The best you could do with a single operation is use a RegEx in the expression selector, but getting RegEx to identify JSON is really tricky, and that's if your implementation even supports RegEx.
I think the best option would be to use two paths:
The first path gets the embedded JSON. $.*.extraData
Parse that value
The second part gets the data you need. $.foo
This requires some extra code, but I think it's your only realistic option.
I am using Application Insights Log-Explorer query window to visualize the below query.
Inside the field customDimensions.RemotePC I am string a json payload.
When I try to index the stored json via propery-indexing, i get the value as null. I tried to access it as array that turns null as well.
Could you please help me to access the FirstName property in below diagram.
Try this:
| extend todynamic(tostring(rpc)).FirstName
I believe that the issue is that rpc is string (although it looks as json). Thus you need to "cast" it to dynamic. You first need to tell the compiler though that this is a string value.
I have made a gridx grid that uses a JsonRest Memory store from the dojo framework
http://dojotoolkit.org/reference-guide/1.10/dojo/store/JsonRest.html
the issue is I do not know how to pull out the sort parameter from the query string.
The url being formatted from the JsonRest call is
/admin/sales?sort(+DealershipName)
using the following statement gives me a null error
String sort = Request.QueryString["sort"].ToString();
Looking at the debugger I see the following (I need more rep to post images :( )
ok I can see that the following variables hold this value.
Request.QueryString = {sort(+DealershipName)}
type : System.Collections.Specialized.NameValueCollection
{System.Web.HttpValueCollection}
but the array is null.
I'm thinking I can do two thing. Parse the string myself or overload the dojo JsonRest Memory store. Parsing the string seems easier but if anyone has any idea or knows any libraries that can help me out. I would greatly appreciate it.
dojo/store/JsonRest has a sortParam property that you can set to the name of a standard query parameter to use instead of sort(...) (which it uses by default to avoid colliding with any standard query parameters).
For example, adding sortParam: 'sort' to the properties passed to the JsonRest constructor will result in the query string including sort=+DealershipName instead.
http://dojotoolkit.org/reference-guide/1.10/dojo/store/JsonRest.html#sorting
If the + also presents a problem, you can also override ascendingPrefix to be an empty string (''). Note that descending sort will still be indicated by a leading - (controllable via descendingPrefix).
This is a somewhat philosophical issue. I have a .net (but could be any platform) based helper library that parses query string values. Take for example a variable that returns an Int32: my framework has an option that specifies whether this value is required or optional. If it is required but not provided, the framework throws an exception. If it is optional and not specified, it returns a null.
Now an edge case has come up based on users hacking (in a good way) our urls. If they specify a variable with either an invalidly formatted Int32 ("&ID=abc") or provide the variable but not specify a value ("&id="), should the framework throw an exception or should it return a null?
Part of me feels that invalid variables or formats should return a null. It might be valid to argue that even if the parameter is optional, an invalidly formatted query string or value should still throw an exception.
Thoughts?
Since this is philophical ...
On something like an ID, I would agree with Shawn that it is a 404, especially if you are thinking in terms of state. There is no object, so not found. But, ID may not tie directly to a resource in all cases.
If the item is truly optional, a null is okay. But optional should mean "if present it makes the call more specific" in this case and there should always be a fallback. I don't see this in ID, unless the ID is keyed to an optional part of the page.
In the long run, I think you should look at the business reason for the page and what each variable means.
I believe that if a variable is optionaly, providing the variable but not specifying the value is equivalent to ommitting the variable itself. In this case, returning null seems OK.
However, providing an invalidly formatted value ought to cause an Exception, since the intent was to provide a value. In this case the user ought to be notified through some sort of validation mechanism.
A HttpException of 404 (Not Found). Your web application framework should know how to catch these errors and redirect to the proper page.
This is actually a not found error because the resources that the ID is pointing to does not exist.
I suspect there's no "right" answer to your question. If I were a developer using your library, I would expect/hope that the public API would include in its code comments, a description of how the function behaves when the URL param includes bad (wrong type) data.
You might also be able to craft your public API to get the best of both worlds: .NET seems to have adopted the "Parse" / "TryParse" approach in many places. If I'm the caller and I want the function to throw if given invalid data, I call Parse(). If I don't want it to throw, I call TryParse(). In my opinion, that is a nice pattern to follow with your API as well.
I have a document based on a xml structure that I have stored in a CouchDB database.
Some of the keys contains namespaces and are on the form "namespace:key":
{"mykey": {"nested:key": "nested value"}}
In the map function, I want to emit the nested value as a key, but the colon inside the name makes it hard...
emit(doc.mykey.nested:key, doc) <-- will not work.
Does anyone know how this can be solved?
A hint that its all just JSON and JavaScript got me some new ideas for searching.
It may be that colon in json keys ain't valid, but I found a way. By looking at the doc object as an hash, I can access my value in the following manner:
Doc.mykey['nested:key']
It works - for now...
That's because Couch is a JSON based document DB, and doc.mykey.nested:key is not a valid JSON identifier. JSON identifiers must match JavaScripts identifiers, and : is not a valid identifier character.
So, the simple answer is: "No, this won't and can't work". You need to change your identifiers.
Actually, I should qualify that.
Couch can use pretty much ANYTHING for it's views et al, and, in theory, works with any payload. But out of the box, it's just JavaScript and JSON.