Parsing nested JSON and creating new keys with jq - jq

I have a JSON file that has hundreds of objects. Each object has 2 keys with nested object\arrays one for assignee and another for epics.
[
{
"id": 1111111111,
"assignee": {
"id": 1234,
"firstName": "John",
"lastName": "Doe",
"email": "Doe#doe.com",
},
"epics": [
{
"id": 111,
"title": "Big Code Issues",
"version": 16
}
],
"location": "Nerdville"
}
]
from the current assignee key. I want to create a two new keys assigneeId and assigneeName. Where the assigneeName is the firstName and lastName combined. The original nested assignee key can be discarded.
from the epics key I would like to create and epicsID key with a value of id from epics. The rest of the keys in the nested key can be discarded.
I am very new to jq but i would like to create following output.
[
{
"id": 1111111111,
"assigneeId": 1234,
"assingeeName": "John Doe",
"epicsid": 111,
"location": "Nerdville"
}
]
I can create and remove keys with JQ but I am have not been able to iterate the JSON file to grab values from with the nested keys and put into new keys. I have been getting errors based on different command I been running but the main one is "Cannot index array with string "epics"".
Any help would be greatly appreciated

After fixing a very small technical problem with your JSON, the following produces the required result:
map( {id,
assigneeId: .assignee.id,
assigneeName: (.assignee | .firstName + " " + .lastName),
epicsid: .epics[0].id,
location } )
You might wish to modify this if the "epics" array has more than one item in it.

Related

Convert the JSON attribute from smallcase to uppercase in XQuery and Marklogic 10

I have loaded the below JSON in MarkLogic and I need to convert the Name property value to upper case:
{ "Name": "User", "Id": "1", "date": "2022-01-01" }
Any way to convert the value to upper case in Marklogic and XQuery, looking like below format?
{ "Name": "USER", "Id": "1", "date": "2022-01-01" }
One way would be to select the Name, use upper-case() and create a text() node, and then use xdmp:node-replace().
Assuming that you have the document assigned to a variable $doc:
let $name := $doc/Name
return xdmp:node-replace($name, text {upper-case($name) } )
For reference, there are a few examples in the documentation of updating different types of properties:
https://docs.marklogic.com/guide/app-dev/json#id_60123

Kusto extractjson not working with email address

I am attempting to use the extractjson() method that includes email addresses in the source data (specifically the # symbol).
let T = datatable(MyString:string)
[
'{"user#domain.com": {"value":10}, "userdomain.com": { "value": 5}}'
];
T
| project extractjson('$.["user#domain.com"].value', MyString)
This results in a null being returned, changing the JSONPath to '$.["userdomain.com"].value' does return the correct result.
Results
I know the # sign is a used as the current node in a filter expression, does this need to be escaped when used with KQL?
Just as a side note, I run the same test using nodes 'jsonpath' package and this worked as expected.
const jp = require('jsonpath');
const data = {"user#domain.com": {"value":10}, "name2": { "value": 5}};
console.log(jp.query(data, '$["user#domain.com"].score'));
you can use the parse_json() function instead, and when you don't have to use extract_json():
print MyString = '{"user#domain.com": {"value":10}, "userdomain.com": { "value": 5}}'
| project parse_json(MyString)["user#domain.com"].value
MyString_user#domain.com_value
10
From the documentation: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/extractjsonfunction

Cosmos DB REPLACE strings in nested Object

I'm using the Cosmos Data migration tool to migrate data between environments. During my migration, I need to update the hostname of the website in the data. I was able to do this pretty easily with a query like this with the top level object data:
SELECT Farms["name"], Farms["farmerInfo"], REPLACE(Farms["websiteLink"], "thiswebsite", "newHostName") AS websiteLink FROM Farms
My Cosmos DB data is structured like (data is just for the example):
{
"name": "Red's Farm",
"websiteLink": "www.thiswebsite.com/goats/",
"farmerInfo": {
"name": "Bob",
"websiteLink": "www.thiswebsite.com/goats/",
"hasGoats": true,
"numGoats": 17
}
}
I don't actually need to modify any of the top level data. The data I need to modify is within the "farmerInfo" object. I've tried a few things but I've had no luck. How can I replace a string in this object using the SQL api?
I want the data to look like this after the migration:
{
"name": "Red's Farm",
"websiteLink": "www.thiswebsite.com/goats/",
"farmerInfo": {
"name": "Bob",
"websiteLink": "www.newHostName.com/goats/", <--- Updated data
"hasGoats": true,
"numGoats": 17
}
}
You can use a SELECT statement in your SELECT statement to build up the sub objects. As example:
SELECT
c.name,
c.websiteLink,
(
SELECT
c.farmerInfo.name,
REPLACE(c.farmerInfo.websiteLink, "thiswebsite", "newHostName") AS websiteLink
) AS farmerInfo
FROM c

JsonPath that combines fields from a complex object

I am using jsonpath_rw_ext for python currently
I am unable to come up with a path string that returns the fields I want without getting errors.
I think I can get most of what I'm describing as individual fields however I can't put then together.
json object is an array of two objects:
[
{
'id': 9707,
'rev': 16,
'fields': {
'System.AssignedTo': {
'id': 12345
'descriptor': 'some-identifier',
'displayName': 'Jeff Saremi',
},
'System.CommentCount': 7
},
{
'id': 9708,
'rev': 10,
'fields': {
'System.AssignedTo': {
'id': 56789
'descriptor': 'another-identifier',
'displayName': 'Someone Else',
},
'System.CommentCount': 2
}
]
What I want in the results is:
id (the topmost level one), fields.System.CommentCount and fields System.AssignedTo.displayName
I do not want deeper "id" levels
Here are what i have tried individually:
jp.match('$[*].id', workitems)
returns the toplevel IDs
jp.match('$..fields["System.CommentCount"]', workitems)
returns the commentcounts promptly
jp.match('$..fields..displayName', workitems)
returns the displayNames
Apparently this cannot be done not with the library I'm using.
I had to create 3 separate lists and zip then together to have one coherent resultset.

Freebase query result buried under nonsense

A beginner's question about freebase:
I am looking for the imdb id of a movie called "O". If I use the searchbox on the freebase.com website and constrain the search by type to all:/film/film, then I get a high quality result with the best match on top:
http://www.freebase.com/search?query=o&lang=en&all=%2Ffilm%2Ffilm&scoring=entity&prefixed=true
But this does not include the imdb id. When I try to recreate and refine this search using the query editor, I can't figure out how to do a "general query". The best I could come up with was doing a fuzzy name search like this:
[{
"type": "/film/film",
"name": null,
"name~=": "o",
"imdb_id": [],
"rottentomatoes_id": []
}]
The result contains exactly the information I want, but the movie "O" is only the 12th result in the list, buried under lots of nonsense:
http://www.freebase.com/query?lang=%2Flang%2Fen&q=[{%22type%22%3A%22%2Ffilm%2Ffilm%22%2C%22name%22%3Anull%2C%22name~%3D%22%3A%22o%22%2C%22imdb_id%22%3A[]%2C%22rottentomatoes_id%22%3A[]}]
How can I improve the quality of my result? What special magic does the "?query=o" use that "name~=":"o" does not have?
When you use query=o, freebase does some smart sorting of the results, display exact matches first, followed by less exact matches.
With your query name ~= o you are not searching for movies with name "O", but for movies that contain "O" in their names (the ~= operator). If you want to search for a specific movie title, then specify the exact name:
[{
"type": "/film/film",
"name": "o",
"imdb_id": [],
"rottentomatoes_id": []
}]
This will result in output:
{
"result": [{
"imdb_id": [
"tt0184791"
],
"name": "O",
"type": "/film/film",
"rottentomatoes_id": [
"o"
]
}]
}
If Search gives you the topic that you want, why not just use the output parameter to add the IMDB ID (or whatever else you want) to the output that you request it return?

Resources