jq rename nested json key and place under same key - jq

I want to rename nested json key and keep under same schema.
Input:
"properties":
{
"name": "Ram",
"age": "17",
"department": "Sony"
}
Code : jq '[.["properties.company"] = .properties.department| del(.properties.department)]' file
Output:
"properties":
{
"name": "Ram",
"age": "17"
}
"properties.company" = "Sony"
Expected Output should be :
"properties":
{
"name": "Ram",
"age": "17",
"company": "Sony"
}
What I am doing wrong ?

You have several options, but for that you need to have a valid input JSON such as:
{
"properties": {
"name": "Ram",
"age": "17",
"department": "Sony"
}
}
With del
This assigns the new property first, then deletes the old one.
.properties | .company = .department | del(.department)
Constructing a new object with {}
This is building a new object from scratch, copying all existing properties and renaming as needed.
.properties |= { name, age, company: .department }
Merging objects
This is a combination of the first two approaches. The property is deleted, while it is being renamed in the new object. Both objects are ultimately merged to form the result.
.properties |= del(.department) + { company: .department }
Changing key name with with_entries
This iterates over all key-value pairs and (re-)assigns the key name.
.properties |= with_entries(.key |= if . == "department" then "company" else . end)
or if you dislike ifs: same as above, but uses select(cond) // alternative to select an alternative value unless the condition is met.
.properties |= with_entries(.key |= (select(. != "department") // "company"))

Related

Query Sum of Nested Array-Length COSMOS DB

Assume this model:
Having a collection of Writers documents, each Writer has some Posts. and each Post contains an array of Comments.
JSON:
{
"id": "1",
"partitionKey": "somePK",
"name": "John",
"posts": [
{
"id": "20",
"title": "post1",
"comments": [
{
"body": "some body"
}
]
},
{
"id": "21",
"title": "post2",
"comments": [
{
"body": "some new body"
}
]
}
]
}
I need a query that returns the following output:
[
{
"WriterName": "John",
"WriterCommentsCount": 2
}
]
I managed to get the writer's comment but I have a problem getting the name(or other properties) beside the WriterCommentsCount. Any idea of how to get the writer's name?
this is what I've tried so far (only the writer's comments count)
Along your thought, if we execute sql
SELECT c.name,array_length(post.comments) as countC FROM c join post in c.posts where c.id = '1'
we'll get response like below:
[
{
"name": "John",
"countC": 1
},
{
"name": "John",
"countC": 3
},
{
"name": "John",
"countC": 0
}
]
So using this SELECT c.name,sum(array_length(post.comments)) as countC FROM c join post in c.posts where c.id = '1' will return error 'Property reference 'c.name' is invalid'. The solution is using 'group by c.name'.
My answer in the comment and the one from #404 are all the same in this point, the difference is that I followed your thought using sum function and he uses count(1) after expand all the child item. My sql costs '3.09 RUs' and his costs the same on my test data.
Here's the situation screenshot.
The easiest is making a flat structure where you have an entry for each comment. After that you can use GROUP BY to group all entries for the same name. Try the following:
SELECT
c.name AS WriterName,
COUNT(1) AS WriterCommentsCount
FROM c
JOIN s IN c.posts
JOIN t IN s.comments
WHERE c.id = '1'
GROUP BY c.name

Rename a stream of JSON entities

I have a stream of JSON entities. Here is one of them called 'info':
null
null
{
"id": "qwefhu214o",
"number": "2346",
"date": "28.01.2019"
}
null
null
{
"id": "esg324lif",
"number": "1378",
"date": "29.05.2019"
}
{
"id": "gwrs853sdf",
"number": "4487",
"date": "20.12.2019"
}
I want to extract keys from nested json, so it looks like this:
null
null
{
"info_id": "qwefhu214o",
"info_number": "2346",
"info_date": "28.01.2019"
}
null
null
{
"info_id": "esg324lif",
"info_number": "1378",
"info_date": "29.05.2019"
}
{
"info_id": "gwrs853sdf",
"info_number": "4487",
"info_date": "20.12.2019"
}
I try this, but it doesn't work:
jqr::jq('.[].info |= with_entries(.key |= "info_" + .)')
Its says:
Error: lexical error: invalid char in json text.
NA
(right here) ------^
I guess it because of NULLs. How could i do that? Should i put somewhere "?" in code ? That code works for cases when there are np NULLs
The "nested JSON" shown is actually a stream of JSON entities,
for which the following filter will produce the desired results:
if . then with_entries(.key |= "info_" + .) else . end
You can easily modify this filter as required.

DocumentDB SQL where Array of Objects matches an array of strings

reference: DocumentDB SQL with ARRAY_CONTAINS
Question: Is there a better and more efficient query for what is happening below?
In the above reference question, the UDF was written to check if an object in the array had a match to the passed in string. In this variant, I am passing into the UDF an array of strings.
Now I have a working O(N^2) version that I would hope CosmosDB had a more efficient solution for.
function ScopesContainsNames(scopes, names){
var s, _i,_j, _ilen, _jLen;
for (_i = 0, _ilen = scopes.length; _i < _ilen; _i++) {
for (_j = 0, _jLen = names.length; _j < _jLen; _j++) {
s = scopes[_i];
n = names[_j];
if (s.name === n) {
return true;
}
}
}
return false;
}
My QUERY looks like this.
SELECT * FROM c WHERE udf.ScopesContainsNames(c.scopes, ["apples", "strawberries", "bananas"])
The following is an example of my Document:
{
"scopes": [
{
"name": "apples",
"displayName": "3048b61e-06d8-4dbf-a4ab-d4c2ba0a8943/a"
},
{
"name": "bananas",
"displayName": "3048b61e-06d8-4dbf-a4ab-d4c2ba0a8943/a"
}
],
"enabled": true,
"name": "dc1e4c12-95c1-4b7f-bf27-f60f0c29bf52/a",
"displayName": "218aea3d-4492-447e-93be-2d3646802ac6/a",
"description": "4aa62367-7421-4fb6-88c7-2699c9c309dd/a",
"userClaims": [
"98988d5b-38b5-400c-aecf-da57d2b66433/a"
],
"properties": {
"437d7bab-a4fb-4b1d-b0b9-f5111d01882a/a": "863defc1-c177-4ba5-b699-15f4fee78ea5/a"
},
"id": "677d4a49-a46c-4613-b3f6-f390ab0d013a",
"_rid": "q6I9AOf180hJAAAAAAAAAA==",
"_self": "dbs/q6I9AA==/colls/q6I9AOf180g=/docs/q6I9AOf180hJAAAAAAAAAA==/",
"_etag": "\"00000000-0000-0000-1ede-f2bc622201d5\"",
"_attachments": "attachments/",
"_ts": 1560097098
}
If i don't misunderstanding your requirement,you need to search the results where any name property of scopes array is included by the ["apples", "strawberries", "bananas"].
No need to use udf, please see the sample documents i made as below:
Using sql:
SELECT distinct c.scopes FROM c
join fruit in c.scopes
where Array_contains(["apples", "strawberries", "bananas"],fruit.name,false)
Result:

Copy struct with nested maps [duplicate]

This question already has answers here:
How to deep copy a map and then clear the original?
(6 answers)
Closed 4 years ago.
What am I trying to do?
Copy a "default" struct into a new one when needed, keeping all it's values.
Details
I am trying to copy a Chat struct:
type ChatData struct {
User map[string]map[string]string `json:"user"`
Chat map[string]string `json:"chat"`
}
type Chat struct {
Settings map[string]map[string]interface{} `json:"settings"`
Data ChatData `json:"data"`
}
I only need to do this when a new chat is introduced, and I check for membership in a map[string]Chat.
//data is a map[string]Chat
if _, exists := data[event.Chat]; !exists {
data[event.Chat] = data["default"]
}
The full default struct is:
{
"default": {
"settings": {
"eightball": {
"messages": [
"yes",
"no",
"maybe"
]
},
"main": {
"blacklistedCommands": [],
"blacklistedUsers": [],
"error": "there was an error - ",
"maxConsecutive": 5,
"permissions": "You don't have permissions for that command.",
"success": "The command was successful.",
"whitelistedCommands": [],
"whitelistedUsers": []
}
},
"data": {
"user": {
"default": {
"admin": "false",
"consecutiveCommands": "0",
"nickname": "",
"sentMessages": "0"
},
"total": {
"admin": "false",
"consecutiveCommands": "0",
"nickname": "",
"sentMessages": "0"
}
},
"chat": {
"commandSender": "",
"lastMessage": "",
"lastSender": "",
"lastTimestamp": "",
"wasCommand":""
}
}
}
}
What have I tried
data[event.Chat] = data["default"]
// causes a reference
data[event.Chat] = &data["default"]
// cannot use &data["default"] (type *Chat) as type Chat in assignment
data[event.Chat] = *data["default"]
// invalid indirect of data["default"] (type Chat)
Do I need to change my use of map[string]Chat to map[string]*Chat and go with the second option? Pointers and references are not my specialty, so help would be appreciated.
Edit
whoever thought I was copying a map, what are you smoking?
I have found in previous cases, that an easy (though not the most efficient) way to copy a complex structure is to Marshal it, and then Unmarshal it into a new variable. This can be done with a variety of encodings. Two easy ones (included in the stdlib) would be json and gob.
There are also plenty of libraries using reflection to achieve a similar goal: https://github.com/jinzhu/copier
But like I said, though not as efficient, this is easy to reason about and takes just a single, simple function to achieve. If performance matters you should prefer gob over json. If performance REALLY matters, then you should prefer another solution altogether.
If you know the data structure it is best for you to fully define your structs. It will be much easier to manage than complex maps and interfaces with reflection to cast data types. Here is a tool to help you convert JSON to Go structs. It isn't perfect and you have to confirm the data types. You may even want to rip out the internal structs and create types for them for readability and maintenance, but it will save you some time.
https://mholt.github.io/json-to-go/
package main
import (
"encoding/json"
"fmt"
"log"
)
var rawDataExample = []byte(`{
"default": {
"settings": {
"eightball": {
"messages": [
"yes",
"no",
"maybe"
]
},
"main": {
"blacklistedCommands": [],
"blacklistedUsers": [],
"error": "there was an error - ",
"maxConsecutive": 5,
"permissions": "You don't have permissions for that command.",
"success": "The command was successful.",
"whitelistedCommands": [],
"whitelistedUsers": []
}
},
"data": {
"user": {
"default": {
"admin": "false",
"consecutiveCommands": "0",
"nickname": "",
"sentMessages": "0"
},
"total": {
"admin": "false",
"consecutiveCommands": "0",
"nickname": "",
"sentMessages": "0"
}
},
"chat": {
"commandSender": "",
"lastMessage": "",
"lastSender": "",
"lastTimestamp": "",
"wasCommand":""
}
}
}
}
`)
type Settings struct {
Default struct {
Settings struct {
Eightball struct {
Messages []string `json:"messages"`
} `json:"eightball"`
Main struct {
BlacklistedCommands []string `json:"blacklistedCommands"`
BlacklistedUsers []string `json:"blacklistedUsers"`
Error string `json:"error"`
MaxConsecutive int `json:"maxConsecutive"`
Permissions string `json:"permissions"`
Success string `json:"success"`
WhitelistedCommands []string `json:"whitelistedCommands"`
WhitelistedUsers []string `json:"whitelistedUsers"`
} `json:"main"`
} `json:"settings"`
Data struct {
User struct {
Default struct {
Admin string `json:"admin"`
ConsecutiveCommands string `json:"consecutiveCommands"`
Nickname string `json:"nickname"`
SentMessages string `json:"sentMessages"`
} `json:"default"`
Total struct {
Admin string `json:"admin"`
ConsecutiveCommands string `json:"consecutiveCommands"`
Nickname string `json:"nickname"`
SentMessages string `json:"sentMessages"`
} `json:"total"`
} `json:"user"`
Chat struct {
CommandSender string `json:"commandSender"`
LastMessage string `json:"lastMessage"`
LastSender string `json:"lastSender"`
LastTimestamp string `json:"lastTimestamp"`
WasCommand string `json:"wasCommand"`
} `json:"chat"`
} `json:"data"`
} `json:"default"`
}
type Data struct {
Events map[string]Settings
}
func main() {
var settings Settings
err := json.Unmarshal(rawDataExample, &settings)
if err != nil {
log.Fatal(err)
}
event := "foo"
d := Data{
Events: make(map[string]Settings),
}
if _, ok := d.Events[event]; !ok {
// event doesn't exist
// add it
d.Events[event] = settings
fmt.Println("event added")
}
if _, ok := d.Events[event]; !ok {
// event exist, this will never be happen
// sanity check
fmt.Println("this will never be printed")
}
fmt.Printf("%+v\n", d)
}
Super simple fix, just wanted to get some input before I did it. Changed the map[string]Chat > map[string]*Chat and made a new *Chat, then copied the data.
data[event.Chat] = &Chat{}
*data[event.Chat] = *data["default"]

Get a tree like structure out of path string

I am stuck since 2 days, as I am not to firm with pointers and recursion. I have an array of path like structures, lets say:
s:=[]string {
"a/b/c",
"a/b/g",
"a/d",
}
With a data structure like this:
type Node struct {
Name string `json:"name"`
Children []Node `json:"children"`
}
I would like to end up with something like this:
{
"name": "a",
"children": [
{
"name": "b",
"children": [
{
"name": "c",
"children": []
},
{
"name": "g",
"children": []
}
]
},
{
"name": "d",
"children": []
}
]
}
I tried to build it with a recursion, which works kind of fine, but only for one string (e.g. "a/b/c"), as soon as I try to implement something which should add missing nodes ("g" in "a/b/g") to a tree I am stuck.
I had something like:
func appendChild(root Node, children []string) Node {
if len(children) == 1 {
return Node{children[0], nil}
} else {
t := root
t.Name=children[0]
t.Children = append(t.Children, appendChild(root, children[1:]))
return t
}
}
Could someone point me to an efficient solution?
https://play.golang.org/p/9pER5cwChF
func AddToTree(root []Node, names []string) []Node {
if len(names) > 0 {
var i int
for i = 0; i < len(root); i++ {
if root[i].Name == names[0] { //already in tree
break
}
}
if i == len(root) {
root = append(root, Node{Name: names[0]})
}
root[i].Children = AddToTree(root[i].Children, names[1:])
}
return root
}
Example output (note that I used omitempty on the children field, because I don't like null entries in my JSONs):
[{
"name": "a",
"children": [{
"name": "b",
"children": [{
"name": "c"
}, {
"name": "g"
}]
}, {
"name": "d"
}]
}]
Notable difference from your version:
It operates on a list of nodes instead of the children of a single node. This is important, as your version assumes that all of the trees have the same single root node (a), when this might not be the case. The only way to handle that in your version is to have a "fake" node at the root.
It does NOT reuse the input node. This is one of the primary problems with your code. If len(children) > 1, you update the input node's name, append to it's children, then recurse. This means that every prior level of the slice becomes part of the children. You need to create a new node instead.
It actually searches the tree. You're not searching the tree to see if the item being inserted already exists, so you duplicate nodes (specifically, node b)

Resources