I need to delete multiple keys at once from some JSON (using jq), and I'm trying to learn if there is a better way of doing this, than calling map and del every time. Here's my input data:
test.json
[
{
"label": "US : USA : English",
"Country": "USA",
"region": "US",
"Language": "English",
"locale": "en",
"currency": "USD",
"number": "USD"
},
{
"label": "AU : Australia : English",
"Country": "Australia",
"region": "AU",
"Language": "English",
"locale": "en",
"currency": "AUD",
"number": "AUD"
},
{
"label": "CA : Canada : English",
"Country": "Canada",
"region": "CA",
"Language": "English",
"locale": "en",
"currency": "CAD",
"number": "CAD"
}
]
For each item, I want to remove the number, Language, and Country keys. I can do that with this command:
$ cat test.json | jq 'map(del(.Country)) | map(del(.number)) | map(del(.Language))'
That works fine, and I get the desired output:
[
{
"label": "US : USA : English",
"region": "US",
"locale": "en",
"currency": "USD"
},
{
"label": "AU : Australia : English",
"region": "AU",
"locale": "en",
"currency": "AUD"
},
{
"label": "CA : Canada : English",
"region": "CA",
"locale": "en",
"currency": "CAD"
}
]
However, I'm trying to understand if there is a jq way of specifying multiple labels to delete, so I don't have to have multiple map(del()) directives?
You can provide a stream of paths to delete:
$ cat test.json | jq 'map(del(.Country, .number, .Language))'
Also, consider that, instead of blacklisting specific keys, you might prefer to whitelist the ones you do want:
$ cat test.json | jq 'map({label, region, locale, currency})'
There is no need to use both map and del.
You can pass multiple paths to del, separated by commas.
Here is a solution using "dot-style" path notation:
jq 'del( .[] .Country, .[] .number, .[] .Language )' test.json
doesn't require quotation marks (which you may feel makes it more readable)
doesn't group the paths (requires you to retype .[] once per path)
Here is an example using "array-style" path notation, which allows you to combine paths with a common prefix like so:
jq 'del( .[] ["Country", "number", "Language"] )' test.json
Combines subpaths under the "last-common ancestor" (which in this case is the top-level list iterator .[])
peak's answer uses map and delpaths, though it seems you can also use delpaths on its own:
jq '[.[] | delpaths( [["Country"], ["number"], ["Language"]] )]' test.json
Requires both quotation marks and array of singleton arrays
Requires you to put it back into a list (with the start and end square brackets)
Overall, here I'd go for the array-style notation for brevity, but it's always good to know multiple ways to do the same thing.
A better compromise between "array-style" and "dot-style" notation mentioned in by Louis in his answer.
del(.[] | .Country, .number, .Language)
jqplay
This form can also be used to delete a list of keys from a nested object (see russholio's answer):
del(.a | .d, .e)
Implying that you can also pick a single index to delete keys from:
del(.[1] | .Country, .number, .Language)
Or multiple:
del(.[2,3,4] | .Country,.number,.Language)
You can delete a range using the range() function (slice notation doesn't work):
del(.[range(2;5)] | .Country,.number,.Language) # same as targetting indices 2,3,4
Some side notes:
map(del(.Country,.number,.Language))
# Is by definition equivalent to
[.[] | del(.Country,.number,.Language)]
If the key contains special characters or starts with a digit, you need to surround it with double quotes like this: ."foo$", or else .["foo$"].
This question is very high in the google results, so I'd like to note that some time in the intervening years, del has apparently been altered so that you can delete multiple keys with just:
del(.key1, .key2, ...)
So don't tear your hair out trying to figure out the syntax work-arounds, assuming your version of jq is reasonably current.
In addition to #user3899165's answer, I found that to delete a list of keys from "sub-object"
example.json
{
"a": {
"b": "hello",
"c": "world",
"d": "here's",
"e": "the"
},
"f": {
"g": "song",
"h": "that",
"i": "I'm",
"j": "singing"
}
}
$ jq 'del(.a["d", "e"])' example.json
delpaths is also worth knowing about, and is perhaps a little less mysterious:
map( delpaths( [["Country"], ["number"], ["Language"]] ))
Since the argument to delpaths is simply JSON, this approach is particularly useful for programmatic deletions, e.g. if the key names are available as JSON strings.
Related
I am new to jq and can't seem to quite get the syntax right for what I want to do. I am executing a command and piping its JSON output into jq. The structure looks like this:
{
"timestamp": 1658186185,
"nodes": {
"x3006c0s13b1n0": {
"Mom": "x3006c0s13b1n0.hsn.cm",
"Port": 15002,
"state": "free",
"pcpus": 64,
"resources_available": {
"arch": "linux",
"gputype": "A100",
"host": "x3006c0s13b1n0",
"mem": "527672488kb",
"ncpus": 64,
"ngpus": 4,
"system": "polaris",
"tier0": "x3006-g1",
"tier1": "g1",
"vnode": "x3006c0s13b1n0"
},
"resources_assigned": {},
"comment": "CHC- Offlined due to node health check failure",
"resv_enable": "True",
"sharing": "default_shared",
"license": "l",
"last_state_change_time": 1658175652,
"last_used_time": 1658175652
},
And so on with a record for each node. In psuedocode, what I want to do is this:
if state is not free then display nodename : {comment = "Why is the node down"}
The nodename is the key, but could be extracted from a field inside the record. However, for future reference, I would like to understand how to get the key. I figured out (I think) that you can't use == on strings, but instead have to use the regex functions.
This gives me the if state is not free part:
<stdin> | jq '.nodes[] | .state | test("free") | not'
This gives me an object with the Mom (which includes the key) and the comment:
jq '.nodes[] | {Mom: .Mom, comment: .comment}'
The question is how do I put all that together? And as for the keys, this gives me a list of the keys: jq '.nodes | keys' but that uses the non-array version of nodes.
One way without touching the keys would be to only select those array items that match the condition, and map the remaining items' value to the comment itself using map_values:
jq '.nodes | map_values(select(.state != "free").comment)'
{
"x3006c0s13b1n0": "CHC- Offlined due to node health check failure"
}
Keeping the whole comments object, which is closer to your desired output, would be similar:
jq '.nodes | map_values(select(.state != "free") | {comment})'
{
"x3006c0s13b1n0": {
"comment": "CHC- Offlined due to node health check failure"
}
}
Accessing the keys directly is still possible though. You may want to have a look at keys, keys_unsorted or to_entries.
I want to go from
[
{"key_skjdghkbs": "deep house"},
{"key_kjsskjbgs": "deadmau5"},
{"key_jhw98w4hl": "progressive house"},
{"key_sjkh348vg": "swedish house mafia"},
{"key_js3485jwh": "dubstep"},
{"key_jsg587jhs": "escape"}
]
to
{
"key_skjdghkbs": "deep house",
"key_kjsskjbgs": "deadmau5"
"key_jhw98w4hl": "progressive house",
"key_sjkh348vg": "swedish house mafia",
"key_js3485jwh": "dubstep",
"key_jsg587jhs": "escape"
}
Each object in the original list has exactly one key but the keys are unique.
I could do something like jq .[] .genre if the keys were the same but they're not.
jq's add function does exactly this
jq 'add'
Try this (assuming your file is named so72297039.json):
jq '[.[] | to_entries] | flatten | from_entries' < so72297039.json
(Edit: OP edited question, so here's relevant answer)
Since duplicate keys are not possible (see other answer) you can use to format like this:
{
"artist": [
"deadmau5",
"swedish house mafia"
],
"genre": [
"deep house",
"progressive house",
"dubstep"
],
"song": [
"escape"
]
}
with a jq call like this:
jq '
map(to_entries)
| flatten
| group_by(.key)
| map({key: first.key, value: map(.value)})
| from_entries
' input.json
If the keys artist, genre, song are known in advance an easier to understand expression can be used.
Using code below to pull data from a local json file.
The file is very large and is nested with objects and arrays. There are multiple objects in the .ratings[] that I would like to extract.
How can I use the pipe operator in the .ratings[] array so that I don't have to retype .ratings[] for each piece of data that I would like to pull?
jq -r '.players[] | [.firstName,.lastName,.tid,.pid,.ratings[].spd,.ratings[].jmp] | join(", ")'
You can enclose it in () to use the pipe sign:
.players[] | [.firstName, .lastName, .tid, .pid, (.ratings[] | .spd, .jmp)] | join(", ")
Try it online
You didn't specify the expected output, so it is not clear if your proposed solution gives you the output you want.
Given the following input:
{
"players": [
{
"firstName": "fname1",
"lastName": "lname1",
"tid": "tid1",
"pid": "pid1",
"ratings": [
{
"spd": "spd1-1",
"jmp": "jmp1-1"
}
]
},
{
"firstName": "fname2",
"lastName": "lname2",
"tid": "tid2",
"pid": "pid2",
"ratings": [
{
"spd": "spd2-1",
"jmp": "jmp2-1"
},
{
"spd": "spd2-2",
"jmp": "jmp2-2"
}
]
},
{
"firstName": "fname3",
"lastName": "lname3",
"tid": "tid3",
"pid": "pid3",
"ratings": [
{
"spd": "spd3-1",
"jmp": "jmp3-2"
},
{
"spd": "spd3-2",
"jmp": "jmp3-2"
},
{
"spd": "spd3-3",
"jmp": "jmp3-3"
}
]
}
]
}
Your solution and the answer from 0ston0 will give you 1 line per player, but a different number of columns per line:
.players[] | [.firstName,.lastName,.tid,.pid,(.ratings[]|.spd,.jmp)] | join(", ")
generates:
fname1, lname1, tid1, pid1, spd1-1, jmp1-1
fname2, lname2, tid2, pid2, spd2-1, jmp2-1, spd2-2, jmp2-2
fname3, lname3, tid3, pid3, spd3-1, jmp3-2, spd3-2, jmp3-2, spd3-3, jmp3-3
This might or might not be what want your result to look like.
A different solution will print one line per rating, but duplicate the players' names. Running:
.players[] | [.firstName,.lastName,.tid,.pid] + (.ratings[]|[.spd,.jmp]) | join(", ")
will result in:
fname1, lname1, tid1, pid1, spd1-1, jmp1-1
fname2, lname2, tid2, pid2, spd2-1, jmp2-1
fname2, lname2, tid2, pid2, spd2-2, jmp2-2
fname3, lname3, tid3, pid3, spd3-1, jmp3-2
fname3, lname3, tid3, pid3, spd3-2, jmp3-2
fname3, lname3, tid3, pid3, spd3-3, jmp3-3
Both solutions are valid for different use cases and depending on how you are going to subsequently process the data.
Apologies if this is basic but the doc for jq is not so good
i have this json:
{
"jsonrpc": "2.0",
"result": [{
"hostid": "10084",
"host": "Zabbix server",
"interfaces": [{
"interfaceid": "1",
"ip": "127.0.0.1"
}]
}, {
"hostid": "10336",
"host": "AUTO",
"interfaces": [{
"interfaceid": "4",
"ip": "1.2.3.4"
}]
}, {
"hostid": "10337",
"host": "AUTOSERVER",
"interfaces": [{
"interfaceid": "5",
"ip": "4.5.6.7"
}]
}, {
"hostid": "10348",
"host": "Server00001",
"interfaces": [{
"interfaceid": "16",
"ip": "4.5.6.7"
}]
}],
"id": 2
}
i need to find a way to use jq to find if "Server0001" exists in one of the hosts
i know i can use grep but i prefer using jq here, like select..
any help or ref toa good doc would be much appriciated
any (see the manual) can return a boolean value if a condition matches with at least one item.
jq 'any(.result[]; .host == "Server0001")'
false
Demo
jq 'any(.result[]; .host == "Server00001")'
true
Demo
You may also want to use the some parameters when invoking jq (see the manual): The --arg option, for instance, lets you add a variable which can be initialized from outside the filter string. And with the -e (or --exit-status) flag you can have jq set the exit status according to the filter's final result. Together, this enables you to use jq like this:
if jq --arg host "Server0001" -e 'any(.result[]; .host == $host)';
then
…
else
…
fi
This question already has answers here:
jq: pass string argument without quotes
(3 answers)
Closed 4 years ago.
How do I pass a variable argument to JQ program that will be used as a filter. Since by default --arg passes the argument as a a string wrapped with quotes the same cannot be used to apply a filter.
here is the JQ program that finds a particular path in the given json and adds a static key value to that path but doesn't work because of the quotes issue.
--argjson name '{ "pattern": "XYZ"}' 'def p: "." + (paths | select(.[-1] == "p-enum") | .[0:-1] | join(".")) ; .|p += $name' sample.json
here is the sample json
{
"type": "object",
"description": "Contains information.",
"properties": {
"type": {
"description": "Type.",
"type": "string",
"p-enum": [
{
"value": "IND",
"description": "Ind."
},
{
"value": "PROP",
"description": "Prop."
}
]
}
}
}
Based on how I interpreted how you were using jq in your other question, it depends on how complicated your filter will be. Any argument that is to be interpreted by jq is not the way you should approach it. This is the equivalent of using eval() and is not only unsupported, but just not a good way to approach this.
If you're simply accessing a property of the input, you have a couple of ways using simple indexing or using getpath/1 for nested paths.
# indexing
# { "properties": ... }
$ jq --arg p 'properties' '.[$p]' input.json
# using getpath
# { "foo": { "bar": ... } }
$ jq --argjson path '["foo","bar"]' 'getpath($path)' input.json