Jq Replace Existing JSON with Values Complex - jq

I need to generate a CloudFormation parameter list (config.json) during a CodeBuild. I have a bash script in my repo that will take a couple of parameters. This will either be production, staging, or development.
Here's the generator.json. The values here will be used to generate config.json.
{
"Parameters" : {
"FargateStackSuffix" : "environment-fargate",
"VPCStackSuffix": "environment-base-vpc",
"ContainerPort" : "80",
"ContainerCpu" : "256",
"ContainerMemory" : "512",
"Path" : "*",
"productionDesiredCount" : "3",
"stagingDesiredCount" : "2",
"developmentDesiredCount" : "1",
"ELBType" : "application",
"ELBIpAddressType": "ipv4",
"productionZone": "service.example.com",
"stagingZone": "service-staging.example.com",
"devZone": "service-dev.example.com"
}
}
For example:
./generate.sh my-service production
Would generate this:
{
"Parameters" : {
"FargateStackSuffix" : "production-fargate",
"VPCStackSuffix": "production-base-vpc",
"ServiceName" : "myservice",
"EnvironmentName" : "production",
"ContainerPort" : "80",
"ContainerCpu" : "256",
"ContainerMemory" : "512",
"Path" : "*",
"DesiredCount" : "3",
"ELBType" : "application",
"ELBIpAddressType": "ipv4",
"Zone": "myservice.example.com"
}
}
As you can see there's a bit of replacement based on additions. Just iterating over the keys won't do this. Is there a way to do all fot eh transformations I need with just jq?
EDIT: I ended up using some sed to filter replace a few values. Now I need to replace the appropriate DesiredCount:
cat .codedeploy/generator.json | jq '[paths(type == "string" and contains("DesiredCount"))]'
[]
My problem is that it's returning an empty array.

You can use object construction to do this.
cat generator.json | jq --arg variable $variable '{Parameters: {DesiredCount: .Parameters.developmentDesiredCount, Zone: .Parameters.devZone, environmentName: "Production", var: $variable}}'
Here I have shown a few solutions to help you.
you can reference the parameters in generator.json by using dot notation in the value for that field in the new object.
You can simply define a string
You can use the args flag to pass in a bash variable.
Output (incomplete, but shows solution)
{
"Parameters": {
"DesiredCount": "1",
"Zone": "service-dev.example.com",
"environmentName": "Production",
"var": "VariableString"
}
}

Here's one such way you can generate that json. Perhaps the tricky part here is generating the dynamic name for the environment DesiredCount and Zone. But jq handles that rather easily.
$ jq --arg ServiceName "myservice" --arg EnvironmentName "production" '.Parameters |=
{
FargateStackSuffix,
VPCStackSuffix,
$ServiceName,
$EnvironmentName,
ContainerPort,
ContainerCpu,
ContainerMemory,
Path,
DesiredCount: ."\($EnvironmentName)DesiredCount",
ELBType,
ELBIpAddressType,
Zone: ."\($EnvironmentName)Zone"
}
' generator.json > config.json
$ cat config.json
{
"Parameters": {
"FargateStackSuffix": "environment-fargate",
"VPCStackSuffix": "environment-base-vpc",
"ServiceName": "myservice",
"EnvironmentName": "production",
"ContainerPort": "80",
"ContainerCpu": "256",
"ContainerMemory": "512",
"Path": "*",
"DesiredCount": "3",
"ELBType": "application",
"ELBIpAddressType": "ipv4",
"Zone": "service.example.com"
}
}

Related

Parse an array of json object using jq

I am trying to parse the below json file and store the result into another json file. How do I achieve this ?
{
"Objects": [
{
"ElementName": "Test1",
"ElementArray": ["abc","bcd"],
"ElementUnit": "4"
},
{
"ElementName": "Test2",
"ElementArray": ["abc","bcde"],
"ElementUnit": "8"
}
]
}
Expected result :
{
"Test1" :[
"abc","bcd"
],
"Test2" :[
"abc","bcde"
]
}
I've tried something on the lines of the below but I seem to be off -
jq '[.Objects[].ElementName ,(.Objects[]. ElementArray[])]' user1.json
jq ".Objects[].ElementName .Objects[].ElementArray" ruser1.json
Your expected output needs to be wrapped in curly braces in order to be a valid JSON object. That said, use from_entries to create an object from an array of key-value pairs, which can be produced by accordingly mapping the input object's Objects array.
.Objects | map({key: .ElementName, value: .ElementArray}) | from_entries
{
"Test1": [
"abc",
"bcd"
],
"Test2": [
"abc",
"bcde"
]
}
Demo
Demo
https://jqplay.org/s/YbjICOd8EJ
You can also use reduce
reduce .Objects[] as $o ({}; .[$o.ElementName]=$o.ElementArray)

Json use Jq to parse, read and return true if enrty found

Apologies if this is basic but the doc for jq is not so good
i have this json:
{
"jsonrpc": "2.0",
"result": [{
"hostid": "10084",
"host": "Zabbix server",
"interfaces": [{
"interfaceid": "1",
"ip": "127.0.0.1"
}]
}, {
"hostid": "10336",
"host": "AUTO",
"interfaces": [{
"interfaceid": "4",
"ip": "1.2.3.4"
}]
}, {
"hostid": "10337",
"host": "AUTOSERVER",
"interfaces": [{
"interfaceid": "5",
"ip": "4.5.6.7"
}]
}, {
"hostid": "10348",
"host": "Server00001",
"interfaces": [{
"interfaceid": "16",
"ip": "4.5.6.7"
}]
}],
"id": 2
}
i need to find a way to use jq to find if "Server0001" exists in one of the hosts
i know i can use grep but i prefer using jq here, like select..
any help or ref toa good doc would be much appriciated
any (see the manual) can return a boolean value if a condition matches with at least one item.
jq 'any(.result[]; .host == "Server0001")'
false
Demo
jq 'any(.result[]; .host == "Server00001")'
true
Demo
You may also want to use the some parameters when invoking jq (see the manual): The --arg option, for instance, lets you add a variable which can be initialized from outside the filter string. And with the -e (or --exit-status) flag you can have jq set the exit status according to the filter's final result. Together, this enables you to use jq like this:
if jq --arg host "Server0001" -e 'any(.result[]; .host == $host)';
then
…
else
…
fi

How to change "key" and "value" keys of to_entries to "ParameterKey" and "ParameterValue" in jq?

I have 3 json files that when merged using * produces the following result:
$ jq -s '.[0] * .[1] * .[2] | to_entries' a.json b.json c.json
[
{
"key" : "...",
"value" : "..."
},
{
"key" : "...",
"value" : "..."
},
{
"key" : "...",
"value" : "..."
}
]
How can I use different names for "key" and "value", e.g.:
[
{
"ParameterKey" : "...",
"ParameterValue" : "..."
},
{
"ParameterKey" : "...",
"ParameterValue" : "..."
},
{
"ParameterKey" : "...",
"ParameterValue" : "..."
}
]
Create a new object and assign the fields correspondigly
… | .[] |= {ParameterKey: .key, ParameterValue: .value}
Demo
Instead of the update operator |= you could also use map
… | map({ParameterKey: .key, ParameterValue: .value})
Demo
Depending on the actual structure of your three input files, this can probably be simplified using reduce instead.
Also, if you want to import the actual replacement names from outside jq (as their sample names suggest), use the --arg parameter to create an externally initialized variable and use it when creating the objects
jq -s --arg key "ParameterKey" --arg value "ParameterValue" '
… {($key):.key, ($value):.value} …
'

JQ to filter only vaule of id

the following is the JSON data. need to get only of id key
{apps:[ {
"id": "/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd",
"cmd": null,
"args": null,
"user": null,
"env": {},
"constraints": [
[
"hostname",
"GROUP_BY",
"5"
]
},
{
"id": "/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd",
"cmd": null,
"args": null,
"user": null,
"env": {},
"constraints": [
[
"hostname",
"GROUP_BY",
"5"
]
]},
output expected is
/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
Thanks in advance
After fixing the errors in your JSON, we can use the following jq filter to get the desired output:
.apps[] | .id
JqPlay Demo
Result jq -r '.apps[] | .id':
/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
You can use map() to create an array from the properties of the objects. Try this:
let data = {apps:[{"id":"/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd","cmd":null,"args":null,"user":null,"env":{},"constraints":["hostname","GROUP_BY","5"]},{"id":"/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd","cmd":null,"args":null,"user":null,"env":{},"constraints":["hostname","GROUP_BY","5"]}]}
let ids = data.apps.map(o => o.id);
console.log(ids);
Note that I corrected the invalid brace/bracket combinations in the data structure you posted in the question. I assume this is just a typo in that example, otherwise there would be parsing errors in the console.

jq: pass variable argument to be used as filter [duplicate]

This question already has answers here:
jq: pass string argument without quotes
(3 answers)
Closed 4 years ago.
How do I pass a variable argument to JQ program that will be used as a filter. Since by default --arg passes the argument as a a string wrapped with quotes the same cannot be used to apply a filter.
here is the JQ program that finds a particular path in the given json and adds a static key value to that path but doesn't work because of the quotes issue.
--argjson name '{ "pattern": "XYZ"}' 'def p: "." + (paths | select(.[-1] == "p-enum") | .[0:-1] | join(".")) ; .|p += $name' sample.json
here is the sample json
{
"type": "object",
"description": "Contains information.",
"properties": {
"type": {
"description": "Type.",
"type": "string",
"p-enum": [
{
"value": "IND",
"description": "Ind."
},
{
"value": "PROP",
"description": "Prop."
}
]
}
}
}
Based on how I interpreted how you were using jq in your other question, it depends on how complicated your filter will be. Any argument that is to be interpreted by jq is not the way you should approach it. This is the equivalent of using eval() and is not only unsupported, but just not a good way to approach this.
If you're simply accessing a property of the input, you have a couple of ways using simple indexing or using getpath/1 for nested paths.
# indexing
# { "properties": ... }
$ jq --arg p 'properties' '.[$p]' input.json
# using getpath
# { "foo": { "bar": ... } }
$ jq --argjson path '["foo","bar"]' 'getpath($path)' input.json

Resources