I have a file which contains below details and it's repeats multiple times. I want to extract the Countryid and History value from below text file
The output should be like
Countryid: 0115 History: 20220621
Could you please help how can I extract above string from this text file using Unix script.
{
"Music": "1410",
"Countryid": "0115",
"History": "20220621",
"Legend": "/api/legacysbo/bondue",
"Sorting": "/api/dmplus/test",
"Nick": "hinduja",
"Scenario": [
"K",
"A",
"S",
"F",
"D"
]
},
{
"Music": "1466",
"Countryid": "1312",
"History": "20221012",
"Legend": "/api/legacysbo/grenob",
"Sorting": "/api/dmplus/prod",
"Nick": "Grenoble",
"Scenario": [
"K",
"A",
"S",
"F",
"D"
]
},
If this is valid json, one solution is to use a proper json parsing tool, such as jq, e.g.
cat test.json
[{
"Music": "1410",
"Countryid": "0115",
"History": "20220621",
"Legend": "/api/legacysbo/bondue",
"Sorting": "/api/dmplus/test",
"Nick": "hinduja",
"Scenario": ["K", "A", "S", "F", "D"]
}, {
"Music": "1466",
"Countryid": "1312",
"History": "20221012",
"Legend": "/api/legacysbo/grenob",
"Sorting": "/api/dmplus/prod",
"Nick": "Grenoble",
"Scenario": ["K", "A", "S", "F", "D"]
}]
jq -r '.[] | "Countryid: \(.Countryid) History: \(.History)"' < test.json
Countryid: 0115 History: 20220621
Countryid: 1312 History: 20221012
Assumptions:
OP wants to print all Countryid/History pairs
the Countryid entry may come before or after History entry
each Countryid entry must have a matching History entry
all sets have a Music entry that comes before the Countryid/History entries
One awk idea:
awk -F '"' '
function print_pair() {
if (countryid && history) # if both variables are non-empty then ...
printf "Countryid: %s History: %s\n", countryid, history
countryid=history="" # reset variables
}
$2 == "Music" { print_pair() } # print previous pair to stdout
$2 == "Countryid" { countryid=$4 }
$2 == "History" { history=$4 }
END { print_pair() } # print last pair to stdout
' input.dat
If we can assume that Countryid always comes before History then we can reduce the code to:
awk -F '"' '
$2 == "Countryid" { countryid=$4 }
$2 == "History" { printf "Countryid: %s History: %s\n", countryid, $4 }
' input.dat
Both of these generate:
Countryid: 0115 History: 20220621
Countryid: 1312 History: 20221012
Related
Given the following input:
{
"text": "a1\nb2"
}
How do I get the following output:
[
{
"letter": "a",
"number": 1
},
{
"letter": "b",
"number": 2
}
]
I've tried using capture with the "g" flag, but this yields two documents instead of a single document with an array of captured inputs:
$ echo '{
"text": "a1\\nb2"
}' | jq '.text | capture("(?<letter>[a-z])(?<number>[0-9])";"g")'
{
"letter": "a",
"number": "1"
}
{
"letter": "b",
"number": "2"
}
Here is a link to the jqplay example.
Why not just wrap the capture in a new array:
.text | [ capture("(?<letter>[a-z])(?<number>[0-9])";"g") ]
JqPlay
Given this input small sample:
{
"_format_version": "1.1",
"_workspace": "test",
"services": [
{
"connect_timeout": 60000,
"host": "host-name-test.com",
"name": "name-of-service",
"path": "/test/oauthpass",
"port": 777,
"protocol": "http",
"read_timeout": 1000,
"retries": 1,
"write_timeout": 1000,
"routes": [
{
"hosts": [
"Google.com"
],
"name": "com.testing.active.oauth",
"methods": [
"POST"
],
"paths": [
"/vendors/otest/pass/?$"
],
"path_handling": "v8",
"preserve_host": false,
"protocols": [
"https"
],
"regex_priority": 0,
"strip_path": true,
"https_redirect_status_code": 426,
"request_buffering": true,
"response_buffering": true
}
]
}
}
trying to get a listing from the data pulling certain values like the listing below:
host-name-test.com, Google.com, POST, HTTPS
the command that I have working so far is
cat /tmp/petecar.json | jq -r ' .services[] | .routes[] | ( .hosts[] + "/" + .paths[]) ' | more
but I can't access the values under services, please provide some sample on how to get the values
routes has an array value and as such cannot be concatenated with a string. You can use join to turn that array into a single string:
jq -r '.services[] | .host + " " + (.routes[].hosts | join(","))'
Output:
host-name-test.com Google.com
Alternatively, using string interpolation) which will automatically serialize any values into their string representation:
jq -r '.services[] | "\(.host) \(.routes[].hosts)"'
Output:
host-name-test.com ["Google.com"]
join and string interpolation can be combined, giving you the identical output of the first command:
jq -r '.services[] | "\(.host) \(.routes[].hosts|join(","))"'
How to get a city name specifying the name of a language with the additional condition of "spoken" or "perhaps"?
{
"Paris": {
"language": {
"fr": "spoken",
"en": "perhaps"
}
},
"London": {
"language": {
"en": "spoken",
"fr": "perhaps",
"ru": "unused"
}
},
"Moscow": {
"language": {
"ru": "spoken",
"en": "perhaps",
"fr": "unused"
}
}
}
E.g:
Input: en;
Output: Paris, London, Moscow
Input: fr;
Output: Paris, London
Input: ru;
Output: Moscow
Convert all entries to an object with key/value fields using to_entries, retain only those objects that do match your conditions (in the .value's object .language the value of a field $lang provided as input variable using --arg equals any one of "spoken" or "perhaps") and output the original entry's .key.
jq -r --arg lang "fr" '
to_entries[]
| select([.value.language[$lang] == ("spoken","perhaps")] | any)
| .key
' input.json
Try it at jqplay.org
With the following inputs:
# file1.json
{
"uid": "1",
"name": "jack"
}
{
"uid": "2",
"name": "jill"
}
# file2.json
{
"fid": "a",
"file": "sample1.txt",
"uid": "1"
}
{
"fid": "b",
"file": "sample2.txt",
"uid": "1"
}
{
"fid": "c",
"file": "sample3.txt",
"uid": "2"
}
How do I go about inserting the name key-value pair to the object in the file2.json. The output I'm trying to get is as follows:
{
"fid": "a",
"file": "sample1.txt",
"uid": "1",
"name": "jack"
}
{
"fid": "b",
"file": "sample2.txt",
"uid": "1",
"name": "jack"
}
{
"fid": "c",
"file": "sample3.txt",
"uid": "2",
"name": "jill"
}
Solutions posted on merge json objects with jq and join two json files based on common key with jq utility or alternative way from command line both seems to only return the last matching pair. See below.
{"uid":"1","name":"jack","fid":"b","file":"sample2.txt"}
{"uid":"2","name":"jill","fid":"c","file":"sample3.txt"}
You will need to "slurp" file1.json, e.g. by invoking jq as follows:
jq -n -f merge.jq --slurpfile file1 file1.json file2.json
where merge.jq contains:
INDEX($file1[]; .uid) as $dict
| inputs
| . + $dict[.uid]
def INDEX
If your jq does not have INDEX/2, then simply add its def:
def INDEX(stream; idx_expr):
reduce stream as $row ({}; .[$row|idx_expr|tostring] = $row);
I want to parse terraform.tfstate (where openstack provider is used), to return instance name and it's internal + floating IP (if assigned).
First select what we are interested in:
jq -r '.modules?[]|.resources[]?|select(.type == "openstack_compute_floatingip_v2", .type == "openstack_compute_instance_v2")' < terraform.tfstate
For simplicity, pre-parsed example with the above part (one FIP and one instance):
{
"type": "openstack_compute_floatingip_v2",
"depends_on": [
"openstack_networking_router_interface_v2.management"
],
"primary": {
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"attributes": {
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"instance_id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"pool": "public",
"region": "RegionOne"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
{
"type": "openstack_compute_instance_v2",
"depends_on": [
"openstack_compute_floatingip_v2.management",
"openstack_compute_secgroup_v2.ssh_only",
"openstack_networking_network_v2.management"
],
"primary": {
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"attributes": {
"access_ip_v4": "10.10.10.5",
"access_ip_v6": "",
"all_metadata.%": "1",
"all_metadata.habitat": "sup",
"availability_zone": "nova",
"flavor_id": "eb36e84e-17c1-42ab-b359-4380f6f524ae",
"flavor_name": "m1.large",
"force_delete": "false",
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"image_id": "c574aeed-e47c-4fb7-9da0-75550b76ee56",
"image_name": "ubuntu-16.04",
"key_pair": "vault-etcd_test_tf",
"metadata.%": "1",
"metadata.habitat": "sup",
"name": "ctl01",
"network.#": "1",
"network.0.access_network": "false",
"network.0.fixed_ip_v4": "10.10.10.5",
"network.0.fixed_ip_v6": "",
"network.0.floating_ip": "",
"network.0.mac": "02:c6:61:f9:ee:7e",
"network.0.name": "management",
"network.0.port": "",
"network.0.uuid": "f2468669-e321-4eb4-9ede-003e362a8988",
"region": "RegionOne",
"security_groups.#": "1",
"security_groups.1845949017": "vault-etcd_test_ssh_only",
"stop_before_destroy": "false"
},
"meta": {
"e2bfb730-ecaa-11e6-8f88-34363bc7c4c0": {
"create": 1800000000000,
"delete": 1800000000000,
"update": 1800000000000
}
},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
Required is to take from "type": "openstack_compute_floatingip_v2" replace .primary.attributes.address and .fixed_ip and from corresponding .instance_id the .name.
So, sth like:
{"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"}
Well, I came with an idea while using walk, but miss how to actually assign the proper value from corresponding instance id:
jq -r "$(cat floating.jq)" terraform.tfstate
floating.jq:
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
.modules?[]|.resources[]?|select(.type ==
"openstack_compute_floatingip_v2", .type ==
"openstack_compute_instance_v2")|
.primary|walk( if type == "object" and .attributes.address then
.attributes.instance_id |= "REFERRED VALUE HERE") else . end)
Let's assume the two related objects are in a file named two.json. Then one way to merge the information from both objects is using the -s command-line option, e.g.
jq -s '
(.[0].primary.attributes | {address, fixed_ip})
+ {name: .[1].primary.attributes.name}' two.json
Output
With your example input, the output would be:
{
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"
}