I am converting a nested list of a specific structure (required by an API) to JSON using toJSON() in jsonlite. However, I need the final JSON to not contain the outer square brackets (also required by an API).
test_list <- list(
list(formName = "test_title", user_id = "test_userid", rows = list(list(row = 0))),
list(formName = "test_title2", user_id = "test_userid2", rows = list(list(row = 0)))
)
jsonlite::toJSON(test_list, pretty = TRUE, auto_unbox = TRUE)
Which gives:
[
{
"formName": "test_title",
"user_id": "test_userid",
"rows": [
{
"row": 0
}
]
},
{
"formName": "test_title2",
"user_id": "test_userid2",
"rows": [
{
"row": 0
}
]
}
]
But I need to remove the first and last square bracket.
I can use purrr::flatten() to remove the top level of the list and thus the square brackets in the JSON, but then toJSON() doesn't seem to like that the list has duplicate names, and renames them to name.1, name.2, name.3 etc (which also isn't allowed by the API).
That is:
jsonlite::toJSON(test_list %>% purrr::flatten(), pretty = TRUE, auto_unbox = TRUE)
Which removes the outer square brackets, but converts the names in the second element to formName.1, user_id.1, rows.1, like so:
{
"formName": "test_title",
"user_id": "test_userid",
"rows": [
{
"row": 0
}
],
"formName.1": "test_title2",
"user_id.1": "test_userid2",
"rows.1": [
{
"row": 0
}
]
}
But this is what I need:
{
"formName": "test_title",
"user_id": "test_userid",
"rows": [
{
"row": 0
}
],
"formName": "test_title2",
"user_id": "test_userid2",
"rows": [
{
"row": 0
}
]
}
That is, formName, user_ud and rows in the second JSON element are not appended with ".1"
Any advice would be greatly appreciated!
This seems to work (ugly but simple):
df = data.frame(a=1,b=2)
js = toJSON(unbox(fromJSON(toJSON(df))))
> js
{"a":1,"b":2}
for some reason, auto_unbox=T does not work:
> toJSON(df,auto_unbox = T)
[{"a":1,"b":2}]
Just edit the JSON. You can do it with gsub, or with stringr. If you use stringr functions, it will lose it's "json" class, but you can put it back:
> x = jsonlite::toJSON(test_list %>% purrr::flatten(), pretty = TRUE, auto_unbox = TRUE)
> gsub("user_id\\.1", "user_id", x)
{
"formName": "test_title",
"user_id": "test_userid",
"rows": [
{
"row": 0
}
],
"formName.1": "test_title2",
"user_id": "test_userid2",
"rows.1": [
{
"row": 0
}
]
}
> y = stringr::str_replace_all(x, "user_id\\.1", "user_id")
> class(y) = "json"
> y
{
"formName": "test_title",
"user_id": "test_userid",
"rows": [
{
"row": 0
}
],
"formName.1": "test_title2",
"user_id": "test_userid2",
"rows.1": [
{
"row": 0
}
]
}
I'll leave it to you to write appropriate regex to make the substitutions you want.
Related
Example json data:
{
"data": [
{
"place": "FM346",
"id": [
"7_day_A",
"7_day_B",
"7_day_C",
"7_day_D"
],
"values": [
0,
30,
23,
43
]
},
{
"place": "LH210",
"id": [
"1_day_A",
"1_day_B",
"1_day_C",
"1_day_D"
],
"values": [
4,
45,
100,
9
]
}
]
}
what i need to transform it into:
{
"data": [
{
"place": "FM346",
"7_day_A": {
"value": 0
},
"7_day_B": {
"value": 30
},
"7_day_C": {
"value": 23
},
"7_day_D": {
"value": 43
}
},
{
"place": "LH210",
"1_day_A": {
"value": 4
},
"1_day_B": {
"value": 45
},
"1_day_C": {
"value": 100
},
"1_day_D": {
"value": 9
}
}
]
}
i have tried this:
{
data:[.data |.[]|
{
place: (.place),
(.id[]):
{
value: (.values[])
}
}]
}
(in jqplay: https://jqplay.org/s/f4BBtN9gwmp)
and this:
{
data:[.data |.[]|
{
place: (.place),
test:
[{
(.id[]):
{
value: (.values[])
}
}]
}]
}
(in jqplay: https://jqplay.org/s/pKIvQe1CzgX)
but they arent grouped in the way i wanted and it gives each value to each id, not the corresponding one.
I have been trying for some time now, but im new to jq and have no idea how to transform it this way, thanks in advance for any answers.
You can use transpose here, which can play a key role in converting the arrays to key/value pairs
.data[] |= {place} +
([ .id, .values ] | transpose | map({(.[0]): { value: .[1] } }) | add)
The solution works by converting the array-of-arrays [.id, .values] by transposing them, i.e. converting
[["7_day_A","7_day_B","7_day_C","7_day_D"],[0,30,23,43]]
[["1_day_A","1_day_B","1_day_C","1_day_D"],[4,45,100,9]]
to
[["7_day_A",0],["7_day_B",30],["7_day_C",23],["7_day_D",43]]
[["1_day_A",4],["1_day_B",45],["1_day_C",100],["1_day_D",9]]
With the transformation done, we construct an object with key as the zeroth index element and value as an object comprising of the value of first index element, and combine the results together with add
Demo - jqplay
Here is the jq I have, it just wants to build a new element and then append it to an array,
[.[] | . as { foo: $foo1, bar: $bar1} |
{
names: ([
$foo1 | range(0;length) as $i |
{ key: ($foo1[$i]) }
] + [{ key: $bar1 }])
} |
{
values: .names,
}
]
And suppose I have a json like this,
{
"foo":[
"key1",
"key2"
],
"bar": "key3"
}
This will generate a json file like this,
[
{
"values": [
{
"key": "key1"
},
{
"key": "key2"
},
{
"key": "key3"
}
]
}
]
But the element should be only appended when the $bar is not an empty string, can I do something like this?
[.[] | . as { foo1: $foo1, bar1: $bar1 if $bar != ""}
...
or do it when is appended,
names: ([
$foo1 | range(0;length) as $i |
{ key: ($foo1[$i]) }
] + [{ key: $bar1 }] | if $bar != "")
thanks in advance for any help!
You could just use select to filter out that case
jq '[{values: (.foo + [.bar | select(. != "")]) | map({key:.})}]'
If .bar == "key3", it prints
[
{
"values": [
{
"key": "key1"
},
{
"key": "key2"
},
{
"key": "key3"
}
]
}
]
If .bar == "", it prints
[
{
"values": [
{
"key": "key1"
},
{
"key": "key2"
}
]
}
]
If .bar does not exist, it will print
[
{
"values": [
{
"key": "key1"
},
{
"key": "key2"
},
{
"key": null
}
]
}
]
If in this last case you want to have the same result as with .bar == "", then change . != "" to values != "" in the filter to consider only values that are not null (or to strings != "" to only consider (non-empty) strings and disregard any other type).
I have a json file pulled from an HPE iLO interface with the snmp configuration. It looks like:
[
{
"Comments": {
"BIOSDate": "01/23/2021",
"BIOSFamily": "U30",
"Manufacturer": "HPE",
"Model": "ProLiant DL380 Gen10",
"SerialNumber": "5UNESX378",
"iLOVersion": "iLO 5 v2.65"
}
},
{
"#HpeiLOSnmpService.v2_3_0.HpeiLOSnmpService": {
"/redfish/v1/Managers/1/SnmpService/": {
"#odata.context": "/redfish/v1/$metadata#HpeiLOSnmpService.HpeIloSnmpService",
"#odata.id": "/redfish/v1/Managers/1/SnmpService",
"Actions": {
"#HpeIloSnmpService.SendSNMPTestAlert": {
"target": "/redfish/v1/Managers/1/SnmpService/Actions/HpeILOSnmpService.SendSNMPTestAlert/"
}
},
"AlertDestinationAssociations": [
{
"SNMPAlertProtocol": "SNMPv3Trap",
"SecurityName": null
}
],
"AlertDestinations": [
"1.2.3.4",
"5.6.7.8",
null,
null
],
"AlertsEnabled": true,
"Name": "SnmpService"
},
"PeriodicHSATrapConfig": "Disabled",
"ReadCommunities": [
"",
"",
""
],
"Role": "",
"RoleDetail": "",
"SNMPAlertDestinations": {
"#odata.id": "/redfish/v1/Managers/1/SnmpService/SNMPAlertDestinations/"
},
"SNMPUsers": {
"#odata.id": "/redfish/v1/Managers/1/SnmpService/SNMPUsers/"
},
"SNMPv1Enabled": false,
"SNMPv3EngineID": "0x8920000000E3028329E002033",
"SNMPv3InformRetryAttempt": 2,
"SNMPv3InformRetryIntervalSeconds": 15,
"Status": {
"State": "Enabled"
},
"TrapCommunities": [
"",
"",
"",
"",
"",
"",
""
],
"TrapSourceHostname": "Manager",
"Users": [
{
"AuthProtocol": "MD5",
"PrivacyProtocol": "DES",
"SecurityName": "",
"UserEngineID": null
},
{
"AuthProtocol": "MD5",
"PrivacyProtocol": "DES",
"SecurityName": "",
"UserEngineID": null
},
{
"AuthProtocol": "SHA",
"PrivacyProtocol": "AES",
"SecurityName": "oneview_4849283d97929392",
"UserEngineID": null
},
{
"AuthProtocol": "MD5",
"PrivacyProtocol": "DES",
"SecurityName": "",
"UserEngineID": null
}
]
}
}
]
I want to select an element in the Users array that has SecurityName set to "" and change that element. I don't need the Comments portion. So, I try to select the section starting with #HpeiLOSnmpService.v2_3_0.HpeiLOSnmpService with:
jq -r '.[] | .#HpeiLOSnmpService.v2_3_0.HpeiLOSnmpService' snmp.json
but it gives me everything without the enclosing array. Anyone have a suggestion?
Thanks!
# starts a comment and your jq program degenerates to .[]|. which is identical to the program .[] (|. is a no-op/the identity filter). This program will simply select all values from the input.
You must quote certain characters, such as #, when they are part of propery names. The following will work with your JSON file:
jq -r '.[] | ."#HpeiLOSnmpService".v2_3_0.HpeiLOSnmpService'
Thanks. Using the quotes around the key with a '#' works. Ultimately, selecting the SecurityName that was unset was done with:
jq -r '.Users[] | select (.SecurityName == "") | {"AuthProtocol":.AuthProtocol,"PrivacyProtocol":.PrivacyProtocol,"SecurityName":.SecurityName,"UserEngineID":.UserEngineID}'
I am trying to convert a data.frame into JSON text. How can I include an array in this data.frame? I.e. the variable characteristics needs to contain multiple elements. I do not succeed in this.
Here is my code:
data <- data.frame(sequenceNbr = c(1:3),
requestTypeCode = "MR001",
managementStartingDate = format(Sys.Date(),"%Y-%m-%d"),
TopicCode = "TH02",
Nbr = c("1760448", "6580364", "1391363"),
TypeCode = "R003008",
char1 = "CK001001", char2 = "CK001002",
stringsAsFactors = FALSE) %>%
group_by(sequenceNbr) %>%
nest( Assignment=c(TopicCode),
entity = c(Nbr),
ToControl = c(TypeCode),
characteristics = c(char1,char2)
)
data <- data %>%
mutate( Assignment = purrr::map(Assignment, as.list),
entity = purrr::map(entity, as.list))
json <- jsonlite::toJSON(list(`DefList`=data),
auto_unbox = TRUE, pretty = TRUE)
This gives as output
{
"DefList": [
{
"sequenceNbr": 1,
"requestTypeCode": "MR001",
"managementStartingDate": "2020-06-16",
"Assignment": {
"TopicCode": "TH02"
},
"entity": {
"Nbr": "1760448"
},
"ToControl": [
{
"TypeCode": "R003008"
}
],
"characteristics": [
{
"char1": "CK001001",
"char2": "CK001002"
}
]
},
{
"sequenceNbr": 2,
"requestTypeCode": "MR001",
"managementStartingDate": "2020-06-16",
"Assignment": {
"TopicCode": "TH02"
},
"entity": {
"Nbr": "6580364"
},
"ToControl": [
{
"TypeCode": "R003008"
}
],
"characteristics": [
{
"char1": "CK001001",
"char2": "CK001002"
}
]
}
]
}
What I want as output is this (there is a difference in the characteristics-element):
{
"DefList": [
{
"sequenceNbr": 1,
"requestTypeCode": "MR001",
"managementStartingDate": "2020-06-16",
"Assignment": {
"TopicCode": "TH02"
},
"entity": {
"Nbr": "1760448"
},
"ToControl": [
{
"TypeCode": "R003008"
}
],
"characteristics": [
"CK001001",
"CK001002"
]
},
{
"sequenceNbr": 2,
"requestTypeCode": "MR001",
"managementStartingDate": "2020-06-16",
"Assignment": {
"TopicCode": "TH02"
},
"entity": {
"Nbr": "6580364"
},
"ToControl": [
{
"TypeCode": "R003008"
}
],
"characteristics": [
"CK001001",
"CK001002"
]
}
]
}
It should also work when characteristics contains only one value. Any idea's? I think the data.frame structure is the limiting factor, since it cannot contain an array. I see in this post : https://blog.exploratory.io/working-with-json-data-in-very-simple-way-ad7ebcc0bb89 that it is correct JSON syntax, but I do not manage to obtain it. Thanks in advance for your help.
I have the following json file:
{
"actions": [
{
"values": "test",
"features": [
{
"v1": 100,
"v2": {
"dates": [
"2020-04-08 06:58:26",
"2020-04-08 06:58:26"
]
}
}
]
}
]
}
I would like to append n-times the object within the "actions" array to the end of it, creating n+1 total objects.
Expected output if n=2:
{
"actions": [
{
"values": "test",
"features": [
{
"v1": 100,
"v2": {
"dates": [
"2020-04-08 06:58:26",
"2020-04-08 06:58:26"
]
}
}
]
},
{
"values": "test",
"features": [
{
"v1": 100,
"v2": {
"dates": [
"2020-04-08 06:58:26",
"2020-04-08 06:58:26"
]
}
}
]
},
{
"values": "test",
"features": [
{
"v1": 100,
"v2": {
"dates": [
"2020-04-08 06:58:26",
"2020-04-08 06:58:26"
]
}
}
]
}
]
}
I found this answer [How can I duplicate an existing object within a JSON array using jq? however it only works with one element at the end.
You can just use a reduce() function with range() together to create the index to include the object at.
jq --arg n 2 'reduce range(0, ($n|tonumber)) as $d (.; .actions[$d+1] += .actions[0] )' json