Reshaping loosely structured JSON (field names stored in array) - jq

Question

I have a unique problem I have been struggling to get my head wrapped around. I have a bit of JSON from an API. The field names are actually stored in .headerData.methodNames and most of the values are stored in .lightValueObjects[].data.
{
"headerData": {
"objectTypeName": "device",
"methodNames": [
"name",
"accessIp",
"deviceType",
"version",
"discoverTime",
"discoverMethod",
"status",
"department",
"description",
"perfMonStatus",
"eventLogStatus",
"maintenance",
"location",
"agentStatus",
"policyName",
"policyId",
"agentType",
"templateIds",
"agentId",
"decommission",
"accountId",
"instanceId",
"agentVersion",
"parserName"
]
},
"lightValueObjects": [
{
"objectId": 17657,
"custId": 1,
"parentId": null,
"collectorId": null,
"data": [
"super",
"192.168.128.222",
"leader",
null,
1637092568000,
"LOG",
1,
"Super",
null,
"",
"Normal",
"",
null,
"",
"",
"",
null,
"",
null,
false,
"",
"",
null,
""
],
"extData": null,
"naturalId": "lead%2dsuper",
"aoSys": false
},
{
"objectId": 883252,
"custId": 1,
"parentId": null,
"collectorId": null,
"data": [
"fw01",
"192.168.128.1",
"fw",
"ANY",
1637098725000,
"LOG",
1,
"Super",
null,
"",
"Normal",
"",
null,
"",
"",
"",
null,
"",
null,
false,
"",
"",
null,
""
],
"extData": null,
"naturalId": "fw01",
"aoSys": false
}
],
"errCode": 0,
"errObj": null,
"dataType": "Device",
"totalCount": 0
}
The following code produces better output, but it's not quite perfect.
.headerData.methodNames as $header | [.lightValueObjects[].data] | map(
. as $o |
reduce .[] as $item({}; ($o | index($item)) as $index |
.[$header[$index]] = (if $item == "" then null else $item end))
)
[
{
"name": "super",
"accessIp": "192.168.128.222",
"deviceType": "leader",
"version": null,
"discoverTime": 1637092568000,
"discoverMethod": "LOG",
"status": 1,
"department": "Super",
"perfMonStatus": null,
"eventLogStatus": "Normal",
"decommission": false
},
{
"name": "fw01",
"accessIp": "192.168.128.1",
"deviceType": "fw",
"version": "ANY",
"discoverTime": 1637098725000,
"discoverMethod": "LOG",
"status": 1,
"department": "Super",
"description": null,
"perfMonStatus": null,
"eventLogStatus": "Normal",
"decommission": false
}
]
Current Problems
There should be 24 fields returned per device, but only 11 are being shown.
I have not been able to figure out how to merge the named values in .lightValueObjects to the final output.
Here is an example of what I am trying to make the output show, (To keep things simple, I am only showing 1 device)
[
{
"name": "super",
"accessIp": "192.168.128.222",
"deviceType": "leader",
"version": null,
"discoverTime": 1637092568000,
"discoverMethod": "LOG",
"status": 1,
"department": "Super",
"description": null,
"perfMonStatus": null,
"eventLogStatus": "Normal",
"maintenance": null,
"location": null,
"agentStatus": null,
"policyName": null,
"agentType": null,
"templateIds": null,
"agentId": null,
"decommission": false,
"accountId": null,
"instanceId": null,
"agentVersion": null,
"parserName": null,
"objectId": 17657,
"custId": 1,
"parentId": null,
"collectorId": null,
"extData": null,
"naturalId": "lead%2dsuper",
"aoSys": false
}
]
Any help would be GREATLY appreciated!!

Answer 1

I don't know if I got all your alignments right, but give this a try:
.headerData.methodNames as $header
| .lightValueObjects | map(
( [$header, .data]
| transpose
| map({(first): last})
| add
) + del(.data)
)
[
{
"name": "super",
"accessIp": "192.168.128.222",
"deviceType": "leader",
"version": null,
"discoverTime": 1637092568000,
"discoverMethod": "LOG",
"status": 1,
"department": "Super",
"description": null,
"perfMonStatus": "",
"eventLogStatus": "Normal",
"maintenance": "",
"location": null,
"agentStatus": "",
"policyName": "",
"policyId": "",
"agentType": null,
"templateIds": "",
"agentId": null,
"decommission": false,
"accountId": "",
"instanceId": "",
"agentVersion": null,
"parserName": "",
"objectId": 17657,
"custId": 1,
"parentId": null,
"collectorId": null,
"extData": null,
"naturalId": "lead%2dsuper",
"aoSys": false
},
{
"name": "fw01",
"accessIp": "192.168.128.1",
"deviceType": "fw",
"version": "ANY",
"discoverTime": 1637098725000,
"discoverMethod": "LOG",
"status": 1,
"department": "Super",
"description": null,
"perfMonStatus": "",
"eventLogStatus": "Normal",
"maintenance": "",
"location": null,
"agentStatus": "",
"policyName": "",
"policyId": "",
"agentType": null,
"templateIds": "",
"agentId": null,
"decommission": false,
"accountId": "",
"instanceId": "",
"agentVersion": null,
"parserName": "",
"objectId": 883252,
"custId": 1,
"parentId": null,
"collectorId": null,
"extData": null,
"naturalId": "fw01",
"aoSys": false
}
]
Demo
Edit: If you want to replace empty strings "" with null, as you did in your approach, replace last from my solution with either (last | select(. != "") // null) or, more explicitly, with (last | if . == "" then null else . end).