Add key to array of objects and persist the change using jq - jq

Question

I am looping over array of s3 buckets in a json file.
[
[
"response": {
"Buckets: : [
{"Name": "foo"},
{"Name": "bar"}
]
}
]
]
I want to loop over each bucket, call the aws s3 api to get the region for each bucket and append the {"region": "region_name"} for each object inside Buckets array and persist the changes to file.
I am struggling to write the modified data to file as such that it doesn't lose all the data.
Below script writes to a temp.json file but it keeps overwriting the data for each run. In the end I only see last element in the Bucket array written to temp.json file
I want only the region key added to each element but keep all the contents of file same.
jq -r '.[][0].response.Buckets[].Name' $REPORT/s3.json |
while IFS= read -r bucket
do
region=$(aws s3api get-bucket-location --bucket $bucket | jq -r '.LocationConstraint')
jq -r --arg bucket "$bucket" --arg region "$region" '.[][0].response.Buckets[] | select(.Name==$bucket) | .region=$region' $REPORT/s3.json | sponge $REPORT/temp.json
done

Answer 1

Keep the context by adding parentheses to the LHS as in (.[][0].response.Buckets[] | select(.Name==$bucket)).region=$region.
jq -r '.[][0].response.Buckets[].Name' $REPORT/s3.json |
while IFS= read -r bucket
do
region=$(aws s3api get-bucket-location --bucket $bucket | jq -r '.LocationConstraint')
jq -r --arg bucket "$bucket" --arg region "$region" '(.[][0].response.Buckets[] | select(.Name==$bucket)).region=$region' $REPORT/s3.json | sponge $REPORT/temp.json
done
With a little bit more refactoring, you could also immediately provide the bucket name to the first inner jq call, which can already create the final objects, which can then be fed as an array to the outer jq call using --slurpfile, for instance. This would move the second inner jq call outside the loop, reducing the overall number of calls by one order.
jq --slurpfile buckets <(
jq -r '.[][0].response.Buckets[].Name' $REPORT/s3.json |
while IFS= read -r bucket; do
aws s3api get-bucket-location --bucket $bucket | jq --arg bucket "$bucket" '{Name: $bucket, region: .LocationConstraint}'
done
) '.[][0].response.Buckets = $buckets' $REPORT/s3.json | sponge $REPORT/temp.json