What is the correct way to add tuple and key-pair values items to a DynamoDB database via Terraform?
I am trying like this:
resource "aws_dynamodb_table_item" "item" {
table_name = aws_dynamodb_table.dynamodb-table.name
hash_key = aws_dynamodb_table.dynamodb-table.hash_key
for_each = {
"0" = {
location = "Madrid"
coordinates = [["lat", "40.49"], ["lng", "-3.56"]]
visible = false
destinations = [0, 4]
}
}
item = <<ITEM
{
"id": { "N": "${each.key}"},
"location": {"S" : "${each.value.location}"},
"visible": {"B" : "${each.value.visible}"},
"destinations": {"L" : [{"N": "${each.value.destinations}"}]
}
ITEM
}
And I am getting the message:
each.value.destinations is tuple with 2 elements
│
│ Cannot include the given value in a string template: string required.
I also have no clue on how to add the coordinates variable.
Thanks!
List should be something like that :
"destinations": {"L": [{ "N" : 1 }, { "N" : 2 }]}
You are trying to pass
"destinations": {"L": [{ "N" : [0,4] }]}
Also you are missing the last } in destinations key
TLDR: I think the problem here is that you are trying to put L(N) - i.e. a list of numeric values, while your current Terraform code tries to put all the destinations into one N/number.
Instead of:
[{"N": "${each.value.destinations}"}]
you need some iteration over destinations and building a {"N": ...} of them.
"destinations": {"NS": ${jsonencode(each.value.destinations)}}
Did the trick!
Related
Is it possible to know with JSONPath that other "paths" exist?
By an existing "path" I mean a string in the form "a.b.c" or "a.b.d" like for this JSON:
{
'a' : {
'b' : [ { 'c' : 0 }, { 'd': 1 ]
}
}
Can a JSONPath be written to tell if there any other "paths" other then a.b.c and a.b.d ?
For example in the following JSON, it should find that there is the property 'e':
{
'a' : {
'b' : [ { 'c' : 0 }, { 'd': 1 ],
'e': 2
}
}
It depends on JSONPath Implementation. The jsonpath options have a setting to output the path or value.
JSONPATH
$..*
All possible Output Paths
[
"$['a']",
"$['a']['b']",
"$['a']['e']",
"$['a']['b'][0]",
"$['a']['b'][1]",
"$['a']['b'][0]['c']",
"$['a']['b'][1]['d']"
]
Tool : https://jsonpath.com/
Checkbox Option : Output paths
Tool : https://jsonpath.herokuapp.com/
Checkbox option : Normalized path expressions
I've got an Image Collection like:
ImageCollection : {
features : [
0 : {
type: Image,
id: MODIS/006/MOD11A1/2019_01_01,
properties : {
LST_Day_1km : 12345,
LST_Night_1km : 11223,
system:index : "2019-01-01",
system:asset_size: 764884189,
system:footprint: LinearRing,
system:time_end: 1546387200000,
system:time_start: 1546300800000
},
1 : { ... }
2 : { ... }
...
],
...
]
From this collection, how can I get an array of objects of specific properties? Like:
[
{
LST_Day_1km : 12345,
LST_Night_1km : 11223,
system:index : "2019-01-01"
},
{
LST_Day_1km : null,
LST_Night_1km : 11223,
system:index : "2019-01-02"
}
...
];
I tried ImageCollection.aggregate_array(property) but it allows only one parameter at one time.
The problem is that the length of "LST_Day_1km" is different from the length of "system:index" because "LST_Day_1km" includes empty values, so it's hard to combine arrays after get them separately.
Thanks in advance!
Whenever you want to extract data from a collection in Earth Engine, it is often a straightforward and efficient strategy to first arrange for the data to be present as a single property on the features/images of that collection, using map.
var wanted = ['LST_Day_1km', 'LST_Night_1km', 'system:index'];
var augmented = imageCollection.map(function (image) {
return image.set('dict', image.toDictionary(wanted));
});
Then, as you're already familiar with, just use aggregate_array to extract that property's values:
var list = augmented.aggregate_array('dict');
print(list);
Runnable complete example:
var imageCollection = ee.ImageCollection('MODIS/006/MOD11A1')
.filterDate('2019-01-01', '2019-01-07')
.map(function (image) {
// Fake values to match the question
return image.set('LST_Day_1km', 1).set('LST_Night_1km', 2)
});
print(imageCollection);
// Add a single property whose value is a dictionary containing
// all the properties we want.
var wanted = ['LST_Day_1km', 'LST_Night_1km', 'system:index'];
var augmented = imageCollection.map(function (image) {
return image.set('dict', image.toDictionary(wanted));
});
print(augmented.first());
// Extract that property.
var list = augmented.aggregate_array('dict');
print(list);
https://code.earthengine.google.com/ffe444339d484823108e23241db04629
I'm passing a JSON object to jq and want to add extra objects to an inner array ('accessories') if its parent array ('platforms') matches a certain name.
Here's my source JSON:
{
"bridge": {
"name": "Homebridge",
"port": 51395
},
"accessories": [],
"platforms": [
{
"name": "Config",
"port": 8581,
"platform": "config"
},
{
"platform": "homebridge-cbus.CBus",
"name": "CBus",
"client_ip_address": "127.0.0.1",
"accessories": [
{
"values": "existing"
}
]
}
]
}
This works beautifully:
jq '.platforms[1].accessories += [{ "values" : "NEW" }]'
... but of course it's poor form to expect platforms[1] to always the be array I want to append to, so I set about trying to form the right syntax for a search or if/then/else to only act on the .name of the appropriate one.
I thought this was my solution:
jq '.platforms[] | if ( .name=="CBus" ) then .accessories += [{ "values" : "NEW" }] else . end'
... until I realised it was only passing the 'platforms' through and eating the 'bridge' object and empty outer 'accessories' array, which I need to retain.
My issue looks to be similar to JQ | Updating array element selected by `select`, but I've tried LOTS of combinations but just can't break through.
Edit: Here's the correct JQPlay I've been working with:
https://jqplay.org/s/dGDswqAEte
Thanks for any help.
That's a good attempt. The key here is to use the select() function to identify the object you are going to update and overwrite the overall array using |= operator, i.e.
.platforms |= ( map(select(.name == "CBus").accessories += [{ "values" : "NEW" }] ) )
For the snippet in your jq-play link (now removed), you need to do
.gcp_price_list."CP-COMPUTEENGINE-OS"
|= with_entries(select(.value.cores == "shared").value.cores = "0.5")
Or if you want to be even more specific, and keep the entry in gcp_price_list configurable, do
.gcp_price_list |=
with_entries (
select(.key == "CP-COMPUTEENGINE-OS").value |=
with_entries(
select(.value.cores == "shared").value.cores = "0.5") )
I have an CSV which contains an edge list, one edge per row. It looks like this:
id1, id2, attr1, attr2, attrX, attrY, attrZ
From this, I want to be able to create (or update) the following, per row:
Vertex A of class X, with id1 and attribute attr1
Vertex B of class X, with id2 and attribute attr2
Edge A->B with edge attributes attrX, attrY, attrZ
This is the configuration file I'm feeding to oetl.sh (using OrientDB 2.2 beta2):
{
"source": { "file": { "path": "/data/sample/test.csv" } },
"extractor": { "row": {} },
"transformers" :
[
{ "csv" : {} },
{ "merge" : { "joinFieldName":"id1", "lookup":"X.id" } },
{ "vertex" : { "class" : "X", "skipDuplicates":true } },
{ "edge" : {
"unresolvedLinkAction" : "WARNING",
"class" : "EdgeTypeClass",
"joinFieldName" : "id2",
"lookup": "X.id",
"edgeFields":{"attrX":"${input.attrX}", "attrY":"${input.attrY}","attrZ":"${input.attrZ}"}
}
},
{ "field" : { "fieldNames" : [ "id1", "id2", "attr1", "attr2", "attrX", "attrY", "attrZ" ], "operation": "remove" } }
],
"loader": {
"orientdb": {
"dbURL": "remote:localhost/test2",
"dbType": "graph"
}
}
}
The sample data I used to run the test is as follows:
10,11,"A","B",100,200,1
11,12,"B","C",110,201,5
12,14,"C","D",90,250,10
14,13,"D","E",105,210,3
When I run the oetl.sh script with the given configuration and sample data, it creates 4 vertices instead of 5 and no edges. There are no attributes on the vertices at all.
So these are the questions:
Is there a way in the vertex clause to specify vertex attributes/fields the same way that one can do for edges (i.e. edgeFields)? The documentation doesn't mention anything about it but it seems odd that you wouldn't be able to do it.
Rather than relying on the edge to create the outbound vertex, should I instead be creating two vertices explicitly and if so how do I specify that in the configuration file? When I try to add two "vertex" clauses it only seems to pick up the last one as the "current" vertex.
It's possible that the specific edge (id1 -> id2) already exists. Is it possible to only update the edge attributes in this case?
My sinking feeling is that given the complexity and number of things I'm trying to pack into this that it will be simpler to write my own ETL (e.g. using the Java API) instead of relying on oetl, but I was hoping I'd be able to avoid doing that if only because it's more maintainable.
I want to write a query in Elastic that applies a filter based on values i have in an array (in my R program). Essentially the query:
Matches a time range (time field in Elastic)
Matches "trackId" field in Elastic to any value in array oth_usr
Return 2 fields - "trackId", "propertyId"
I have the following primitive version of the query but do not know how to use the oth_usr array in a query (part 2 above).
query <- sprintf('{"query":{"range":{"time":{"gte":"%s","lte":"%s"}}}}',start_date,end_date)
view_list <- elastic::Search(index = "organised_recent",type = "PROPERTY_VIEW",size = 10000000,
body=query, fields = c("trackId", "propertyId"))$hits$hits
You need to add a terms query and embed it as well as the range one into a bool/must query. Try updating your query like this:
terms <- paste(sprintf("\"%s\"", oth_usr), collapse=", ")
query <- sprintf('{"query":{"bool":{"must":[{"terms": {"trackId": [%s]}},{"range": {"time": {"gte": "%s","lte": "%s"}}}]}}}',terms,start_date,end_date)
I'm not fluent in R syntax, but this is raw JSON query that works.
It checks whether your time field matches given range (start_time and end_time) and whether one of your terms exact matches trackId.
It returns only trackId, propertyId fields, as per your request:
POST /indice/_search
{
"_source": {
"include": [
"trackId",
"propertyId"
]
},
"query": {
"bool": {
"must": [
{
"range": {
"time": {
"gte": "start_time",
"lte": "end_time"
}
}
},
{
"terms": {
"trackId": [
"terms"
]
}
}
]
}
}
}