Is it possible to list all secondary index values previously saved in Riak bucket (leveldb, 2i enabled)? Having the following data in Riak:
curl -XPOST localhost:8098/types/indexes/buckets/users/keys/x -H 'x-riak-my_bin: a'
curl -XPOST localhost:8098/types/indexes/buckets/users/keys/y -H 'x-riak-my_bin: b'
I'd like to be able to send a query which would return me {a, b} as a result (or possibly a stream handle to the same result, if it's too big). Something like:
curl -XGET localhost:8098/types/indexes/buckets/users/index/my_bin/_
Is is possible get such information from Riak?
Note: I am NOT interested in getting the actual object keys - {x, y} in our case.
I don't believe that functionality is a provided. It seems that Riak stores the 2i values in a form like {i,Bucket,Index,Value,Key}. An ranged index request then does a fold starting from {i,Bucket,Index,FirstValue,_} and returns what it finds. The default is to return the keys found, and you can request the values as well, but not the value without the keys. So you would have to use either map-reduce or some client-side code to remove the bucket/key and deduplicate the value list.
Related
I'm trying to run a batch-get-item from CLI and getting an error that I cannot pass more than 100 keys:
failed to satisfy constraint: Member must have length less than or equal to 100
This is the command I use
aws dynamodb batch-get-item \
--request-items file://request-items.json \
--return-consumed-capacity TOTAL > user_table_output.txt
I would like to know if there's a way I can add pagination to my query?
or is there another way I can run the query?
I have ~4000 keys which I need to query.
Thanks in advance.
You'll have to break your keys down into batches of no more than 100. Also, keep in mind that the response may not include all the items you requested if the size of the items being returned exceeds 16MB. If that happens the response will include UnprocessedKeys, which can be used to request the keys that were not retrieved.
BatchGetItem has information about the API, or you can view the AWS CLI v2 doc here.
I am writing get-item command for dynamodb of the aws.
Here is my command:
aws dynamodb get-item --table-name foo --key '{\" bar \":{\"S\":\"aaaa\"},\"timestamp\":{\"N\":\"1603610188890\"}}'
As you can see, the table foo has composite primary key:
partition key "bar" and sort key "timestamp".
What is the proper syntax to use comparison for the sort key "timestamp"?
How I can to change my command to get the items whose timestamp is between 1603010188890 and 1603610188890?
Thanks.
The get-item operation can only retrieve a single item, with a specific key. To retrieve items - possibly more than one - in a certain sort-key range you need to use a different request - query.
The "query" request has a key-conditions or key-condition-expression (these are the older and newer, respectively, syntax, for the same thing). With that parameter you can say that you want items where the partition key is equal something, and the sort key is between two values.
After parsing the JSON data in a column within my Kusto Cluster using parse_json, I'm noticing there is still more data in JSON format nested within the resulting projected value. I need to access that information and make every piece of the JSON data its own column.
I've attempted to follow the answer from this SO post (Parsing json in kusto query) but haven't been successful in getting the syntax correct.
myTable
| project
Time,
myColumnParsedJSON = parse_json(column)
| project myColumnParsedNestedJSON = parse_json(myColumnParsedJSON.nestedJSONDataKey)
I expect the results to be projected columns, each named as each of the keys, with their respective values displayed in one row record.
please see the note at the bottom of this doc:
It is somewhat common to have a JSON string describing a property bag in which one of the "slots" is another JSON string. In such cases, it is not only necessary to invoke parse_json twice, but also to make sure that in the second call, tostring will be used. Otherwise, the second call to parse_json will simply pass-on the input to the output as-is, because its declared type is dynamic
once you're able to get parse_json to properly have your payload parsed, you could use the bag_unpack plugin (doc) in order to achieve this requirement you mentioned:
I expect the results to be projected columns, each named as each of the keys, with their respective values displayed in one row record.
I can't figure out why batches in Azure log analytics queries do not work as I expect from documentation.
For example, this query should return two tabular results.
let m = materialize(StormEvents | summarize n=count() by State);
m | where n > 2000;
m | where n < 10
I try to do the same, and always get result only for the first of my subqueries, e.g. in this case only one table and one entry is returned, while the second subquery is not executed (I can leave mistakes in it and they are not caught upon execution).
let someMetrics = materialize (customMetrics | where timestamp > ago (1h));
someMetrics | take 1;
someMetrics | take 3
I think this is limitation of the current Analytics UI. If you submit this query through API layer (DEMO is here), it will come back with two tables.
Here is curl script for reference:
curl "https://api.applicationinsights.io/v1/apps/DEMO_APP/query?query=let%20req%20%3D%20materialize(requests%7C%20where%20timestamp%20%3E%20ago(10m))%3Breq%20%7C%20take%201%3Breq%20%7C%20take%202" -H "x-api-key: DEMO_KEY"
This will return two tables (two metadata parts, one for each table + two result sets, one after each metadata).
I am looking to add some items into DynamoDB via console. (Please see screenshot below). When I click the "Save" button, nothing is happening. No new items are getting created. I also checked the DynamoDB JSON checkbox to convert the JSON into DynamoDB compatible JSON and clicked save button again, but nothing is happening. Can someone please advise, what am I doing wrong ? There are no error messages either.
You haven't provided your table definition so it's difficult to say exactly what a valid item for your table would look like but I can tell you for sure that:
1) You shouldn't be creating an array of JSON objects: each item you create must be an individual valid JSON object. Like so:
{
"sub": 1234,
"EventID": ["B213", "B314"]
}
2) Each item you create must include attributes matching the item schema for your table. This means that if your table has just a partition key defined then each item must include one attribute whose name matches the name of the partition key. If the table has both partition and sort key then each item you create must include at least two attributes, one matching the partition key, the other matching the sort key. And finally, the partition and sort keys must be string or numeric.
Assuming your table has a partition key called sub and no sort key, then the item example above would work.
update
Based on the comment it sounds like the OP was looking for a way to insert multiple items in a single operation. This not possible with the console, and actually it goes deeper than that: Dynamo fundamentally operates one a single item at a time for write operations. It is of course possible to batch up to 25 item writes using the API but that is just a convenience.
If you need to add multiple items to your table, consider writing a small script using the AWS CLI or the API. It’s is relatively easy to do!
The scripting solution looks something like this:
aws-dynamodb-upload-json.sh
#!/bin/sh
set -e
# parse
if [[ $# -eq 0 ]]; then set -- "--help"; fi
if [[ "$1" = "--help" ]]; then
echo "Usage:"
echo " aws-dynamodb-upload-json {table-name} {file.json}"
exit 1
fi
# config
AWS_DYNAMODB_UPLOAD_TABLE_NAME=$1
AWS_DYNAMODB_UPLOAD_FILE_INPUT=$2
echo "Configuration"
echo "AWS_DYNAMODB_UPLOAD_TABLE_NAME=$AWS_DYNAMODB_UPLOAD_TABLE_NAME"
echo "AWS_DYNAMODB_UPLOAD_FILE_INPUT=$AWS_DYNAMODB_UPLOAD_FILE_INPUT"
# main
jq -c '.[]' < $AWS_DYNAMODB_UPLOAD_FILE_INPUT |
while read -r row
do
echo "Entry: $row"
echo ""
aws dynamodb put-item \
--region us-east-1 \
--table-name $AWS_DYNAMODB_UPLOAD_TABLE_NAME \
--item \
"$row"
done
This does rely on the aws CLI and the jq CLI to be installed and on your $PATH.
Hopefully AWS adds an easier way to do this VIA the web interface someday.