I have a NodeJS app using Express-Js framework and the latest Express-Validator package. I have an input field that I want to "escape" and save the result in AWS DynamoDB.
The parameter is req.body.dealer_code and the value...
Before
req.body.dealer_code = "<script>alert('hello')</script>"
After
req.body.dealer_code = "<script>alert('hello')</script>"
When the value is inserted into the DynamoDB, it appears as "<script>alert('hello')</script>" in the Tree view.
What goes wrong here?
Thanks.
After an experiment, the string is indeed stored as an escaped string.
{ Item:
{ escaped: '<script>alert('hello')</script>'
}
}
Related
I'm currently working on pagination in DynamoDB using the JS AWS-SDK's executeStatement using PartiQL, but my returned object does not contain a NextToken (only the Items array), which is used to paginate.
This is what the code looks like (pretty simple):
const statement = `SELECT "user", "id" FROM "TABLE-X" WHERE "activity" = 'XXXX'`;
const params = {Statement: statement};
try {
const posted = await dynamodb.executeStatement(params).promise();
return { posted: posted };
} catch(err) {
throw new Error(err);
}
I was wondering if anyone has dealt with pagination using PartiQL for DynamoDB.
Could this be because my partition key is a string type?
Still trying to figure it out.
Thanks, in advance!
It turns out that if you want a NextToken DO NOT use version 2 of the AWS SDK for JavaScript. Use version 3. Version 3 will always return a NextToken, even if it is undefined.
From there you can figure out your limits, etc (default limit until you actually get a NextToken is 1MB). You'll need to look into the dynamodb v3 execute statement method.
You can also look into dynamodb paginators, which I've never used, but plan on studying.
I am fairly new in this realm and any help is appreciated
I have a table in Dynamodb database named Tenant as below:
"TenantId" is the hash primary key and I have no other keys. And I have a field named "IsDeleted" which is boolean
Table Structure
I am trying to run a query to get the record with specified "TenantId" while it is not deleted ("IsDeleted == 0")
I can get a correct result by running the following code: (returns 0 item)
var filter = new QueryFilter("TenantId", QueryOperator.Equal, "2235ed82-41ec-42b2-bd1c-d94fba2cf9cc");
filter.AddCondition("IsDeleted", QueryOperator.Equal, 0);
var dbTenant = await
_genericRepository.FromQueryAsync(new QueryOperationConfig
{
Filter = filter
}).GetRemainingAsync();
But no luck when I try to get it with following code snippet (It returns the item which is also deleted) (returns 1 item)
var queryFilter = new List<ScanCondition>();
var scanCondition = new ScanCondition("IsDeleted", ScanOperator.Equal, new object[]{0});
queryFilter.Add(scanCondition);
var dbTenant2 = await
_genericRepository.LoadAsync("2235ed82-41ec-42b2-bd1c-d94fba2cf9cc", new DynamoDBOperationConfig
{
QueryFilter = queryFilter,
ConditionalOperator = ConditionalOperatorValues.And
});
Any Idea why ScanCondition has no effect?
Later I also tried this: (throw exception)
var dbTenant2 = await
_genericRepository.QueryAsync("2235ed82-41ec-42b2-bd1c-d94fba2cf9cc", new DynamoDBOperationConfig()
{
QueryFilter = new List<ScanCondition>()
{
new ScanCondition("IsDeleted", ScanOperator.Equal, 0)
}
}).GetRemainingAsync();
It throws with: "Message": "Must have one range key or a GSI index defined for the table Tenants"
Why does it complain about Range key or Index? I'm calling
public AsyncSearch<T> QueryAsync<T>(object hashKeyValue, DynamoDBOperationConfig operationConfig = null);
You simply cant query a table only giving a single primary key (only hash key). Because there is one and only one item for that primary key. The result of the Query would be that still that single item, which is actually Load operation not Query. You can only query if you have composite primary key in this case (Hash (TenantID) and Range Key) or GSI (which doesn't impose key uniqueness therefore accepts duplicate keys on index).
The second code attempts to filter the Load. DynamoDBOperationConfig's QueryFilter has a description ...
// Summary:
// Query filter for the Query operation operation. Evaluates the query results and
// returns only the matching values. If you specify more than one condition, then
// by default all of the conditions must evaluate to true. To match only some conditions,
// set ConditionalOperator to Or. Note: Conditions must be against non-key properties.
So works only with Query operations
Edit: So after reading your comments on this...
I dont think there conditional expressions are for read operations. AWS documents indicates they are for put or update operations. However, not being entirely sure on this since I never needed to do a conditional Load. There is no such thing like CheckIfExists functionality as well in general. You have to read the item and see if it exists. Conditional load will still consume read throughput so your only advantage would be only NOT retrieving it in other words saving the bandwith (which is very negligible for single item).
My suggestion is read it and filter it in your application layer. Dont query for it. However what you can also do is if you very need it you can use TenantId as hashkey and isDeleted for range key. If you do so, you always have to query when you wanna get a tenant. With the query you can set rangeKey(isDeleted) to 0 or 1. This isnt how I would do it. As I said, would just read it and filter it at my application.
Another suggestion thing could be setting a GSI on isDeleted field and writing null when it is 0. This way you can only see that attribute in your table when its only 1. GSI on such attribute is called sparse index. Later if you need to get all the tenants that are deleted (isDeleted=1) you can simply scan that entire index without conditions. When you are writing null when its 0 dynamoDB wont put it in the index at the first place.
for some reason I want to use book.randomID as key in amazon DynamoDB table using java code. when i tried id added a new field in the item named "book.randomID"
List<KeySchemaElement> keySchema = new ArrayList<KeySchemaElement>();
keySchema.add(new KeySchemaElement().withAttributeName("conceptDetailInfo.conceptId").withKeyType(KeyType.HASH)); // Partition
and here is the json structure
{
"_id":"123",
"book":{
"chapters":{
"chapterList":[
{
"_id":"11310674",
"preferred":true,
"name":"1993"
}
],
"count":1
},
"randomID":"1234"
}
}
so is it possible to use such element as key. if yes how can we use it as key
When creating DynamoDB tables AWS limits it to the types String, Binary and Number. Your attribute book.random seems to be a String.
As long as it's not one of the other data types like List, Map or Set you should be fine.
Just going to AWS console and trying it out worked for me:
I have a DynamoDB database with an attribute Event_Name which has uppercase values, for example KRISHNA. When I specify a Scan FilterExpression comparitor CONTAINS with a lowercase value, for example krishna, the item with value KRISHNA is not returned. When I use the uppercase value it returns the item. Please help me.
For reference my code is:
var params = {
TableName: "User",
FilterExpression: "NOT userId in (:a) and contains(Event_Name, :name)",
ExpressionAttributeValues: {
":a": {
S: $scope.userid
},
":name": {
S: namekey
}
}
};
using dynamodb scan method
Probably you already figured out, but since I stumbled upon this question and it's not closed, here is a link in AWS forum addressing the issue
https://forums.aws.amazon.com/thread.jspa?threadID=92159
DynamoDB is case sensitive. If your data is case insensitive, one solution is to lower case or upper case the data before storing it in DynamoDB. Then you can get around this by querying for all lower case or all upper case. You will need to take locale into account for locale-sensitive ordering.
So there is nothing wrong you are doing, you just were expecting something that is not available with DynamoDB
I've read some sample code (especially from the Couchbase Model Views demo project link) and realized the map() function is so strange.
function(doc, meta) {
if (doc.type == "beer" && doc.name){
emit(doc.name, null);
}
}
Why the emit function's value is null, but the result from GetView("beers", "beer") is getting the value perfectly???
Please help me out !
In couchbase usally view's result set is built in background. If you have for example 1 million documents each 4Kb size without any views it take ~4Gb on disk. When you create a view with map function like
function(doc, meta) {
emit(doc.name, doc);
}
As a result it take additional 4Gb on disk for view results because view results are stored separately. And in most cases (if you query view with param Stale=Ok) couchbase returns result from that "precompiled" set of records, couchbase doesn't scan all docs on each query.
So emmiting null in map functions is used to prevent disk space usage and it also increase speed of indexing process.
Now the second question about couchbase magic when "result from GetView("beers", "beer") is getting the value perfectly". Couchbase get(key) and getMulti(keys) operations are very fast. So when you query view that emits null it returns not only nulls, also it returns document ids. Then you can manually use getMulti for that array of document ids to get doc's value or in some SDKs there is query param called IncludeDocs that will do the same automatically.