DynamoDB Set of Objects Insert and Get - amazon-dynamodb

I have the following DynamoDB Table
#Builder
#DynamoDBTable(tableName = "rule_templates")
#Setter
#NoArgsConstructor
#AllArgsConstructor
public class DynamoRuleTemplate {
private String id;
private DataType dataType;
private String displayName;
private String helpText;
private String expression;
private Set<Param> params;
#DynamoDBHashKey(attributeName = "id")
public String getId() {
return id;
}
#DynamoDBAttribute(attributeName = "data_type")
public DataType getDataType() {
return dataType;
}
#DynamoDBAttribute(attributeName = "display_name")
public String getDisplayName() {
return displayName;
}
#DynamoDBAttribute(attributeName = "help_text")
public String getHelpText() {
return helpText;
}
#DynamoDBAttribute(attributeName = "expression")
public String getExpression() {
return expression;
}
#DynamoDBTypeConverted(converter = ParamSet.class)
#DynamoDBAttribute(attributeName = "params")
public Set<Param> getParams() {
return params;
}
}
Here is the Param class:
#Data
public class Param {
private DataType dataType;
private String identifier;
private String displayName;
private String helpText;
}
I am pre-filling the table with some data in the form of json through AWS CLI like:
{
"id": {
"S": "string-length-range"
},
"data_type": {
"S": "STRING"
},
"display_name": {
"S": "String length range"
},
"expression": {
"S": "$val.length() <= $max && $val.length() >= $min"
},
"help_text": {
"S": "String length range"
},
"params": {
"L": [
{
"M": {
"dataType": {
"S": "INTEGER"
},
"displayName": {
"S": "max"
},
"identifier": {
"S": "$max"
}
}
},
{
"M": {
"dataType": {
"S": "INTEGER"
},
"displayName": {
"S": "min"
},
"identifier": {
"S": "$min"
}
}
}
]
}
}
But while doing a GET from the table using JAVA application, I'm getting the following error for params field: com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException: expected S in value {L: [{M: {identifier={S: $max,}, displayName={S: max,}, dataType={S: INTEGER,}},}, {M: {identifier={S: $min,}, displayName={S: min,}, dataType={S: INTEGER,}, rule={M: {expression={S: $min >= 0,}},}},}],}
Can someone help me with the correct json so that I don't get the above error?

Related

Trouble saving IDictionary type property in ElasticSearch using NEST

I'm trying to add data to elastic search. When I'm pushing the IDictionary<string,object> type properties to Elasticsearch it stores the data in a different way and returns me different result.
Model
using Nest;
namespace FrontLine.Db.Elasticsearch.Models
{
[ElasticsearchType(RelationName = "event")]
public class Event
{
public DateTime TimeStamp { get; set; }
[Text(Name = "event_name")]
public string EventName { get; set; } = null!;
[Object]
public IDictionary<string,object> Properties { get; set; } = null!;
}
}
Data that I'm saving
[
{
"timeStamp": "2022-06-14T20:33:14.866Z",
"eventName": "hello1",
"properties": {
"additionalProp1": "string",
"additionalProp2": "string",
"additionalProp3": "string"
}
}
]
Result I'm getting from Elasticsearch
[
{
"timeStamp": "2022-06-14T20:33:14.866Z",
"eventName": "hello1",
"properties": {
"additionalProp1": {
"valueKind": 3
},
"additionalProp2": {
"valueKind": 3
},
"additionalProp3": {
"valueKind": 3
}
}
}
]
What should I change in or how should I map it to get the correct data form elastic search?

How to pass JSON array of objects as a parameter to ARRAY_CONTAINS function using SQL API

My Cosmos DB SQL query includes the following call to the the ARRAY_CONTAINS function:
WHERE ARRAY_CONTAINS(#targetActions, {'aid': c.id, 'at': atil.actionType }, true)
I need to use the following JSON array as the #targetActions parameter:
[
{
"aid": "vdb-rc-vote-16375",
"at": "cast-rc-vote"
},
{
"aid": "vdb-rc-vote-16312",
"at": "cast-rc-vote"
},
{
"aid": "vdb-rc-vote-16692",
"at": "cast-rc-vote"
}
]
I use the following code to set the parameter on my query definition:
myQueryDefinition.WithParameter("#targetActions", targetActions);
I tried the following for the targetActions value
JArray
string
IEnumerable<TargetActionType> where TargetActionType is a .NET class with the same properties (aid and at) that are in the JSON objects.
My query does not return the expected result with any of the above. However, it works fine if I hard-code the JSON in the query as follows:
WHERE ARRAY_CONTAINS(
[
{
"aid": "vdb-rc-vote-16375",
"at": "cast-rc-vote"
},
{
"aid": "vdb-rc-vote-16312",
"at": "cast-rc-vote"
},
{
"aid": "vdb-rc-vote-16692",
"at": "cast-rc-vote"
}
]
, {'aid': c.id, 'at': atil.actionType }, true)
According to my test,JArray is ok,you can try again.
Below is my test code:
string sqlQueryText = "SELECT * FROM c WHERE ARRAY_CONTAINS(#targetActions, {'aid': c.id, 'at': c.at }, true)";
JObject jo1 = new JObject { { "aid", "vdb-rc-vote-16375" },{ "at", "cast-rc-vote" } };
JObject jo2 = new JObject { { "aid", "vdb-rc-vote-16312" }, { "at", "cast-rc-vote" } };
JObject jo3 = new JObject { { "aid", "vdb-rc-vote-16692" }, { "at", "cast-rc-vote" } };
JArray targetActions = new JArray();
targetActions.Add(jo1);
targetActions.Add(jo2);
targetActions.Add(jo3);
QueryDefinition queryDefinition = new QueryDefinition(sqlQueryText).WithParameter("#targetActions", targetActions);
FeedIterator queryResultSetIterator = this.container.GetItemQueryStreamIterator(queryDefinition);
while (queryResultSetIterator.HasMoreResults)
{
using (ResponseMessage response = await queryResultSetIterator.ReadNextAsync())
{
StreamReader streamReader = new StreamReader(response.Content);
string responseContentAsString = await streamReader.ReadToEndAsync();
Console.WriteLine(responseContentAsString);
}
}
Here is result:
{"_rid":"R-5RALZpDGQ=","Documents":[{"id":"vdb-rc-vote-16375","at":"cast-rc-vote"}],"_count":1}
This is my document of cosmos db:
{
"id": "vdb-rc-vote-16375",
"at": "cast-rc-vote"
}
Hope this can help you:).

Google DLP userdefined output for sensitive data

I'm having this request body for Google DLP as a text value. Is there any way to configure userdefined RedactConfig to modify the output..?. Is there any way to achieve that one..?
{
"item":{
"value":"My name is Alicia Abernathy, and my email address is aabernathy#example.com."
},
"deidentifyConfig":{
"infoTypeTransformations":{
"transformations":[
{
"infoTypes":[
{
"name":"EMAIL_ADDRESS"
}
],
"primitiveTransformation":{
"replaceWithInfoTypeConfig":{
}
}
}
]
}
},
"inspectConfig":{
"infoTypes":[
{
"name":"EMAIL_ADDRESS"
}
]
}
}
Is there any way to configure userdefined RedactConfig to modify the output..?
And I need the following O/P from Google DLP.
{
"item": {
"value": "My name is Alicia Abernathy, and my email address is {{__aabernathy#example.com__[EMAIL_ADDRESS]__}}."
},
"overview": {
"transformedBytes": "22",
"transformationSummaries": [
{
"infoType": {
"name": "EMAIL_ADDRESS"
},
"transformation": {
"replaceWithInfoTypeConfig": {}
},
"results": [
{
"count": "1",
"code": "SUCCESS"
}
],
"transformedBytes": "22"
}
]
}
}
So you don't actually want to anonymize text, you just want to add information to it? This API isn't suitable for that ... your best bet is to just use inspectContent and with the byte offsets in the findings do your own transforms.
Something like this in pseudocode ...
private static final void labelStringWithFindings(
String stringToLabel,
InspectContentResponse dlpResponse) {
StringBuilder output = new StringBuilder();
final byte[] messageBytes = ByteString.copyFromUtf8(
stringToLabel).toByteArray();
ImmutableList sortedFindings =
sort(dlpResponse.getResult().getFindingsList());
int lastEnd = 0;
for (Finding finding : sortedFindings) {
String quote = Ascii.toLowerCase(finding.getQuote());
String infoType = finding.getInfoType().getName();
String surrogate = String.format("{{__%s__[%s]__}}",
quote, infoType);
final byte[] surrogateBytes = surrogate.getBytes(StandardCharsets.UTF_8);
int startIndex = (int) finding.getLocation().getByteRange().getStart();
int endIndex = (int) finding.getLocation().getByteRange().getEnd();
if (lastEnd == 0 || startIndex > lastEnd) {
output.write(messageBytes, lastEnd, startIndex - lastEnd);
output.write(surrogateBytes, 0, surrogate.length);
}
if (endIndex > lastEnd) {
lastEnd = endIndex;
}
}
if (messageBytes.length > lastEnd) {
output.write(messageBytes, lastEnd, messageBytes.length - lastEnd);
}
return output.toString();
}

Is it possible to deserialize inline arrays with Json.NET?

Let me try to simplify my question by extracting and abstracting relevant information from a complex object and large JSON text.
Class Foo()
{
string Name;
Version[] versions;
}
JsonText(converted from XML) can be something like this:
"Foo": {
"Name": "test",
"versions": {
"Major": "1",
"Minor": "1"
},
"versions": {
"Major": "2",
"Minor": "1"
},
"versions": {
"Major": "3",
"Minor": "1"
}
}
or the following:
"Foo": {
"Name": "test",
"versions": {
"Major": "1",
"Minor": "1"
}
}
JsonConvert.DeserializeObject(JsonText) results in:
Cannot deserialize the current JSON object (e.g. {"name":"value"}) into type 'Version[]' because the type requires a JSON array (e.g. [1,2,3]) to deserialize correctly.
Is it possible to do this desrialization?
Updated based on commment
Your Json doesn't look valid. A Json needs to be enclosed with "[" "]". For example,
[ "Ford", "BMW", "Fiat" ]
If you are allowed to modify your Json, You could modify the Json as following
{
"Foo":
{
"Name": "test",
"versions": [
{
"Major": "1",
"Minor": "1"
},
{
"Major": "2",
"Minor": "1"
},
{
"Major": "3",
"Minor": "1"
}]
}
}
Based on your comment, you would like to avoid the Json Array Syntax if there is only a single element. On other cases, you would like to stick to Array Syntax. In such scenario,you could write a Custom Json Converter.
class CustomConverter<T> : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(List<T>));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JToken token = JToken.Load(reader);
if (token.Type == JTokenType.Array)
{
return token.ToObject<List<T>>().ToArray();
}
return new List<T> { token.ToObject<T>() }.ToArray();
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
You can decorate your class with JsonConverter attribute to use the Custom Json Converter.
class Wrapper
{
public Foo Foo{get;set;}
}
class Foo
{
[JsonProperty("Name")]
public string Name{get;set;}
[JsonProperty("versions")]
[JsonConverter(typeof(CustomConverter<Version>))]
public Version[] versions{get;set;}
}
Now you can deserialize as
var result =JsonConvert.DeserializeObject<Wrapper>(str);
This would now work, if you use either of following Json definition.
{
'Foo':
{
'Name': 'test',
'versions': [
{
'Major': '1',
'Minor': '1'
},
{
'Major': '2',
'Minor': '1'
},
{
'Major': '3',
'Minor': '1'
}]
}
}
Single Version (Without Array syntax)
{
'Foo':
{
'Name': 'test',
'versions':
{
'Major': '1',
'Minor': '1'
}
}
}

deserialize user list with json.net

hello I have a problem to deserialize this json
{
"166": {
"uid": "166",
"name": "test",
"mail": "test#test.com"
},
"167": {
"uid": "167",
"name": "test",
"mail": "test#test.com"
},
"168": {
"uid": "168",
"name": "test",
"mail": "test#test.com"
}
}
I use json.net and would retrieve all users, but the first element is always changing its name.
Any idea how to treat it?
You can handle this situation by deserializing into a Dictionary<string, User>, assuming you have a User class defined like this:
public class User
{
public string uid { get; set; }
public string name { get; set; }
public string mail { get; set; }
}
Here is a demo:
class Program
{
static void Main(string[] args)
{
string json = #"
{
""166"": {
""uid"": ""166"",
""name"": ""Joe"",
""mail"": ""joe#example.org""
},
""167"": {
""uid"": ""167"",
""name"": ""Pete"",
""mail"": ""pete#example.org""
},
""168"": {
""uid"": ""168"",
""name"": ""Fred"",
""mail"": ""fred#example.org""
}
}";
Dictionary<string, User> users =
JsonConvert.DeserializeObject<Dictionary<string, User>>(json);
foreach (KeyValuePair<string, User> kvp in users)
{
Console.WriteLine(kvp.Key);
Console.WriteLine(" uid: " + kvp.Value.uid);
Console.WriteLine(" name: " + kvp.Value.name);
Console.WriteLine(" mail: " + kvp.Value.mail);
}
}
}
Here is the output:
166
uid: 166
name: Joe
mail: joe#example.org
167
uid: 167
name: Pete
mail: pete#example.org
168
uid: 168
name: Fred
mail: fred#example.org

Resources