Google DLP userdefined output for sensitive data - google-cloud-dlp

I'm having this request body for Google DLP as a text value. Is there any way to configure userdefined RedactConfig to modify the output..?. Is there any way to achieve that one..?
{
"item":{
"value":"My name is Alicia Abernathy, and my email address is aabernathy#example.com."
},
"deidentifyConfig":{
"infoTypeTransformations":{
"transformations":[
{
"infoTypes":[
{
"name":"EMAIL_ADDRESS"
}
],
"primitiveTransformation":{
"replaceWithInfoTypeConfig":{
}
}
}
]
}
},
"inspectConfig":{
"infoTypes":[
{
"name":"EMAIL_ADDRESS"
}
]
}
}
Is there any way to configure userdefined RedactConfig to modify the output..?
And I need the following O/P from Google DLP.
{
"item": {
"value": "My name is Alicia Abernathy, and my email address is {{__aabernathy#example.com__[EMAIL_ADDRESS]__}}."
},
"overview": {
"transformedBytes": "22",
"transformationSummaries": [
{
"infoType": {
"name": "EMAIL_ADDRESS"
},
"transformation": {
"replaceWithInfoTypeConfig": {}
},
"results": [
{
"count": "1",
"code": "SUCCESS"
}
],
"transformedBytes": "22"
}
]
}
}

So you don't actually want to anonymize text, you just want to add information to it? This API isn't suitable for that ... your best bet is to just use inspectContent and with the byte offsets in the findings do your own transforms.
Something like this in pseudocode ...
private static final void labelStringWithFindings(
String stringToLabel,
InspectContentResponse dlpResponse) {
StringBuilder output = new StringBuilder();
final byte[] messageBytes = ByteString.copyFromUtf8(
stringToLabel).toByteArray();
ImmutableList sortedFindings =
sort(dlpResponse.getResult().getFindingsList());
int lastEnd = 0;
for (Finding finding : sortedFindings) {
String quote = Ascii.toLowerCase(finding.getQuote());
String infoType = finding.getInfoType().getName();
String surrogate = String.format("{{__%s__[%s]__}}",
quote, infoType);
final byte[] surrogateBytes = surrogate.getBytes(StandardCharsets.UTF_8);
int startIndex = (int) finding.getLocation().getByteRange().getStart();
int endIndex = (int) finding.getLocation().getByteRange().getEnd();
if (lastEnd == 0 || startIndex > lastEnd) {
output.write(messageBytes, lastEnd, startIndex - lastEnd);
output.write(surrogateBytes, 0, surrogate.length);
}
if (endIndex > lastEnd) {
lastEnd = endIndex;
}
}
if (messageBytes.length > lastEnd) {
output.write(messageBytes, lastEnd, messageBytes.length - lastEnd);
}
return output.toString();
}

Related

API should return IEnum with array type wrapper

I want to transform my API return result
{ "type": "animal" , "age": 3 }
into
{ "myAnimal": [ "type": "cat", "age": 3 ] }
I tried doing
var myAnimal = JsonConvert.SerializeObject(new { MyAnimal = originalResult });
and return it
However, the API return type is set up as <IEnumerable<AnimalModel> whereas variable myAnimal is string type after JsonConvert. So there's a conflict.
I still want to return IEnum, is there anyway for me to work on this?
Thank you in advance!
The json you want is not valid. But if you want to convert an object to an array, IMHO , the easiest way is to parse your origional result
var origAnimal = new { type = "animal", age = 3 };
var newAnimal = new JObject();
newAnimal["myAnimal"] = new JArray(JObject.FromObject(origAnimal).Properties()
.Select(jo => new JObject(jo)));
var json = newAnimal.ToString();
valid json
{
"myAnimal": [
{
"type": "animal"
},
{
"age": 3
}
]
}
or you could be needed this json
var newAnimal = new JObject();
newAnimal["myAnimal"] = new JArray( JObject.FromObject(origAnimal));
var json = newAnimal.ToString();
json
{
"myAnimal": [
{
"type": "animal",
"age": 3
}
]
}
or you have acces to API you will have to change the input parameter to JObject. In this case you don need to parse
public IActionResult MyApi(JObject myAnimal)
{
var newAnimal = new JObject();
newAnimal["myAnimal"] = new JArray(myAnimal.Properties().Select(jo => new JObject(jo)));
...
}

Meteor Apollo GraphQL Interfaces: return an array of object and invoking resolver in another resolver

I'm having some issues with GraphQl interfaces. Let take the example in the doc:
interface Vehicle {
maxSpeed: Int
}
type Airplane implements Vehicle {
maxSpeed: Int
wingspan: Int
}
type Car implements Vehicle {
maxSpeed: Int
licensePlate: String
}
type Query {
vehicle: Vehicle
}
const resolvers = {
Vehicle: {
__resolveType(obj, context, info){
if(obj.wingspan){
return 'Airplane';
}
if(obj.licensePlate){
return 'Car';
}
return null;
},
},
Query: {
vehicle: () => {
// Query Airplane and Car an merge in one single array.
}
},
};
Background: I create 2 different DBs for each interfaces implementation: one for Car and one for Airplane. But I did not create a DB for Vehicle since it just an interface that does not keep data in itself. (I hope this is fine)
The first problem is how to access Car and Airplane resolvers to query their respective DBs in the Vehicle resolver? So in Query.vehicle(), I would like to just call the resolver defined for fetching all cars to get all cars and call the one for fetching airplanes to get all airplanes in the DB. (note using Car.find().fetch() is not always effective because before doing the query, lots of other query needs to be done also avoid code duplication).
The Second problem is that in the example, obj.wingspan assume that obj should be an object. So returning an array from Query.vehicle(), won't help because this query is used in Vehicle.__resolveType. So from what I've experienced with GraphQl interfaces, it only returns a single object from the query.
My aim is to obtain with the with the following query:
{
vehicle {
maxSpeed
... on Car {
licensePlate
}
... on Airplane {
wingspan
}
}
}
the following result:
{
"data": {
"car": [
{
"maxSpeed": 200,
"licensePlate": 1.8
},
{
"maxSpeed": 120,
"licensePlate": "String"
},
...
],
"Airplane": [
{
"maxSpeed": 500,
"wingspan": "String"
},
{
"maxSpeed": 700,
"wingspan": 1.5
},
...
]
}
}
It sounds like you may be trying to return multiple objects. If so, you'll need to return an array of objects:
type Query {
vehicles: [ Vehicle ]
}
If you're just trying to return a single object by ID, in your vehicle resolver, just try both databases and return the one with the result:
const resolvers = {
Query: {
vehicle: (parent, args) => {
const { id } = args;
const [ car, plane ] = await Promise.all(
await carDB.get(id),
await planeDB.get(id)
)
return car || plane
}
}
}
Edit
In case you didn't already get to this:
To get what you're describing, you would need something else:
type-definitions
interface Vehicle {
maxSpeed: Int
}
type Airplane implements Vehicle {
maxSpeed: Int
wingspan: Int
}
type Car implements Vehicle {
maxSpeed: Int
licensePlate: String
}
type Query {
vehicles: VehiclesConnection
}
type VehiclesConnection {
car: [ Car ]
airplane: [ Airplane ]
}
resolvers
const resolvers = {
Query: {
vehicle: () => {
const [ cars, planes ] = await Promise.all(
await carDB.findAll(),
await planeDB.findAll()
)
return { cars, planes }
}
}
}
query
{
vehicle {
car {
maxSpeed
licensePlate
}
airplane {
maxSpeed
wingspan
}
}
}
alternate query
{
vehicle {
car {
...VehicleFragment
}
airplane {
...VehicleFragment
}
}
}
fragment VehicleFragment on Vehicle {
maxSpeed
... on Car {
licensePlate
}
... on Airplane {
wingspan
}
}
result
{
"data": {
"car": [
{
"maxSpeed": 200,
"licensePlate": 1.8
},
{
"maxSpeed": 120,
"licensePlate": "String"
},
...
],
"Airplane": [
{
"maxSpeed": 500,
"wingspan": "String"
},
{
"maxSpeed": 700,
"wingspan": 1.5
},
...
]
}
}

Trouble using date_range in Elastic Search; aggregations not working

I have a form that allows for several fields to search for records in a database. It's able to search by first name, last name, location, etc, but I'm having trouble adding date ranges to the query. Each record has a start year and an end year, and my form has fields Date To and From. My code checks if each form in the field is filled out, and if so appends to a query string; once the query string is complete, it is passed into a Connection.Post search. Everything works except for To and From, but when I add these fields I get a 404 error.
I have a QueryBuilder method that checks the fields and builds the query, here is how it starts:
string query = "{\"query\": { \"bool\": { \"must\": [ ";
bool emptyQ = true;
Here is the first_name part for example:
if (!String.IsNullOrWhiteSpace(q.Name.First))
{
if (exact == true)
{
query += "{ \"match\": { \"first_name\": \"" + q.Name.First + "\" }}";
}
else
{
// no fuzziness allowed in the search term; how can i fix this? (i.e. Sam -> Samuel, but Isac -\> Isaac)
query += "{ \"match_phrase_prefix\": { \"first_name\": \"" + q.Name.First + "\"}}";
}
emptyQ = false;
}
Here is the part that isn't working right:
bool aggs = false;
if (q.FromYear != 0 || q.ToYear != DateTime.Now.Year)
{
if (emptyQ == false)
{
query += "] } }, ";
}
if (emptyQ == true)
{
query += "{";
}
DateTime fromyear = Convert.ToDateTime("01/01/" + q.FromYear.ToString());
if (q.FromYear == 0)
{
fromyear = Convert.ToDateTime("00/00/0000" + q.FromYear.ToString());
}
DateTime toyear = Convert.ToDateTime("12/31/" + q.ToYear.ToString());
query += "\"aggregations\": { \"range\": { \"date_range\": { \"field\": \"start_date\", \"ranges\": [{\"from\": \"" + fromyear + "\"}]}}}";
query += "{\"range\": { \"date_range\": { \"field\": \"end_date\", \"ranges\": [{\"to\": \"" + toyear +
"\"}] }}}}";
aggs = true;
}
if (aggs == false)
{
query += " ] } } }";
}
i.e. when one or both of the date fields is filled out, the query is invalid. When every field is filled out, this is what the query builder returns:
{"query":
{ "bool":
{ "must": [
{ "match_phrase_prefix": { "first_name": "sam"}},
{ "fuzzy": { "last_name": "durand" }},
{ "match": { "record_type": { "query": "Baptism", "type": "phrase"} }},
{ "fuzzy_like_this":
{ "fields": [ "city", "region", "county", "country" ],
"like_text": "Massachusetts"
}
}
]}
},
"aggregations":
{ "range":
{ "date_range":
{ "field": "start_date",
"ranges": [{"from": "1/1/1600 12:00:00 AM"}]
}
}
},
{"range":
{ "date_range":
{ "field": "end_date",
"ranges": [{"to": "12/31/2014 12:00:00 AM"}]
}
}
}
}
Which gives me a 400 bad request error. Again, when the year fields are not filled out, it works just fine. I'm sure it's a problem with how I'm formatting my query, but I'm not sure what I'm doing wrong. Also, is there an easier way I can search for a date range without using aggregations? Is there a search method that I can put in the "query { " itself, rather than after it?
If you're trying to filter the results by a date range you don't want to use the date_range aggregation, you want to use the range filter. That would give you something more like:
{"query":
{ "bool":
{ "must": [
{
"range" : {
"start_date" : {
"gte": "1/1/1600 12:00:00 AM",
}
}
},
{
"range" : {
"end_date" : {
"lte": 12/31/2014 12:00:00 AM",
}
}
},
]}
}
Turns out I didn't have to do anything fancy at all, I just used a simple range:
query += "{ \"range\": { \"start_date.year\": { \"gte\": " + fyear + ", \"lte\": " + tyear + "} }}, " +
"{ \"range\": { \"end_date.year\": { \"gte\": " + fyear + ", \"lte\": " + tyear + "} }}] } } }, ";
My problem was in assuming that datetime objects would not be read correctly in a range query, but you can in fact use datetime objects in a range.

Json Structure Issue

A third party API is returning JSON in the following format
{
"group1": {
"Colour": "Blue",
"Name": "Dave"
},
"group2": {
"Colour": "Red",
"Name": "Karen"
},
"group3": {
"Colour": "Green",
"Name": "Ryan"
}
}
I'm finding the outer 'groupX' identifier to problematic when attempting to deserialize the JSON using JSON.NET.
Does anyone know best to parse JSON in this format?
{
"employees": [
{ "first-name":"John" , "last-name":"Doe" },
{ "first-name":"Anna" , "last-name":"Smith" },
{ "first-name":"Peter" , "last-name":"Jones" }
]
}
Your JSON is Correct.
if tou want to check than use below url JSONlint
Define a class Group like this:
class Group
{
public string Colour { get; set; }
public string Name { get; set; }
}
Then you can deserialize like this:
var dict = JsonConvert.DeserializeObject<Dictionary<string, Group>>(json);
Here's a quick demo:
string json = #"
{
""group1"": {
""Colour"": ""Blue"",
""Name"": ""Dave""
},
""group2"": {
""Colour"": ""Red"",
""Name"": ""Karen""
},
""group3"": {
""Colour"": ""Green"",
""Name"": ""Ryan""
}
}";
var dict = JsonConvert.DeserializeObject<Dictionary<string, Group>>(json);
foreach (var kvp in dict)
{
Console.WriteLine(kvp.Key);
Group group = kvp.Value;
Console.WriteLine(" Colour: " + group.Colour);
Console.WriteLine(" Name: " + group.Name);
}
Output:
group1
Colour: Blue
Name: Dave
group2
Colour: Red
Name: Karen
group3
Colour: Green
Name: Ryan

Make a http get request to rest api and convert to a collection

Is it possible to make a Meteor.http.get("localhost:4000/api/resource.json") and use the returned response as Meteor.collection?
Sure. Let's say the JSON looks like:
{
'results': [
{
'name': 'bob',
'eyes': 'brown'
},
{
'name': 'sue',
'eyes': 'blue'
}
]
}
To insert that into a collection you would do:
Meteor.http.get("localhost:4000/api/resource.json", function(error, result) {
if (result.statusCode === 200) {
for (var i = 0; i < result.data.results.length; i++) {
MyCollection.insert(result.data.results[i])
}
}
else {
console.log(result);
}
});

Resources