Datapower script - parsing JSON - ibm-datapower

I am parsing the incoming JSON and checking for a surname value. If it starts with 'C' then response should be sent back, else reject it.
{
"KeyPersons": [{
"Person": {
"KeyPersonType": "D",
"Forename": "user502",
"Surname": "C",
"DateOfBirth": "1995-09-14",
"Gender": "M"
}
How do I parse this JSON using datapower gateway script?
I tried the below and it didnt work.
session.input.readAsJSON(function (error, json)
if(json.KeyPersons.Person.Surname == 'C')
session.output.write(json);

KeyPersons is an array so you'd need an index for it and also in case of checking multiple instances you'd need to loop them.
Also if you want to check the first letter only your code won't work:
if(json.KeyPersons[0].Person.Surname.slice(0, 1) === 'C')
session.output.write(json);
It might also be a good idea to check for existence of the object, e.g.:
if(json.KeyPersons && json.KeyPersons.length
&& json.KeyPersons[0].Person
&& json.KeyPersons[0].Person.Surname
&& json.KeyPersons[0].Person.Surname.slice(0, 1) === 'C')
session.output.write(json);

Related

Weaviate: using near_text with the exact property doesn't return a distance of 0

Here's a minimal example:
import weaviate
CLASS = "Superhero"
PROP = "superhero_name"
client = weaviate.Client("http://localhost:8080")
class_obj = {
"class": CLASS,
"properties": [
{
"name": PROP,
"dataType": ["string"],
"moduleConfig": {
"text2vec-transformers": {
"vectorizePropertyName": False,
}
},
}
],
"moduleConfig": {
"text2vec-transformers": {
"vectorizeClassName": False
}
}
}
client.schema.delete_all()
client.schema.create_class(class_obj)
batman_id = client.data_object.create({PROP: "Batman"}, CLASS)
by_text = (
client.query.get(CLASS, [PROP])
.with_additional(["distance", "id"])
.with_near_text({"concepts": ["Batman"]})
.do()
)
print(by_text)
batman_vector = client.data_object.get(
uuid=batman_id, with_vector=True, class_name=CLASS
)["vector"]
by_vector = (
client.query.get(CLASS, [PROP])
.with_additional(["distance", "id"])
.with_near_vector({"vector": batman_vector})
.do()
)
print(by_vector)
Please note that I specified both "vectorizePropertyName": False and "vectorizeClassName": False
The code above returns:
{'data': {'Get': {'Superhero': [{'_additional': {'distance': 0.08034378, 'id': '05fbd0cb-e79c-4ff2-850d-80c861cd1509'}, 'superhero_name': 'Batman'}]}}}
{'data': {'Get': {'Superhero': [{'_additional': {'distance': 1.1920929e-07, 'id': '05fbd0cb-e79c-4ff2-850d-80c861cd1509'}, 'superhero_name': 'Batman'}]}}}
If I look up the exact vector I get 'distance': 1.1920929e-07, which I guess is actually 0 (for some floating point evil magic), as expected.
But if I use near_text to search for the exact property, I get a distance > 0.
This is leading me to believe that, when using near_text, the embedding is somehow different.
My question is:
Why does this happen?
With two corollaries:
Is 1.1920929e-07 actually 0 or do I need to read something deeper into that?
Is there a way to check the embedding created during the near_text search?
here is some information that may help:
Is 1.1920929e-07 actually 0 or do I need to read something deeper into that?
Yes, this value 1.1920929e-07 should be interpreted as 0. I think there are some unfortunate float32/64 conversions going on that need to be rooted out.
Is there a way to check the embedding created during the near_text search?
The embeddings are either imported or generated during object creation, not at search-time. So performing multiple queries on an unchanged object will utilize the same search vector.
We are looking into both of these issues.

Add values to a JSON array if outer array's name == 'something'

I'm passing a JSON object to jq and want to add extra objects to an inner array ('accessories') if its parent array ('platforms') matches a certain name.
Here's my source JSON:
{
"bridge": {
"name": "Homebridge",
"port": 51395
},
"accessories": [],
"platforms": [
{
"name": "Config",
"port": 8581,
"platform": "config"
},
{
"platform": "homebridge-cbus.CBus",
"name": "CBus",
"client_ip_address": "127.0.0.1",
"accessories": [
{
"values": "existing"
}
]
}
]
}
This works beautifully:
jq '.platforms[1].accessories += [{ "values" : "NEW" }]'
... but of course it's poor form to expect platforms[1] to always the be array I want to append to, so I set about trying to form the right syntax for a search or if/then/else to only act on the .name of the appropriate one.
I thought this was my solution:
jq '.platforms[] | if ( .name=="CBus" ) then .accessories += [{ "values" : "NEW" }] else . end'
... until I realised it was only passing the 'platforms' through and eating the 'bridge' object and empty outer 'accessories' array, which I need to retain.
My issue looks to be similar to JQ | Updating array element selected by `select`, but I've tried LOTS of combinations but just can't break through.
Edit: Here's the correct JQPlay I've been working with:
https://jqplay.org/s/dGDswqAEte
Thanks for any help.
That's a good attempt. The key here is to use the select() function to identify the object you are going to update and overwrite the overall array using |= operator, i.e.
.platforms |= ( map(select(.name == "CBus").accessories += [{ "values" : "NEW" }] ) )
For the snippet in your jq-play link (now removed), you need to do
.gcp_price_list."CP-COMPUTEENGINE-OS"
|= with_entries(select(.value.cores == "shared").value.cores = "0.5")
Or if you want to be even more specific, and keep the entry in gcp_price_list configurable, do
.gcp_price_list |=
with_entries (
select(.key == "CP-COMPUTEENGINE-OS").value |=
with_entries(
select(.value.cores == "shared").value.cores = "0.5") )

Is there an alternative to writeFields?

This article stated that "[writeFields] is now deprecated".
Additionally, I cannot find any documentation for writeFields, it is not even listed as part of Request in the documentation anymore.
Problem
The problem I am facing with Cloud Firestore Security Rules is that verifying that only particular fields are modified requires massive amounts of conditions.
For example, if I want to verify that the only modified value of a document is cakes, I have to write the following rule:
allow update: if request.resource.data.size() == 20
&& request.resource.data.likes == resource.data.likes
&& request.resource.data.name == resource.data.name
&& request.resource.data.date == resource.data.date
&& request.resource.data.body == resource.data.body
&& request.resource.data.title == resource.data.title
&& request.resource.data.tags == resource.data.tags
&& request.resource.data.comments == resource.data.comments
&& request.resource.data.answers == resource.data.answers
&& request.resource.data.awards == resource.data.awards
&& request.resource.data.image == resource.data.image
&& request.resource.data.link == resource.data.link
&& request.resource.data.format == resource.data.format
&& request.resource.data.type == resource.data.type
&& request.resource.data.user == resource.data.user
&& request.resource.data.views == resource.data.views
&& request.resource.data.reports == resource.data.reports
&& request.resource.data.roles == resource.data.roles
&& request.resource.data.category == resource.data.category
&& request.resource.data.votes == resource.data.votes
&& request.resource.data.cakes is int;
Using writeFields, the exact same rule would have looked like this:
allow update: if request.writeFields.hasOnly(['cakes']) && request.resource.data.cakes is int;
What can I do to decrease the code size of my rules / what is the alternative to writeFields?
Limits
There are two limits mentioned in the documentation that make this problem even worse:
Maximum number of expressions evaluated per request: 1,000
Maximum size of a ruleset: 64 KB
I expect to reach both of these at some point with this limitation.
Yes! There is now a replacement called "Map Diffs". Check this syntax out:
allow update: if request.resource.data.diff(resource.data).affectedKeys().hasOnly(['cakes'])
&& request.resource.data.cakes is int;
Unfortunately, what you're doing right now is currently your best option.
The Firebase rules team is working on a language improvement to make it easier to compare/diff map type objects, which will drastically cut down on the number of expressions it takes to do this sort of thing, but there is no timeline for that right now. Please stay tuned.
I end up writing these helper function:
// returns true if all the fields changed are contained in the array parameter
function onlyAllowedFields(fieldsArray) {
return fieldsArray.toSet().hasAll(request.resource.data.diff(resource.data).affectedKeys());
}
// returns true if none of the fields are changed
function noneOfDisallowedFields(fieldsArray) {
return request.resource.data.diff(resource.data).affectedKeys().hasAny(fieldsArray.toSet()) == false
}
I found more helpful this way instead of using hasOnly() that would require that all fields that could be changed must have been changed.

Ggmap "dsk" rate limit

I am tryng to geolocate a vector making use of the R library Ggmap.
location_google_10000 <- geocode(first10000_string, output = "latlon",
source = "dsk", messaging = FALSE)
The problem is that I am using "dsk" -The data science toolkit API- and therefore it has not rate limits as Google (limites to 2500 coordinates per day). However, when I try to run with a vector that contains more than 2500, it pops the following message:
Error: google restricts requests to 2500 requests a day for non-business use.
I have tried to run the code with dsk with 1000 addresses, and later checking if actually google or dsk api has been used:
> geocodeQueryCheck()
2500 geocoding queries remaining.
So for some reason it does not allow me to use more than 2500 with the "dsk", but I am sure that its not using google.
I just ran into the same issue and found your post. I was able to work around this by setting the client and signature values to dummy values, e.g.
geocode(myLocations, client = "123", signature = "123", output = 'latlon', source = 'dsk')
The issue appears to be with this part of the geocode function...
if (length(location) > 1) {
if (userType == "free") {
limit <- "2500"
}
else if (userType == "business") {
limit <- "100000"
}
s <- paste("google restricts requests to", limit, "requests a day for non-business use.")
if (length(location) > as.numeric(limit))
stop(s, call. = F)
userType is set above in this part of the code...
if (client != "" && signature != "") {
if (substr(client, 1, 4) != "gme-")
client <- paste("gme-", client, sep = "")
userType <- "business"
}
else if (client == "" && signature != "") {
stop("if signature argument is specified, client must be as well.",
call. = FALSE)
}
else if (client != "" && signature == "") {
stop("if client argument is specified, signature must be as well.",
call. = FALSE)
}
else {
userType <- "free"
}
So, if client and signature parameters are empty userType is set to "free" and in turn the limit is set to 2,500. By providing values for these parameters you are seen as a 'business' user with a limit of 100,000. This is a good check if the user is assumed to be using 'Google' as opposed to 'dsk' as the source, but is overzealous if the source is 'dsk' and should probably be overridden. To be simple minded maybe something like...
if (source == "google") {
if (client != "" && signature != "") {
if (substr(client, 1, 4) != "gme-")
client <- paste("gme-", client, sep = "")
userType <- "business"
}
else if (client == "" && signature != "") {
stop("if signature argument is specified, client must be as well.",
call. = FALSE)
}
else if (client != "" && signature == "") {
stop("if client argument is specified, signature must be as well.",
call. = FALSE)
}
else {
userType <- "free"
}
} else {
userType <- "business"
}
That would cause problems if client or signature parameters were planned for other sources though. I'll ping the package author.

Trouble checking if url param exists

I'm having a bit of trouble trying to find if my url parameters exist or not.
I have tried the following:
// doesn't work
(Request.QueryString["showTop"] != "" && Request.QueryString["showTop"] != null)
// doesn't work
(Request.Params["showTop"] != "" && Request.Params["showTop"] != null)
I am trying to find the correct value. The full statement looks like:
showTop = (Request.QueryString["showTop"] != "" &&
Request.QueryString["showTop"] != null) ?
Request.QueryString["showTop"] : (10).ToString();
Which works fine, if showTop exists with a value.
This is being done within the view.
Try the following:
showTop = string.IsNullOrEmpty(Request["showTop"]) ? "10" : Request["showTop"];
Assuming you want "showTop"to default to "10".
First check whether QueryString has keys or not by calling this method.
bool qKeys = Request.QueryString.HasKeys();

Resources