I have AWS step machine and one of the step is used to notify failure using SNS service. I want to select some metadata from input json into outgoing message. So i am trying to concatenate constant string with jsonpath like below
"Notify Failure": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Message.$": "A job submitted through Step Functions failed for document id $.document_id",
"Subject":"Job failed",
"TopicArn": "arn:aws:sns:us-west-2:xxxxxxx:xxxxxxxx"
},
"End": true
}
where document_id is one of the property in input json
However when i try save state machine defination i get error
There is a problem with your ASL definition, please review it and try
again The value for the field 'Message.$' must be a valid JSONPath
I was able to solve a similar issue using:
"Message.$": "States.Format('A job submitted through Step Functions failed for document id {}', $.document_id)",
Described in a AWS News Blog post.
The JSONPath implementation referenced from the AWS Step Functions documentation supports String concatenation via $.concat($..prop) but sadly this does not work when deployed to AWS, suggesting that AWS uses a different implementation.
Therefore there is no way to do string concatenation with JSONPath in AWS.
As the message suggest you need to provide a valid JSONPath.
"Message.$": "$.document_id"
You cannot use any string interpolation as it invalidates the JSONPath format. You will need to construct the message in the preceding state.
I know that this thread is quite old, but I think it might be useful for some people.
It IS actually possible to concatenate strings or JSONPaths in AWS Step Functions thanks to the function States.Format.
The principle is the same as the string format method in Python.
Example with strings
"States.Format('{}<separator_1>{}<separator_2>{}', 'foo', 'bar', 'baz')"
will give you
'foo<separator_1>bar<separator_2>baz'
Example with JSONPaths
"States.Format('{}<separator>{}', $.param_1, $.param_2)"
will give you
'<value of param_1><separator><value of param_2>'
NB: You can also combine strings with JSONPaths.
Hope it helps!
Related
I am currently trying to populate a wikidata instance via POST requests. For this purpose I use the requests library in Python together with the MediaWiki API.
So far I managed to create claims with different datatypes (like String, Quantity, Wikidata items, Media ...). The general scheme I use is this (with different value strings for each datatype):
import requests
session = requests.Session()
# authenticate and obtain a csrf_token
parameters = {
'action': 'wbcreateclaim',
'format': 'json',
'entity': 'Q1234',
'snaktype': 'value',
'property': 'P12',
'value': '{"time": "+2022-02-19T00:00:00Z", "timezone": 0, "precision": 11, "calendarmodel": "http://www.wikidata.org/entity/Q1985727"}',
'token': csrf_token,
'bot': 1,
}
r = session.post(api_url, data=parameters)
print(r.json())
Every attempt to insert data of type time leads to an invalid-snak error (info: "invalid snak data.").
The following changes did not solve the problem:
submitting the value string as dictionary value (without the single quotes),
putting the numeric values into (double) quotes,
using a local item for the calendarmodel ("http://localhost:8181/entity/Q73"),
adding before and after keys in the dictionary,
omitting timezone, precision, calendarmodel and combinations thereof,
formatting the time string as 2022-02-19,
submitting the request with administrator rights (though the error message does not
suggest a problem with insufficient user rights).
Do you know, what I'm doing wrong here? What must the value field look like?
I am aware of the fact that special libraries or interfaces for exist for these tasks. But I do want to use the Wikidata API directly with the requests library in Python.
Thank you very much for your help!
Installed software versions:
MediaWiki: 1.36.3
PHP: 7.4.27
MariaDB 10.3.32-MariaDB-1:10.3.32+maria~focal
ICU 67.1
It works if the value string is generated from the dictionary via json.dumps().
Taking the car dashboard example, I altered the initial #genre node to be #genre:classical. I also added a list to the contex
"choices":["Beethoven","Mahler 9","Brahms 3rd"]
and the Watson response is "I have 3 selections". The condition on the next node is $choices.contains(input.text). The "Found a match" response is just for testing. It looks like this:
When I test this in the api tool and type "Beethoven" both "Found a match" and "Great choice!..." appear. Same for the other two choices, but only if I type the exact choice, e.g., "Mahler 9". Typing "Mahler" or "mahler" doesn't get a match. I read through the SpEL documentation but couldn't see a way in a one-line condition to parse through the list looking for partial matches.
So my question is, is there an condition expression that would match partial user input, e.g., "Mahler"? I'll be using the Java SDK to code the app server, so alternatively I wondered if I could add a temporary #entity just for this sequence instead of using the context list then delete it when the conversation is done? Or is there a way to construct a more complex condition in the MessageRequest and will Watson recognize it? Or is this just not the right way to go about this? Any pointers, examples or docs much appreciated.
So my question is, is there an condition expression that would match partial user input
You can't add temporary entities or intents. As adding them forces Watson to start training itself (even if you could it through code).
You can however create quite complex regular expressions, pass them in as a context variable.
For example your advanced node can have:
{
"output": {
"text": "Please ask me a question."
},
"context": {
"rx": "fish|[0-9]+"
}
}
Then in you condition you would write.
input.text.matches(context.rx)
This will then trigger if the person mentions a number, or the word fish. So you can create your partial user input checking that way.
I have a template to create a key vault and a secret within it. I also have a service fabric template, that requires 3 things from the key vault: the Vault URI, the certificate URL, and the certificate thumbprint.
If I create the key vault and secret with powershell, it is easy to manually copy these 3 things from the output, and paste them into the parameters of the service fabric template. However, what I am hoping to do, due to the fact that this cert has the same life cycle as the service fabric cluster, is to link from the key vault template to the service fabric template, so when I deploy the key vault and secret (which btw is a key that has been base 64 encoded to a string. I could have this as a secret in yet another key vault...), I can pass the 3 values on as parameters.
So I have two questions.
How do I retrieve the 3 values in the arm template. Powershell outputs them as 'ResourceId' of the key vault, 'Id' of the secret, and 'Version' of the secret. My attempt:
"sourceVaultValue": {
"value": "resourceId('Microsoft.KeyVault/vaults/', parameters('keyVaultName')"
},
"certificateThumbprint": {
"value": "[listKeys(resourceId('secrets', parameters('secretName')), '2015-06-01')"
},
"certificateUrlValue": { "value": "[concat('https://', parameters('keyVaultName'), '.vault.azure.net:443/secrets/', parameters('secretName'), resourceId('secrets', parameters('secretName')))]"
But the certificateUrlValue is incorrect. You can see I tried with and without listKeys, but neither seemed to work... (The thumbprint is within the certUrl itself)
If I were to get the correct values, I would like to try pass them as parameters to the next template. The template in question has quite a few more parameters than the 3 I want to pass however. So is it possible to have a parametersLink element to link to the parameter file, as well as a parameters element for just those 3? Or is there an intended way of doing this?
Cheers
Ok, try this when you get back to the keyboard...
1) for the uri, you can use an output like:
"secretUri": {
"type": "string",
"value": "[reference(resourceId('Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), parameters('secretName'))).secretUri]"
}
For #2, you cannot mix and match the link and some values, it's one or the other.
A couple thoughts on how you could do this (it depends a bit on how you want to structure the rest of your deployment)...
One way to think of this is instead of nesting the SF, deploy them in the same template since they have the same lifecycle
instead of nesting the SF template, nest the KV template and reference the outputs of that deployment in the SF template...
Aside from that I can't think of anything elegant - since you want to pass "dynamic" params to a nested deployment really the only way to do that is to dynamically write the param file behind the link or pass all the params into the deployment resource.
HTH - LMK if it doesn't...
Can't Reference a secret with dynamic id !!!!
The obvious problems with this way of doing things are:
Someone needs to type the cleartext password which means:
it needs to be known to anyone who provisions the environment and how do I feed it into an automated environment deployment? If I store the password in a parameter… ???????
"variables": {
"tenantPassword": {
"reference": {
"keyVault": {
"ID": "[concat(subscription().id,'/resourceGroups/',parameters('keyVaultResourceGroup'),'/providers/Microsoft.KeyVault/vaults/', parameters('VaultName'))]"
},
"secretName": "tenantPassword"
}
}
},
I'm trying to use Riak's mapreduce via http. his is what i'm sending:
{
"inputs":{
"bucket":"test",
"key_filters":[["matches", ".*"]]
},
"query":[
{
"map":{
"language":"erlang",
"source":"value(RiakObject, _KeyData, _Arg) -> Key = riak_object:key(RiakObject), Count = riak_kv_crdt:value(RiakObject, <<\"riak_kv_pncounter\">>), [ {Key, Count} ]."
}
}
]}
Riak fails with "[worker_startup_failed]", which isn't very informative. Could anyone please help me get this to actually execute the function?
WARNING
Allowing arbitrary Erlang functions via map-reduce is a security risk. Any valid Erlang can be executed, including sending your entire data set offsite or formatting the hard drive.
You have been warned.
However, if you implicitly trust any client that may connect to your cluster, you can allow Erlang source to be passed in a map-reduce request by setting {allow_strfun, true} in the riak_kv section of app.config, (or in the advanced.config if you are using riak.conf).
Once you have allowed passing an Erlang function in a map-reduce phase, you need to pass in a function of the form fun(RiakObject,KeyData,Arg) -> [result] end. Note that this must be an anonymous fun, so fun is a keyword, not a name, and it must end with end.
Your function should handle the case where {error,notfound} is passed as the first argument instead of an object. Simply adding a catch-all clause to the function could accomplish that.
Perhaps something like:
{
"inputs":{
"bucket":"test",
"key_filters":[["matches", ".*"]]
},
"query":[
{
"map":{
"language":"erlang",
"source":"fun(RiakObject, _KeyData, _Arg) ->
Key = riak_object:key(RiakObject),
Count = riak_kv_crdt:value(
RiakObject,
<<\"riak_kv_pncounter\">>),
[ {Key, Count} ];
(_,_,_) -> [{error,0}]
end."
}
}
]}
Allowing the source to be passed in the request is very useful while developing and debugging. For production, you really should put the functions in a dedicated pre-compiled module that you copy to the code path of each node so that the phase spec can specify the module and function by name instead of providing arbitrary code.
{"map":{
"language":"erlang",
"module":"yourprecompiledmodule",
"function":"functionname"}}
You need to enable allow_strfun on all nodes in your cluster. To do so in Riak 2, you will need to use the advanced.config file to add this to the riak_kv configuration:
[
{riak_kv, [
{allow_strfun, true}
]}
].
The other option is to create your own Erlang module by using the compiler shipped with Riak and placing the *.beam file in a well-known location for Riak to find. The basho-patches directory is one such place.
Please see the documentation as well:
advanced.config
Installing custom Erlang code
HTTP MapReduce
Using MapReduce
Advanced MapReduce
MapReduce / curl example
Is it possible to filter data returned by the Firebase REST API using query parameters? I don't see it mentioned one way or an other in the docs, but the client libraries support it, so I'm hoping it's possible. Thanks.
It might be a bit late to answer, but Firebase does allow querying data via REST.
You can use the orderby option together with limitToLast, startAt etc just like you would when using the SDK.
Checkout the Firebase guide for more details
I fought a little bit to have it working.
I actually needed 2 things:
combine limitToLast with orderBy as mentioned by idan
URL getUrl = new URL( url + "news.json?orderBy=\"timestamp\"&limitToLast=5" );
add a rule in the database to declare an index on this "column"
"news" : { ".indexOn": "timestamp" }
Firebase provides querying parameters. However, I don't think they are the querying parameters you are expecting them to be, which are ones that filter data. Firebase REST API provides querying options like auth, print, callback, format, and download. Check docs here
Without ordering by certain field, By combining params orderBy="$key" and limitToLast=5 you can get the last 5 of inserted data ordered by it's key
The documentation can be looked at here