I have an Elastic Search cluster with a lot of nice data, that I have created some nice Kibana dashboards for.
For the next level I decided to take a look at scripted fields to make some of the dashboards even nicer.
I want to translate some of the numeric fields into more easily understandable text values. As an example of what I want to do and what I have tried I will use the http response status code field, that most will understand quite easily but also illustrates the problem.
We log the numeric status code (200, 201, 302, 400, 404, 500 etc.) I can create a data table visualization that tells me the count for each of these status codes. But I would like to display the text reason in my dashboard.
I can create a painless script with a lot of IF statements like this:
if (doc['statuscode'].value == 200) {return "OK";}
if (doc['statuscode'].value == 201) {return "Created";}
if (doc['statuscode'].value == 400) {return "Bad Request";}
return doc['statuscode'].value;
But that isn't very nice I think.
But since I will most likely have about 150 different values and that list won't change very often, so I can live with maintaining a static map. But I haven't found any examples of implementing a map or dictionary in painless scripting.
I was thinking of implementing something like this:
Map reasonMap;
reasonMap[200] = 'OK';
reasonMap[201] = 'Created';
def reason = reasonMap[doc['statuscode'].value];
if (reason != null)
{
return reason;
}
return doc['statuscode'].value;
I haven't been able to make this code work though. The question is also if this will perform well enough for a map with up to 150 values.
Thanks
EDIT
After some trial and error... and a lot of googling, this is what I came up with that works (notice that the key needs to start with a character and not a number):
def reasonMap =
[
's200': 'OK',
's201': 'Created'
];
def key = 's' + doc['statuscode'].value
def reason = reasonMap[key];
if (reason != null)
{
return reason;
}
return doc['statuscode'].value;
Should it be
def reason = reasonMap[doc['statuscode']value];
It will perform well with a Map of 150 values.
Related
I have a function that will download an image collection as a TFrecord or a geotiff.
Heres the function -
def download_image_collection_to_drive(collection, aois, bands, limit, export_format):
if collection.size().lt(ee.Number(limit)):
bands = [band for band in bands if band not in ['SCL', 'QA60']]
for aoi in aois:
cluster = aoi.get('cluster').getInfo()
geom = aoi.bounds().getInfo()['geometry']['coordinates']
aoi_collection = collection.filterMetadata('cluster', 'equals', cluster)
for ts in range(1, 11):
print(ts)
ts_collection = aoi_collection.filterMetadata('interval', 'equals', ts)
if ts_collection.size().eq(ee.Number(1)):
image = ts_collection.first()
p_id = image.get("PRODUCT_ID").getInfo()
description = f'{cluster}_{ts}_{p_id}'
task_config = {
'fileFormat': export_format,
'image': image.select(bands),
'region': geom,
'description': description,
'scale': 10,
'folder': 'output'
}
if export_format == 'TFRecord':
task_config['formatOptions'] = {'patchDimensions': [256, 256], 'kernelSize': [3, 3]}
task = ee.batch.Export.image.toDrive(**task_config)
task.start()
else:
logger.warning(f'no image for interval {ts}')
else:
logger.warning(f'collection over {limit} aborting drive download')
It seems whenever it gets to the second aoi it fails, Im confused by this as if ts_collection.size().eq(ee.Number(1)) confirms there is an image there so it should manage to get product id from it.
line 24, in download_image_collection_to_drive
p_id = image.get("PRODUCT_ID").getInfo()
File "/lib/python3.7/site-packages/ee/computedobject.py", line 95, in getInfo
return data.computeValue(self)
File "/lib/python3.7/site-packages/ee/data.py", line 717, in computeValue
prettyPrint=False))['result']
File "/lib/python3.7/site-packages/ee/data.py", line 340, in _execute_cloud_call
raise _translate_cloud_exception(e)
ee.ee_exception.EEException: Element.get: Parameter 'object' is required.
am I falling foul of immutable server side objects somewhere?
This is a server-side value, problem, yes, but immutability doesn't have to do with it — your if statement isn't working as you intend.
ts_collection.size().eq(ee.Number(1)) is a server-side value — you've described a comparison that hasn't happened yet. That means that doing any local operation like a Python if statement cannot take the comparison outcome into account, and will just treat it as a true value.
Using getInfo would be a quick fix:
if ts_collection.size().eq(ee.Number(1)).getInfo():
but it would be more efficient to avoid using getInfo more than needed by fetching the entire collection's info just once, which includes the image info.
...
ts_collection_info = ts_collection.getInfo()
if ts_collection['features']: # Are there any images in the collection?
image = ts_collection.first()
image_info = ts_collection['features'][0] # client-side image info already downloaded
p_id = image_info['properties']['PRODUCT_ID'] # get ID from client-side info
...
This way, you only make two requests per ts: one to check for the match, and one to start the export.
Note that I haven't actually run this Python code, and there might be some small mistakes; if it gives you any trouble, print(ts_collection_info) and examine the structure you actually received to figure out how to interpret it.
I'm trying to use SequenceReader<T> in .Net Core Preview 8 to parse Guacamole Protocol network traffic.
The traffic might look as follows:
5.error,14.some text here,1.0;
This is a single error instruction. There are 3 fields:
OpCode = error
Reason = some text here
Status = 0 (see Status Codes)
The fields are comma delimited (semi-colon terminated), but they also have the length prefixed on each field. I presume that's so that you could parse something like:
5.error,24.some, text, with, commas,1.0;
To produce Reason = some, text, with, commas.
Simple comma delimited parsing is simple enough to do (with or without SequenceReader). However, to utilise the length I've tried the following:
public static bool TryGetNextElement(this ref SerializationContext context, out ReadOnlySequence<byte> element)
{
element = default;
var start = context.Reader.Position;
if (!context.Reader.TryReadTo(out ReadOnlySequence<byte> lengthSlice, Utf8Bytes.Period, advancePastDelimiter: true))
return false;
if (!lengthSlice.TryGetInt(out var length))
return false;
context.Reader.Advance(length);
element = context.Reader.Sequence.Slice(start, context.Reader.Position);
return true;
}
Based on my understanding of the initial proposal, this should work, though also could be simplified I think because some of the methods in the proposal make life a bit easier than that which is available in .Net Core Preview 8.
However, the problem with this code is that the SequenceReader does not seem to Advance as I would expect. It's Position and Consumed properties remain unchanged when advancing, so the element I slice at the end is always an empty sequence.
What do I need to do in order to parse this protocol correctly?
I'm guessing that .Reader here is a property; this is important because SequenceReader<T> is a mutable struct, but every time you access .SomeProperty you are working with an isolated copy of the reader. It is fine to hide it behind a property, but you'd need to make sure you work with a local and then push back when complete, i.e.
var reader = context.Reader;
var start = reader.Position;
if (!reader.TryReadTo(out ReadOnlySequence<byte> lengthSlice,
Utf8Bytes.Period, advancePastDelimiter: true))
return false;
if (!lengthSlice.TryGetInt(out var length))
return false;
reader.Advance(length);
element = reader.Sequence.Slice(start, reader.Position);
context.Reader = reader; // update position
return true;
Note that a nice feature of this is that in the failure cases (return false), you won't have changed the state yet, because you've only been mutating your local standalone clone.
You could also consider a ref-return property for .Reader.
So I made this very small function. it is a bonehead easy function but frankly borderline my capabilities.. Im learning. The function works as expected, but I would like to go further. I would like to make it so I can either give it an argument (a username) and just get the information for that single user, or default to reporting all users. is this possible w/o starting over from what I have so far?
I have just poked around and seen some examples but nothing that I can fit into my script. that I can understand at least.
import boto3
iam = boto3.client('iam')
def user_group():
for myusers in iam.list_users()['Users']:
Group = iam.list_groups_for_user(UserName=myusers['UserName'])
print("User: " + myusers['UserName'])
for groupName in Group['Groups']:
print("Group: " + groupName['GroupName'])
print("----------------------------")
user_group()
I would like to have the ability to run this script in two fashions.
1) add an argument(s) of 'username' so I can get the response for a particular user
2) default to getting response for all users if no argument is given.
This can be done by using an argument with a default value:
def user_group(user = None):
if user is None:
print("No user")
else:
print(user)
user_group()
user_group('some user')
prints
No user
some user
In your case you may want to write
def user_group(user = None):
users_to_list = iam.list_users()['Users'] if user is None else [user]
for myusers in user_to_list:
...
I want to create a lottery skill that takes 6 numbers from the user.
I'm currently learning by going through the samples and developer guides, and I can go through the guides and get a working skill that will take one input and then end the session. But I believe I need to create a dialog somehow, which is where I get stuck.
Design-wise, I'd like the dialog to go like this:
Alexa: Please provide the first number
User: 1
Alexa: and now the second...
User: 2
etc etc
But I think it would be OK if it went like this:
Alexa: Please call out 6 numbers
User: 1, 2, 3, 4, 5, 6.
Is this even possible? Will I have to create a custom slot type called "Numbers" and then put in the numbers, eg 1-50 or whatever the limit is?
At best, I can currently get it to ask for one number, so its really the dialog interaction that I'm stuck on. Has anyone ever even done anything like this?
Thanks.
Yes to both questions. You could string together a response with 6 different custom slots. "User: My numbers are {num1}, {num2}, {num3}, {num4}, {num5}, {num6} " and make them all required using the skills beta developer. However, it will be a rather bad user experience if the user does not phrase their answer appropriately and Alexa has to ask follow up questions to obtain each number. The last problem you'll run into is that while a custom slot could be defined to contain the numbers 1-50 alexa will generally recognize similar values to those provided in a custom slot, such as numbers from 50-99. It would then be up to you to check that the values you receive are between 1 and 50. If not you'd want to ask the user to provide a different number in the appropriate range.
Conclusion: You'll want to have individual interactions where a user provides a single number at a time.
Alexa:"you will be prompted for 6 numbers between 1 and 50 please state them one at a time. Choose your first number."
User:"50"
Alexa:"Your First number is 50, Next number."...
You can implement this using a single intent. let's name that intent GetNumberIntent. GetNumberIntent will have sample uterances along the line of
{number}
pick {number}
choose {number}
where {number} is a custom slot type or simply AMAZON.NUMBER. It will then be up to you to check that the number is between 1 and 50.
I program in Node.js using the SDK. Your implementation may vary depending upon your language choice.
What I would do is define 6 different state handlers. Each handler should have the GetNumberIntent. When a GetNumberIntent is returned if the slot value is apropriate store the value to the session data and or dynamodb and move forward to the next state. If the slot value is invalid stay for example at state "NumberInputFiveStateHandlers" until a good value is received then change state to the next "NumberInputSixStateHandlers"
var NumberInputFiveStateHandlers = Alexa.CreateStateHandler(states.NUMFIVEMODE, {
'NewSession': function () {
this.emit('NewSession'); // Uses the handler in newSessionHandlers
},
//Primary Intents
'GetNumberIntent': function () {
let message = ` `;
let reprompt = ` `;
let slotValue = this.event.request.intent.slots.number.value;
if(parseInt(slotValue) >= 1 && parseInt(slotValue) <= 50){
this.handler.state = states.NUMSIXMODE;
this.attributes['NUMBERFIVE'] = this.event.request.intent.slots.number.value;
message = ` Your fifth number is `+slotValue+`. please select your sixth value. `;
reprompt = ` please select your sixth value. `;
}else{
message = ` The number `+slotValue)+` is not in the desired range between 1 and 50. please select a valid fifth number. `;
reprompt = ` please select your fifth value. `;
}
this.emit(':ask',message,reprompt);
},
//Help Intents
"InformationIntent": function() {
console.log("INFORMATION");
var message = ` You've been asked to choose a lottery number between 1 and 50. Please say your selection.`;
this.emit(':ask', message, message);
},
"AMAZON.StopIntent": function() {
console.log("STOPINTENT");
this.emit(':tell', "Goodbye!");
},
"AMAZON.CancelIntent": function() {
console.log("CANCELINTENT");
this.emit(':tell', "Goodbye!");
},
'AMAZON.HelpIntent': function() {
var message = `You're playing lottery. you'll be picking six numbers to play the game. For help with your current situation say Information. otherwise you may exit the game by saying quit.`;
this.emit(':ask', message, message);
},
//Unhandled
'Unhandled': function() {
console.log("UNHANDLED");
var reprompt = ' That was not an appropriate response. Please say a number between 1 and 50.';
this.emit(':ask', reprompt, reprompt);
}
});
This is an example of the fifth request. You'll have 6 identical states like this one that string back to back. Eventually you'll end up with 6 session values.
this.attributes['NUMBERONE']
this.attributes['NUMBERTWO']
this.attributes['NUMBERTHREE']
this.attributes['NUMBERFOUR']
this.attributes['NUMBERFIVE']
this.attributes['NUMBERSIX']
You can then use these values for your game.
If you have not used the alexa-sdk before you must remember to register your state handlers and add your modes to the states variable.
alexa.registerHandlers(newSessionHandlers, NumberInputOneStateHandlers, ... NumberInputSixStateHandlers);
var states = {
NUMONEMODE: '_NUMONEMODE',
...
...
NUMSIXMODE: '_NUMSIXMODE',
}
This answer is not intended to cover the basics of coding using Alexas-SDK. There are other resourced for more specific questions on that topic.
Alternatively, because your intent is identical [GetNumberIntent], you may be able to get by with a single StateHandler that pushes new valid numbers onto an array until the array is the desired length. That would simply require more logic inside the Intent Handler and a conditional to break out of the state once the array is of length 6.
Try the code above first because it's easier to see the different states.
EDIT: I accidentally misrepresented the problem when trying to pare-down the example code. A key part of my code is that I am attempting to sort the array after adding elements to it. The hang appears on sort, not insert. The following abstracted code will consistently hang:
<?=
local('a' = array)
#a->insert('test1' = map('a'='1'))
#a->insert('test2' = map('b'='2')) // comment-out to make work
#a->sort
#a
?>
I have a result set for which I want to insert a pair of values into an array for each unique key, as follows:
resultset(2) => {
records => {
if(!$logTypeClasses->contains(field('logTypeClass'))) => {
local(i) = pair(field('logTypeClass'), map('title' = field('logType'), 'class' = field('logTypeClass')))
log_critical(#i)
$logTypeClasses->insert(#i) // Lasso hangs on this line, will return if commented-out
}
}
}
Strangely, I cannot insert the #i local variable into thread variable without Lasso hanging. I never receive an error, and the page never returns. It just hangs indefinitely.
I do see the pairs logged correctly, which leads me to believe that the pair-generating syntax is correct.
I can make the code work as long as the value side of the pair is not a map with values. In other words, it works when the value side of the pair is a string, or even an empty map. As soon as I add key=value parameters to the map, it fails.
I must be missing something obvious. Any pointers? Thanks in advance for your time and consideration.
I can verify the bug with the basic code you sent with sorting. The question does arise how exactly one sorts pairs. I'm betting you want them sorted by the first element in the pair, but I could also see the claim that they should be sorted by last element in the pair (by values instead of by keys)
One thing that might work better is to keep it as a map of maps. If you need the sorted data for some reason, you could do map->keys->asArray->sort
Ex:
local(data) = map('test1' = map('a'=2,'b'=3))
#data->insert('test2' = map('c'=33, 'd'=42))
local(keys) = #data->keys->asArray
#keys->sort
#keys
Even better, if you're going to just iterate through a sorted set, you can just use a query expression:
local(data) = map('test1' = map('a'=2,'b'=3))
#data->insert('test2' = map('c'=33, 'd'=42))
with elm in #data->eachPair
let key = #elm->first
let value = #elm->second
order by #key
do { ... }
I doubt you problem is the pair with map construct per se.
This test code works as expected:
var(testcontainer = array)
inline(-database = 'mysql', -table = 'help_topic', -findall) => {
resultset(1) => {
records => {
if(!$testcontainer->contains(field('name'))) => {
local(i) = pair(field('name'), map('description' = field('description'), 'name' = field('name')))
$testcontainer->insert(#i)
}
}
}
}
$testcontainer
When Lasso hangs like that with no feedback and no immediate crash it is usually trapped in some kind of infinite loop. I'm speculating that it might have to do with Lasso using references whenever possible. Maybe some part of your code is using a reference that references itself. Or something.