I'm using the CalcHistogram endpoint to query the total number of paper entities for every year from around 1980 to 2018. A typical response looks like this:
{
"expr": "Y=2001",
"num_entities": 4179575,
"histograms": []
}
That's 4179575 papers from the year 2001.
However, starting at around year 2002 (the exact year is not consistent), the return values are being aborted.
{
"expr": "Y=2002",
"histograms": [],
"aborted": true
}
This is what my request looks like. I've tried using both GET and POST methods.
GET
https://api.labs.cognitive.microsoft.com/academic/v1.0/calchistogram? expr=Y=2002&model=latest&count=10&offset=0 HTTP/1.1
Host: api.labs.cognitive.microsoft.com
Any ideas on why this is being aborted or how I can find the total number of papers for each year?
Thanks!
Found that CalcHistogram endpoint also accepts a timeout parameter. The api will try to evaluate the query until timeout (which defaults to 1000 milli) is reached, at which point it returns aborted: true.
So, just add timeout: 5000 to your request.
Related
Does anyone know whether the HERE Isoline Routing API has a limit on how many hours you can plug in?
For example, 6 hours works perfectly fine and has a service area output:
'https://isoline.router.hereapi.com/v8/isolines?transportMode=car&origin=52.51578,13.37749&range[type]=distance&range[values]=21600&routingMode=fast'
However, I get a error if I try and do 12 hours
'https://isoline.router.hereapi.com/v8/isolines?transportMode=car&origin=52.51578,13.37749&range[type]=distance&range[values]=43200&routingMode=fast'
In .parse_response(i, out$responses()[[i]]) :
https://isoline.router.hereapi.com/v8/isolines: Request 'id = 1' failed.
Status 400; Bad Request; Bad request syntax or unsupported method.
Is there a max time you can use on the API?
Thanks!
The maximum value for the time is 9 hours.
I can write a query in application insights that gives me a percentage as a scalar. I want to create alert if that percentage is > X . How can this be done using log based alerts?
Basically, I have a lot of machines that send telemetry to application insights. Sometimes they log some exceptions. I send MachineName in customDimensions for all the logs. So I can get the names of all the machines that sent logs in last 24 hours. The exceptions are also sent with MachineName in customDimensions. When a particular error is raised by more than X% machines in last 24 hours, I want to raise an alert.
The way to write alert logic is using 'Number of Results' which cannot be used for this since it automatically adds '|count' to the query. The other way is using 'Metric Measurement', which I am guessing should help me raise an alert like this but I'm unable to figure out how.
I can get the total machine count by this query:
let num_machines = traces
| summarize by tostring(customDimensions["MachineName"])
| count;
I can get the number of machines that reported an exception like this:
let num_error_machines = exceptions
| where customDimensions["Message"] contains "ExceptionXRaised"
| summarize by tostring(customDimensions["MachineName"])
| count;
finally, i can get the percentage of machines that raised the issue like this:
print toscalar(num_error_machines)*100/toscalar(num_machines)
I am not sure how to use this result to raise an alert using MetricMeasurement. This needs to be modified somehow to get AggregatedValue and use bin, I am not sure if that is possible / how that query will be.
Sorry for the late reply. I've tested in my side and met many problems indeed.
I found that alert rule doesn't support to monitor the percentage number of the result, it only supports the numbers of query result and Metric measurement. So I think you may give up the percentage and use the num_err_machine like the screenshot below
Pls note, you can't append " ; " at the end of the query or it will give an error like The request had some invalid properties
Im trying to update an entity using the PATCH /v2/entities/id/attrs endpoint. The service of the id is known; however, the servicepath is not.
My first attempt was to leave the header empty or rather not send it at all. For a GET query, this does the job right, however for a PATCH, this returns me a 200. First question, is this the expected behavior?
Second try was to use a wildcard. I know that my entity is at SP /automatization/<something>. I tried PATCHing to /automatization/# and got a 204, but the entity did not change.
My last try was to use a list of known SPs: I know the entity I want is in one of a number of SPs, so a used a comma separated list as the SP header. Same as above: 204 but no change.
Am I missing something?
Orion version:
{
"orion": {
"version": "2.3.0-next",
"uptime": "2 d, 1 h, 50 m, 52 s",
"git_hash": "c99fef1b5c93bf1673d407968c2c101f01b95980",
"compile_time": "Tue Mar 10 09:46:52 UTC 2020",
"compiled_by": "root",
"compiled_in": "0a42bdf43030",
"release_date": "Tue Mar 10 09:46:52 UTC 2020",
"doc": "https://fiware-orion.rtfd.io/"
}
}
It's unfortunate that the service path isn't returned in the payload data of the queries. That would have solved your problem. If I were you (and if you have access to the DB) I'd use the mongo shell to find the service path of your entity.
About getting a 204, that seems like a bug. The entity you want to patch isn't found (as you put in the wrong service path) and a 404 should be returned.
Lists and wildcards as service paths are only for queries, never for creation/updates. The broker should complain there too, with a 400 and 'Invalid Service-path', if you ask me ...
Seems like we have two issues here, apart from the one to include service path in the response ...
So I have a basic
$.ajax( { url : 'MyController/MyAction',
method : 'POST',
async: true,
.
. } );
that can be called very frequently as it is event-driven. Like it may be called 50 times in 1 second if the user is being obnoxious. It updates values in the database.
My friend told me that it's possible that the updates may be sent to the database in the wrong order. Is this true? This is causing me major cognitive dissonance and I can't sleep tonight.
I should mention that these values being updates in the database are associated with a user. In particular, the data is like
data : { userId : '21EC2020-3AEA-4069-A2DD-08002B30309D',
answerId : '69',
val : 'd' }
where the only values changing in rapid succession are answerId and val.
Unfortunately, my understand is "YES. The order is not guaranteed".
You send http request 50 times in 1 second and server save it to DB.
When the network is good and server is strong, it is okay to be saved to DB in order.
But if http sever is busy or network is interrupted, it does not guarantee the data will still always ordered in real happened sequence. Ex, 1 or 2 data order will be exchanged in DB.
My suggestion is : if the order is very critical and the update traffic is heavy, you should add a happen time in http data and save it to DB.
When you select data from DB, you can order by the happen time and it will make sure it has the correct order as event happens to avoid the mis-order caused by server or network busy.
I'm using Graphite and Collectd to monitor my server. In particular, I'm using the tail pluggin to count failed SSH logins. I'm using a counter for this metric, so expect to see 1, 2, 3, 0, etc... for data points. However, what I'm seeing is 0.1, 0.2, 0.3, 0, etc... It seems to me like Graphite is providing counts-per-second. I say this because my retention policy is one data point every 10 seconds for two hours. So 1 failed login per 10 seconds = 0.1 per second. I'm looking at this in a graph. It looks like this:
Furthermore, when I scale out to the next retention level, the numbers get adjusted accordingly: so 1 failed login which was shown as 0.1 is now shown as much less than this: 0.017 or something.
I don't think this is related to the aggregation method used: even the finest data is off. How can I get Graphite to treat this metric as a pure, raw, counter?
Here's my storage-schemas.conf (the retention policy):
[my_server]
pattern = .*
retentions = 10s:2h,1m:2d,30m:400d
Here's my configuration of the collectd tail plugin:
<Plugin "tail">
<File "/var/log/auth.log">
Instance "auth"
<Match>
Regex "sshd[^:]*: Failed password"
DSType "CounterInc"
Type "counter"
Instance "sshd-invalid_user"
</Match>
</File>
</Plugin>
And here's my configuration of the write_graphite pluggin (which sends data to graphite):
<Plugin write_graphite>
<Node "my_server_name">
Host "localhost"
Port "2003"
Protocol "tcp"
LogSendErrors true
Prefix "collectd."
#Postfix ""
StoreRates true
AlwaysAppendDS false
EscapeCharacter "_"
</Node>
</Plugin>
I tried setting StoreRates false for the write_graphite pluggin, but this didn't work. It did change the behaviour: when I performed a single failed SSH login, that metric shows as 1. However, it didn't drop back down to 0. When I performed two more failed logins, the metric pops up to 3.
Also of interest: I've also loaded the users pluggin which simply shows the number of users logged in and it's working great: shows 1 when I SSH in, two when I SSH in again, and back to 1 when I exit one SSH. For both settings of StoreRates. So it seems like what I want is possible somehow. Maybe not with the tail pluggin though.
The SSH logins with StoreRates false along with correct behaviour for Users Logged in can be seen in these graphs:
Any ideas? Thanks,
You are asking the system to count the number of events. And this is exactly what it's doing: it's counting the number of failed logins since its startup. Whether you're using StoreRates or not simply changes the way that information is displayed: as a rate or as the raw counter. A counter may never decrease! What you're actually asking for is a counter that resets itself upon reading: count the number of failed logins since the last time collectd checked.
As it happens the ABSOLUTE data source type in rrdtool can be used to achieve this, but that won't help you.
Step back, and think about what you're trying to achieve: the number of failed logins per second seems to me like a perfectly sane metric!
Although swissunix's answer is very helpful, to achieve the behaviour I was looking for, I ended up using Logster instead of Collectd. With Logster, you write the bit of code that parses the file as well as the bit that returns the metric. So although dividing a count by the time is common with Logster, you don't have to do this if you don't want to: there's lots of flexibility.
I've put my parsers here: https://github.com/camlee/logster-parsers
If you set StoreRates to false, in graphite you can apply the derivative function to the ever-increasing counter to get your rate of increase per retention interval, which would match your requirement.
E.g. in your example of reporting 1 failed login, then 2, you saw the values 1 and 3. The derivative is 1 and 2: the failed logs per interval that graphite tracks.