I have a data set like this for single server. So multiple servers are having multiple records like this in a kusto table eg Table1.
TimeStamp State Servername Type
7/13/2021 Healthy abcdefgh Server
7/13/2021 Repair abcdefgh Server
7/14/2021 Repair abcdefgh Server
7/15/2021 Repair abcdefgh Server
7/15/2021 Healthy abcdefgh Server
7/15/2021 Healthy abcdefgh Server
7/16/2021 Repair abcdefgh Server
7/17/2021 Repair abcdefgh Server
7/17/2021 Repair abcdefgh Server
7/17/2021 Repair abcdefgh Server
7/18/2021 Repair abcdefgh Server
7/18/2021 Repair abcdefgh Server
7/19/2021 Repair abcdefgh Server
7/19/2021 Repair abcdefgh Server
I need to know the list of servers which are stuck in repair state for more than 10 days at once. (query should not add the previous repair state time as for the above server it was in repair and came to healthy). Just need the consecutive repair time.
Can someone help?
I have made the following query but this does not work as expected.
Table1
| order by TIMESTAMP desc
| extend prevstate = prev(State)
| where prevstate == State
| summarize arg_max(TIMESTAMP,*) by Servername //summarizing the top time//
| extend endtime = TIMESTAMP //assigning the top time//
| join kind= innerunique (Table1) //joining same table to find the startdate//
on Servername
| order by TIMESTAMP desc
| extend prevstate = prev(State)
| where prevstate == State
| summarize arg_min(TIMESTAMP,*) by Servername //summarizing the start time//
| extend starttime= TIMESTAMP
| extend Duration = datetime_diff('day',endtime,starttime)
| project Server, State, Duration
First group records into uninterrupted partitions with the same Servername and State, then check whether the oldest and youngest observations are more than 10 days apart:
datatable(TimeStamp:datetime, Servername:string, State:string)
[
'2021-07-01', 'abc', 'Healthy',
'2021-07-02', 'abc', 'Repair',
'2021-07-04', 'def', 'Healthy',
'2021-07-05', 'abc', 'Healthy',
'2021-07-07', 'abc', 'Repair',
'2021-07-10', 'def', 'Healthy',
'2021-07-18', 'abc', 'Repair',
]
| order by Servername, TimeStamp
| extend new_partition = Servername != prev(Servername) or State != prev(State)
| extend partition_id = row_cumsum(toint(new_partition))
| where State == 'Repair'
| summarize
repair_start = min(TimeStamp),
repair_last = max(TimeStamp)
by Servername, partition_id
| project Servername, repair_start, repair_last, duration = repair_last - repair_start
| where duration > 10d
Related
I am converting a Splunk Dasboard to a Kusto Dash board
Below is the Splubk Query that give me the minites since as 60
index=V source=c:\\files\\build\\* | head 1 | eval minutesSince=round((now() - _indextime)/60,0) | table minutesSince
I try to create the same in Kusto but the time is not matching gives me 0 minutesSince. But the Data is the same in Splunk and Kusto. Not sure what part of the Query should I correct. Thanks for the support.
| extend minutesSince = (now() - ingestion_time())/60
| project minutesSince,BuildId
| limit 1```
you could try something like the following:
for the entire table:
TableName
| summarize minutes_since_last_ingestion = (now() - max(ingestion_time())) / 1m
or, per record:
TableName
| extend minutes_since_ingestion = (now() - ingestion_time()) / 1m
We're running into Kusto has_any limit of 10K.
Sample code
// Query: Get failed operations for migrated apps
let migrationsTimeDiff = 15d;
let operationsDiffTime = 24h + 1m;
let migratedApps = FirstTable
| where TimeStamp >= ago(migrationsTimeDiff)
| where MetricName == "JobSucceeded"
| project
MigrationTime = PreciseTimeStamp,
appName = tostring(parse_json(Annotations).AppName)
| project appName;
SecondTable
| where TimeStamp > ago(operationsDiffTime)
| where Url has_any (appName)
| where Result == "Fail"
Is there a way to restructure the query via joins?
Alternatively is it possible to loop in batches of 10k?
Thanks for reading!
If Url is an exact match to appName, then you should use:
SecondTable
| where TimeStamp > ago(operationsDiffTime)
| where Url in (appName) // 'in' instead of 'has_any'
| where Result == "Fail"
Otherwise, you'll need to extract the application name from the Url using extend, and then use in like I suggested above, so your query will look like this:
SecondTable
| where TimeStamp > ago(operationsDiffTime)
| extend ExtractedAppNameFromUrl = ...
| where ExtractedAppNameFromUrl in (appName) // 'in' instead of 'has_any'
| where Result == "Fail"
I have Sessions table
Sessions
|Timespan|Name |No|
|12:00:00|Start|1 |
|12:01:00|End |2 |
|12:02:00|Start|3 |
|12:04:00|Start|4 |
|12:04:30|Error|5 |
I need to extract from it duration of each session using KQL (but if you could give me suggestion how I can do it with some other query language it would be also very helpful). But if next row after start is also start, it means session was abandoned and we should ignore it.
Expected result:
|Duration|SessionNo|
|00:01:00| 1 |
|00:00:30| 4 |
You can try something like this:
Sessions
| order by No asc
| extend nextName = next(Name), nextTimestamp = next(timestamp)
| where Name == "Start" and nextName != "Start"
| project Duration = nextTimestamp - timestamp, No
When using the operator order by, you are getting a Serialized row set, which then you can use operators such as next and prev. Basically you are seeking rows with No == "Start" and next(Name) == "End", so this is what I did,
You can find this query running at Kusto Samples open database.
let Sessions = datatable(Timestamp: datetime, Name: string, No: long) [
datetime(12:00:00),"Start",1,
datetime(12:01:00),"End",2,
datetime(12:02:00),"Start",3,
datetime(12:04:00),"Start",4,
datetime(12:04:30),"Error",5
];
Sessions
| order by No asc
| extend Duration = iff(Name != "Start" and prev(Name) == "Start", Timestamp - prev(Timestamp), timespan(null)), SessionNo = prev(No)
| where isnotnull(Duration)
| project Duration, SessionNo
I installed an InnoDB Cluster recently and trying to create a table without any primary key or equivalent to test the cluster index concept where "InnoDB internally generates a hidden clustered index named GEN_CLUST_INDEX on a synthetic column containing row ID values if the table has no PRIMARY KEY or suitable UNIQUE index".
I created table as below:
create table repl_test (Name varchar(10));
Checked for the creation of GEN_CLUST_INDEX:
select * from mysql.innodb_index_stats where database_name='test' and table_name = 'repl_test';
+---------------+------------+-----------------+---------------------+--------------+------------+-------------+-----------------------------------+
| database_name | table_name | index_name | last_update | stat_name | stat_value | sample_size | stat_description |
+---------------+------------+-----------------+---------------------+--------------+------------+-------------+-----------------------------------+
| test | repl_test | GEN_CLUST_INDEX | 2019-02-22 06:29:26 | n_diff_pfx01 | 0 | 1 | DB_ROW_ID |
| test | repl_test | GEN_CLUST_INDEX | 2019-02-22 06:29:26 | n_leaf_pages | 1 | NULL | Number of leaf pages in the index |
| test | repl_test | GEN_CLUST_INDEX | 2019-02-22 06:29:26 | size | 1 | NULL | Number of pages in the index |
+---------------+------------+-----------------+---------------------+--------------+------------+-------------+-----------------------------------+
3 rows in set (0.00 sec)
But, when I try to insert row, I get the below error:
insert into repl_test values ('John');
ERROR 3098 (HY000): The table does not comply with the requirements by an external plugin.
2019-02-22T14:32:53.177700Z 594022 [ERROR] [MY-011542] [Repl] Plugin group_replication reported: 'Table repl_test does not have any PRIMARY KEY. This is not compatible with Group Replication.'
Below is my conf file:
[client]
port = 3306
socket = /tmp/mysql.sock
[mysqld_safe]
socket = /tmp/mysql.sock
[mysqld]
socket = /tmp/mysql.sock
port = 3306
basedir = /mysql/product/8.0/TEST
datadir = /mysql/data/TEST/innodb_data
log-error = /mysql/admin/TEST/innodb_logs/mysql.log
log_bin = /mysql/binlog/TEST/innodb_logs/mysql-bin
server-id=1
max_connections = 500
open_files_limit = 65535
expire_logs_days = 15
innodb_flush_log_at_trx_commit=1
sync_binlog=1
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
binlog_checksum = NONE
enforce_gtid_consistency = ON
gtid_mode = ON
relay-log=<<hostname>>-relay-bin
My MySQL version: mysql Ver 8.0.11 for linux-glibc2.12 on x86_64 (MySQL Community Server - GPL)
Auto-generation of the PK should work fine for non-clustered setups. However, InnoDB Cluster (Group Replication) and Galera need a user PK (or UNIQUE that can be promoted).
If you like, file a documentation bug with bugs.mysql.com complaining that this restriction is not clear. Be sure to point to the page that needs fixing.
In APS 1.8.1, I have defined a process where each task has a candidate group.
When I login in with a user that belongs to a candidate group, I cannot see the process instance.
I have found out that when I try to access the process instances, APS executes the following query in the database:
select distinct RES.* , DEF.KEY_ as PROC_DEF_KEY_, DEF.NAME_ as PROC_DEF_NAME_, DEF.VERSION_ as PROC_DEF_VERSION_, DEF.DEPLOYMENT_ID_ as DEPLOYMENT_ID_
from ACT_HI_PROCINST RES
left outer join ACT_RE_PROCDEF DEF on RES.PROC_DEF_ID_ = DEF.ID_
left join ACT_HI_IDENTITYLINK I_OR0 on I_OR0.PROC_INST_ID_ = RES.ID_
WHERE RES.TENANT_ID_ = 'tenant_1'
and
( (
exists(select LINK.USER_ID_ from ACT_HI_IDENTITYLINK LINK where USER_ID_ = '1003' and LINK.PROC_INST_ID_ = RES.ID_)
)
or (
I_OR0.TYPE_ = 'participant'
and
I_OR0.GROUP_ID_ IN ('1','2','2023','2013','2024','2009','2025','2026','2027','2028','2029','2007','2018','2020','2017','2015','2012','2003','2021','2019','2004','2002','2005','2030','2031','2032','2011','2006','2008','2014','2010','2016','2022','2033','2034','2035','2036','2037','1003')
) )
order by RES.START_TIME_ desc
LIMIT 50 OFFSET 0
This query does not return any record for two reasons:
In my ACT_HI_IDENTITYLINK no tasks have both the group_id_ and the proc_inst_id_ set.
The type of the record is "candidate" but the query is looking for "participant"
select * fro m ACT_HI_IDENTITYLINK;
-[ RECORD 1 ]-+----------
id_ | 260228
group_id_ |
type_ | starter
user_id_ | 1002
task_id_ |
proc_inst_id_ | 260226
-[ RECORD 2 ]-+----------
id_ | 260294
group_id_ | 2006
type_ | candidate
user_id_ |
task_id_ | 260293
proc_inst_id_ |
-[ RECORD 3 ]-+----------
id_ | 260300
group_id_ | 2009
type_ | candidate
user_id_ |
task_id_ | 260299
proc_inst_id_ |
-[ RECORD 4 ]-+----------
id_ | 262503
group_id_ |
type_ | starter
user_id_ | 1002
task_id_ |
proc_inst_id_ | 262501
-[ RECORD 5 ]-+----------
id_ | 262569
group_id_ | 2016
type_ | candidate
user_id_ |
task_id_ | 262568
proc_inst_id_ |
-[ RECORD 6 ]-+----------
id_ | 262575
group_id_ | 2027
type_ | candidate
user_id_ |
task_id_ | 262574
proc_inst_id_ |
Why the query is looking only for "participant" and why the records that have type_ = 'candidate' do not have any proc_inst_id_ set ?
UPDATE:
The problem with the constraint "participant" has a simple workaround: it would be enough to add the same candidate group as a participant.
See also Feature allowing "Participant" configuration in BPM Modeler
Unfortunately this is not enough to solve the second problem. The record is still not returned because the column proc_inst_id_ is not set.
I tried to update the column manually on the "participant" record and I have verified that doing so the page is accessible and works well.
Does anyone know why the column is not set ?
A possible solution (or workaround until ACTIVITI-696 is fixed) is to add each group added as candidate to a task as a participant of the process instance.
There is a REST API that does it:
POST /enterprise/process-instances/{processInstanceId}/identitylinks
What this API does should be done by a task listener that will automatically add the candidate groups of the created task as participant of the process instance.
To add the new identity link, use in the listener the following lines:
ActivitiEntityEvent aee = (ActivitiEntityEvent)activitiEvent;
TaskEntity taskEntity = (TaskEntity)aee.getEntity();
List<IdentityLinkEntity> identities = taskEntity.getIdentityLinks();
if (identities != null) {
for (IdentityLinkEntity identityLinkEntity : identities) {
String groupId = identityLinkEntity.getGroupId();
runtimeService.addGroupIdentityLink(activitiEvent.getProcessInstanceId(), groupId, "participant");
};
}
first try to check that your workflow is really started by access to "workflow I have started". You should see your task in "active task" if not, that means there is some errors in your definitions. If everything is ok, check your group name and don’t forget to add "GROUP_"myGRPName.
If you want to see the workflow instances it’s simpler with web script and services.