How does Spunk prioritize conditional case functions? Lets say I have a case function with 2 conditions - they work fine, and results are as expected, but then lets say I flip the conditions. What I see happen when I flip the conditions in the case function the results are not correct. Shouldn't Splunk be able to still check which condition it applies to even though I have flipped the conditions? Example below:
Case: TimeSchedule should output the closest 7th min or 37th min - so every half hour past the 7th min or 37th min from the zipTime_epoch.
Works Fine as output TimeSchedule should be 2021-03-06 23:37:59.000000
| makeresults
| eval zipTime="2021-03-06 23:35:59.000"
| eval zipTime_epoch=strptime(zipTime, "%Y-%m-%d %H:%M:%S.%6N")
| eval lastunzip_hour=tonumber(strftime(zipTime_epoch, "%H"))
| eval lastunzip_min=tonumber(strftime(zipTime_epoch, "%M"))
| eval lastunzip_sec=round(zipTime_epoch%60,6)
| eval TimeSchedule=strftime(case(lastunzip_min%30 < 7, zipTime_epoch-
zipTime_epoch%1800+420+lastunzip_sec,lastunzip_min!=37 AND lastunzip_min!=7, zipTime_epoch-zipTime_epoch%1800+2220+lastunzip_sec,1=1,zipTime_epoch),"%Y-%m-%d %H:%M:%S.%6N")
Does not work fine when case in conditions are flipped- output should be 2021-03-06 23:37:59.000000 instead.
| makeresults
| eval zipTime="2021-03-06 23:35:59.000"
| eval zipTime_epoch=strptime(zipTime, "%Y-%m-%d %H:%M:%S.%6N")
| eval lastunzip_hour=tonumber(strftime(zipTime_epoch, "%H"))
| eval lastunzip_min=tonumber(strftime(zipTime_epoch, "%M"))
| eval lastunzip_sec=round(zipTime_epoch%60,6)
| eval TimeSchedule=strftime(case( lastunzip_min!=37 AND lastunzip_min!=7, zipTime_epoch-
zipTime_epoch%1800+2220+lastunzip_sec,lastunzip_min%30 < 7, zipTime_epoch_epoch- zipTime_epoch_epoch%1800+420+lastunzip_sec,1=1,zipTime_epoch),"%Y-%m-%d %H:%M:%S.%6N")
| table TimeSchedule, zipTime, lastunzip_hour, lastunzip_min, lastunzip_sec, zipTime_epoch
The case function evaluates each case in the order given. The first to evaluate to true is the one that prevails.
In the example, lastunzip_min is 35. 35%30 is 5. 35 is neither 37 nor 7 and 5<7. That makes both conditions in the case function true so the order of processing is significant.
Related
I am trying to display result using Kusto KQL query in pie chart.The goal is to display pie chart as half n half color in case of failure and full color in case of pass.
Basically log from a site displays rows as pass and failed row .In case where all are pass , pie chart should display 100 % same color.In case of even single failure in any rows , it should display 50% one color and 50% other color.Below query works when 1) When all rows are pass as full color 2) when some are pass and some fail or even one fails (displays pie chart in half n half) color 3)BUT WHEN ALL ROW HAS FAILS ,this is displaying in one color and not splitting pie chart in half n half
QUERY I USED:
results
| where Name contains "jobqueues"
| where timestamp > ago(1h)
| extend PASS = (ErLvl)>2 )
| extend FAIL = ((ErLvl<2 )
| project PASS ,FAIL
| extend status = iff(PASS==true,"PASS","FAIL")
| summarize count() by status
| extend display = iff(count_>0,1,0)
| summarize percentile(display, 50) by status
| render piechart
Please suggest what can be done to solve this problem.Thanks in advance.
Let's summarize your question:
There are only two outcomes of your query:
A piechart showing 50% vs 50%
A piechart showing 100%
From your description we learn that when
All rows are PASS we plot piechart 2.
Any row has FAIL we plot piechart 1.
Lets see how we can achieve this after this line from your code:
| extend status = iff(PASS==true,"PASS","FAIL")
| summarize count() by status
We should have a table looking like so:
status
count_
PASS
x
FAIL
y
Looks like we need to perform some logic on this. You were originally plotting based on the operation result. My idea was to just generate a table of pass = 1 and fail = 1 for the 50%v50% case and another table of pass = 1 and fail = 0 for the 100% case.
So following that logic we need to perform the following mapping:
status
count_
status
count2
fail
>0
maps to
fail
1
pass
>0
pass
1
status
count_
status
count2
fail
>0
maps to
fail
1
pass
=0
pass
1
status
count_
status
count2
fail
=0
maps to
fail
0
pass
>0
pass
1
Logical representation:
(given count_ >=0):
if fail > 0 count2 = 0 else count 1
pass is always equal to 1
We only need to apply this to the row where status == FAILED but summarize doesn't guarantee a row if there are no observations
Guarantee summarize results:
| extend fail_count = iif(status == "FAIL", count_, 0)
| extend pass_count = iif(status == "PASS", count_, 0)
| project fail_count,pass_count
| summarize sum(fail_count), sum(pass_count)
Apply logic
| extend FAIL = iff(sum_fail_count > 0, 1, 0)
| extend PASS = 1
| project FAIL, PASS
Now our result is looking like:
PASS
FAIL
1
1 or 0
In order to plot this as a pie chart we just need to transpose it so the columns PASSED and FAILED are rows of the "status" column.
We can use a simple pack and mv-expand for this
//transpose for rendering
| extend tmp = pack("FAIL",FAIL,"PASS",PASS)
| mv-expand kind=array tmp
| project Status=tostring(tmp[0]), Count=toint(tmp[1])
| render piechart
And that's it!~
Final query:
results
| where Name contains "jobqueues"
| where timestamp > ago(1h)
| extend PASS = (ErLvl)>2 )
| extend FAIL = ((ErLvl<2 )
| project PASS ,FAIL
| extend status = iff(PASS==true,"PASS","FAIL")
| summarize count() by status
//ensure results
| extend fail_count = iif(status == "FAIL", count_, 0)
| extend pass_count = iif(status == "PASS", count_, 0)
| project fail_count,pass_count
| summarize sum(fail_count), sum(pass_count)
//apply logic
| extend FAIL = iff(sum_fail_count > 0, 1, 0)
| extend PASS = 1
| project FAIL, PASS
//transpose for rendering
| extend Temp = pack("FAIL",FAIL,"PASS",PASS)
| mv-expand kind=array Temp
| project Status=tostring(Temp[0]), Count=toint(Temp[1])
| render piechart
The following query returns the data that I need:
let timeSpn = bin(ago(60m),1m);
requests
| where cloud_RoleName == "myApp"
| where success == "False"
| where timestamp > timeSpn
| make-series count() on timestamp from timeSpn to now() step 1m by application_Version
The problem is that the result consist of 2 lines (one for each application_Version and not 120 lines (one for each minute and for each version).
I have to use make-series and not the simple summarize because I need the "zero" values.
You can do it using the mv-expand operator
Here's an example from Back-fill Missing Dates With Zeros in a Time Chart:
let start=floor(ago(3d), 1d);
let end=floor(now(), 1d);
let interval=5m;
requests
| where timestamp > start
| make-series counter=count() default=0
on timestamp in range(start, end, interval)
| mvexpand timestamp, counter
| project todatetime(timestamp), toint(counter)
| render timechart
I would like to know how I could eliminate nothing elements in a Julia array (1D) like the one below. It was built from reading a text file with lines with no relevant information mixed with lines with relevant information. "nothing" is type Void and I would like to clean the array of all of it.
nothing
nothing
nothing
nothing
nothing
" -16.3651\t 0.1678\t -4.6997\t -14.0152\t -2.6855\t -16.0294\t -7.8049\t -27.1912\t -5.0354\t -14.5187\t\r\n"
" -16.4490\t -1.0910\t -3.6087\t -12.6724\t -1.5945\t -14.7705\t -7.2174\t -25.2609\t -3.7766\t -14.3509\t\r\n"
" -16.4490\t -2.2659\t -2.4338\t -10.9100\t -0.5875\t -13.6795\t -6.7139\t -22.9950\t -2.9373\t -14.0991\t\r\n"
testvector[testvector.!=nothing] is also a very readable option.
benchmarking can help choose the most efficient code.
How are you reading that file?
You can filter out nothings from an array:
filter(x -> !is(nothing, x), [nothing, 42]) # => Any[42]
But you may want to clean your data first, with a tsv (tab separated values) file like this:
-16.3651 0.1678 -4.6997 -14.0152 -2.6855 -16.0294 -7.8049 -27.1912 -5.0354 -14.5187
-16.4490 -1.0910 -3.6087 -12.6724 -1.5945 -14.7705 -7.2174 -25.2609 -3.7766 -14.3509
-16.4490 -2.2659 -2.4338 -10.9100 -0.5875 -13.6795 -6.7139 -22.9950 -2.9373 -14.0991
Using readdlm:
julia> readdlm("data.tsv")
3x10 Array{Float64,2}:
-16.3651 0.1678 -4.6997 -14.0152 … -27.1912 -5.0354 -14.5187
-16.449 -1.091 -3.6087 -12.6724 -25.2609 -3.7766 -14.3509
-16.449 -2.2659 -2.4338 -10.91 -22.995 -2.9373 -14.0991
Using DataFrmaes.readtable:
julia> df = readtable("data.tsv");
julia> names!(df, [symbol(x) for x in 'A':'J'])
2x10 DataFrames.DataFrame
| Row | A | B | C | D | E | F | G |
|-----|---------|---------|---------|----------|---------|----------|---------|
| 1 | -16.449 | -1.091 | -3.6087 | -12.6724 | -1.5945 | -14.7705 | -7.2174 |
| 2 | -16.449 | -2.2659 | -2.4338 | -10.91 | -0.5875 | -13.6795 | -6.7139 |
| Row | H | I | J |
|-----|----------|---------|----------|
| 1 | -25.2609 | -3.7766 | -14.3509 |
| 2 | -22.995 | -2.9373 | -14.0991 |
one simple way is using filter! function to update your vector like this:
testvector=[fill(nothing,10) ; [1,2,3]];
# =>13-element Array{Any,1}:
# nothing
# nothing
# nothing
# nothing
# nothing
# nothing
# nothing
# nothing
# nothing
# nothing
# 1
# 2
# 3
filter!(x->x!=nothing, testvector)
# => 3-element Array{Any,1}:
# 1
# 2
# 3
thanks #Daniel Arndt
EDIT, Refer to this paragraph from Julia doc:
nothing is a special value that does not print anything at the
interactive prompt. Other than not printing, it is a completely normal
value and you can test for it programmatically.
I think all of the conditions below, reach us to the same result
x!=nothing
x!==nothing
!is(x,nothing)
!isa(x,Void)
typeof(x)!=Void
To add to the answers above, it appears:
filter(!isnothing, [nothing, 42])
is a working shorthand for filter(x -> !isnothing(x), [nothing, 42]), and will correctly return 42.
Dear All,
At the end, the code became this:
tmpFile=open(fileName)
tmp=readdlm(tmpFile);
ind=pmap(typeof,tmp[:,1]).!=SubString{ASCIIString}; # if the first column typeof is string, than pmap will return false, else, it return true. This will provide an index of valid/not valid rows.
tmpClean=tmp[ind,:]; # only valid rows will be used
If you may have any suggestion to improve it, I would appreciate it. Thank you for your help.
+----------+------------+------+------+--------------+---------+---------+
| | SUBJ | MIN | MAX | RESULT | STATUS | PERCENT |
| +------------+------+------+--------------+---------+---------+
| | Subj1 | 35 | 100 | 13 | FAIL | 13.00% |
|EXAM NAME | Subj2 | 35 | 100 | 63 | PASS | 63.00% |
| | Subj3 | 35 | 100 | 35 | PASS | 35.00% |
| +------------+------+------+--------------+---------+---------+
| | Total | 105 | 300 | 111 | PASS | 37.00% |
+----------+------------+------+------+--------------+---------+---------+
This is my report viewer report format.The SubTotal row counts the
total of all the above column.Every thing is fine. But in the status
column its showing Pass. I want it to show fail if there is single
fail in the status column. I am generating Status if Result < Min then
it is fail or else it is pass. Now how to change the SubTotal row
below depending upon the condition. And is there any way to show the
Subtotal row directly from database. Any suggestion.
The easiest way to do this would be to use custom code (right-click non-display area of report, choose Properties and click the Code tab) - calculate the pass/fail score in the detail, display it in the group footer and reset it in the group header:
Dim PassFail As String
// Reset Pass or Fail status in group header
Public Function ResetAndDisplayStatusTitle() AS String
PassFail = "PASS" // Initialise status to pass
ResetAndDisplayStatusTitle = "Status"
End Function
// Calculate pass/fail on each detail row and remember any fails
Public Function CalculatePassFail() As String
Dim ThisResult As String
// Calculate whether this result is pass or fail
If Fields!Result.Value < Fields!Min.Value Then
ThisResult = "FAIL"
Else
ThisResult ="PASS"
End If
// Remember any failure as overall failure
If ThisResult = "FAIL" Then PassFail = "FAIL"
CalculatePassFail = ThisResult
End Function
Then you tie in the custom code to your cells in your table as follows:
In the value for the status column in your group header you put:
=Code.ResetAndDisplayStatusTitle()
In the value for the status column in the detail row you put:
=Code.CalculatePassFail()
In the value for the status column in the group footer you put:
=Code.PassFail
With respect to getting the subtotal row from the database directly from the database, there are a couple of ways depending on what result you are after.
Join the detail row to a subtotalling row in your SQL (so that the subtotal fields appear on every row in the dataset) and use those fields.
Again, use custom code (but this is probably overly complicated for subtotalling)
However, these tricks are only for strange circumstances and in general the normal out-of-the-box subtotalling can be tweaked to give the result you are after. If there is something specific you want to know, it is probably best to explain the problem in a separate question so that issue can be dealt with individually.
Months ago I ended up with a sub statement that originally worked with my input data. It has since stopped working causing me to re-examine my ugly process. I hate to share it but it accomplished several things at once:
active$id[grep("CIR",active$description)] <- sub(".*CIR0*(\\d+).*","\\1",active$description[grep("CIR",active$description)],perl=TRUE)
This statement created a new id column by finding rows that had an id embedded in the description column. The sub statement would find the number following a "CIR0" and populate the id column iff there was an id within a row's description. I recognize it is inefficient with the embedded grep subsetting either side of the assignment.
Is there a way to have a 'sub' replacement be NA or empty if the pattern does not match? I feel like I'm missing something very simple but ask for the community's assistance. Thank you.
Example with the results of creating an id column:
| name | id | description |
|------+-----+-------------------|
| a | 343 | Here is CIR00343 |
| b | | Didn't have it |
| c | 123 | What is CIR0123 |
| d | | CIR lacks a digit |
| e | 452 | CIR452 is next |
I was struggling with the same issue a few weeks ago. I ended up using the str_match function from the stringr package. It returns NA if the target string is not found. Just make sure you subset the result correctly. An example:
library(stringr)
str = "Little_Red_Riding_Hood"
sub(".*(Little).*","\\1",str) # Returns 'Little'
sub(".*(Big).*","\\1",str) # Returns 'Little_Red_Riding_Hood'
str_match(str,".*(Little).*")[1,2] #Returns 'Little'
str_match(str,".*(Big).*")[1,2] # Returns NA
I think in this case you could try using ifelse(), i.e.,
active$id[grep("CIR",active$description)] <- ifelse(match, replacement, "")
where match should evaluate to true if there's a match, and replacement is what that element would be replaced with in that case. Likewise, if match evaluates to false, that element's replaced with an empty string (or NA if you prefer).