I am having a field named rpc in my elasticsearch database and I am displaying it using Kibana. When I search in search bar of kibana like:
rpc:*
It display all the values of rpc field but I want to have only those value to be displayed which are unique.
I have been playing around with Kibana4 since a couple of weeks now. I find it intuitive and simple and the experience has been great till now. Following your question, I tried getting unique results via a Data Table visualization. Why? Because I personally find it easier to understand. Following are the steps:
1. Get unique count
Create the visualization (Visualize -> Data Table). First lets get
the count of how many unique entries we have for a particular field
(We will use this in the later part for verification). I'm using
clientip.raw but as I see, it will work just fine with any friendly
field name too.
2. Set the aggregation right
Set you aggregation back to count and have a Split Rows as follows. Not doing this will give you count 1 for each field value (since it is looking for unique counts) when you populate the table. Noteworthy part is setting the Top field to 0. Because Kibana won't let you enter anything else than a digit (Obviously!). This was the tricky part. Hit Apply and you'll get the results. Unique field values and the count of each of them.
3. Verification:
Going to the last page of the table, we see there are exactly 543 results. This is how I know it works.
What Next?
You save this visualization and add it to a Dashboard. There you can always check the request, query, response and other stats.
Just an addition to the above mathakoot answer.
For the user of newer version (which do not allow bucket size of 0 anymore) just set a value greater than the maximum number of result
And report the value in the Options>Per Page field
I am using Kibana 6 so the UI looks a bit different than the older answers here.
Here is what worked for me
Create a visualization from your query, I used a line graph type (don't think it matters)
Under Data, set metrics aggregation = "Unique Count" and set field to your field.
Set x-axis aggregation = "Terms" and set field to your field.
Set Size > your number of records
Under Metrics and Axes, disable drawing of the graph, circles, and labels (this really helps the UI not lag)
Run query and then click "Inspect" and download CSV
Data
Metrics & Axes
I wanted to achieve something similar but I'm stuck with Kibana 3.1.
I simply added a panel of type "TERMS" and configured its Field = User-agent and left everything else on default values. This gave me a nice bar chart with one bar for each User-agent.
Related
I have a DataTable that I'm passing to a FlexCel report. It contains a variable number of columns, so I'm using the Full Dataset feature (e.g. <#table_name.*>).
However, only a subset of the fields are dynamically generated (I have a variable number of attachments). The column name for each attachment field starts with a common word (e.g. "Attachment0", "Attachment1", etc).
What I would like to do is output the known finite set of fields and then the variable number of attachments. It would be nice if I could write something like <#table_name.Attachment*> (and <#table_name.Attachment**>). Is there any way in FlexCel Reports I can achieve the same result?
A side benefit to such a solution means that I could keep the formatting for the known/finite set of fields.
Update
I added place holder columns to the document, each with a <#delete column> tag, so that the un-wanted columns/data are removed.
Although this works, it's not ideal. For example, if I want to see how the columns fit in the page width (in print preview), then I need to hide the columns. Then I have to remember to un-hide them again, so other developers can see/understand my handy work.
It would be much more straight forward if I could filter the fields before they're output to the document.
I realised there's an alternate way around this problem. I broke up the data into two sets of data - <#table_name.*> and <#table_name_attachments.*>.
The fixed set of fields are in the first table and the variable set of fields is in the second table (all the "Attachment*" fields). When the report is run, I place them next to each other (in the same order) in the same worksheet. This means I have two table ranges - "_table_name_" and "_table_name_attachments_" on the one sheet.
Now I'm able to run my print preview without hiding/re-showing the columns-to-be-deleted. I've also eliminated human error - it was all to easy to accidentally set the wrong number of padded/delete columns.
being a newbie to SSRS, I am trying to figure out the following:
say for instance I have a dataset which does a :
SELECT [cols...] from [some view]
I want to be able to further filter this based on parameters given from an ASP.NET site (I am using the AJAX control toolkit for the report viewer). There could be x amount of parameters and potentially can be filtered on 1 or more columns.
First question is, how would I filter the dataset and pass along the parameters along with which field the filter should apply to? I may have [col1] and I want to filter it with x values.
Second question Is, I want to be able to group the results per page based upon a column. So for each grouped result set, I want them to be displayed per page (per group per page).
Then on the headers of the page, I want it to display what the page grouping is. How would I do this?
In terms of what have I tried - nothing as I DO NOT KNOW HOW, it is why I am asking the question here to see what the experts (you) can suggest and guide me.
thank you!
To do this you can create Parameters in SSRS, they do not need to be in your query or anything. Then, go to your tablix and click either ROW or column depending on the filter type and set it show/hide visibility. For example I have a report that has personal information, so i have true/false parameter that hides/shows those columns, similar I have one that hides/shows any row with a -1 for the total paid.
I curled the sample bank data to elasticsearch. On Kibana I see under the Indices bank and on visualize all the fields (age, balance, etc.) on the left but cannot see charts on the right. It always says No Results Found. On the left I selected Y-Axis - Agg as Average and Field as Balance and X-Axis Agg as Terms and Field as Age, all from drop downs. Then I used the APPLY action. No error is displayed, but neither is a chart.
What am I doing wrong here? Using curl I can check the elastic search and see that the data exists. No matter what I do, I do not see any logging in the elasticsearch logs under /usr/share/elasticsearch/logs.
I don't know if you figured this out or not but I had the exact same problem. I was using a mapping type that did not contain the field I was trying to visualize. Make sure the _type field contains a mapping with the correct indexed fields.
When setting up my initial index patterns, I glossed over specifying the 'Default timestamp' field, and the UI didn't insist on having one. The fix was to delete my index patterns and re-create them them, with the specified timestamp field. Everything snapped into place.
I am working with SSRS and having trouble creating a graphical chart to display data. My dataset is very simple, as it only has one column which is a datetime field (known as CreatedOn).
My SQL query is:
SELECT [CreatedOn]
FROM [Incident]
WHERE CreatedOn > DATEADD(m,-11, DATEADD(
month, DATEDIFF(month, 0,CURRENT_TIMESTAMP), 0))
My goal is to have the query results show up in a chart and have them grouped by year and by month, as well as sorted. For example, I should be able to see that 40 incidents were created in February 2005. Those 40 incidents would be represented graphically as a single bar (it is a bar graph) with the number 40 on top (or somewhere).
My issue is that either my chart shows up completely blank, or the report fails to run. When it fails to run, the error I receive is:
the value expression for field 'CreatedOn' contains an error: conversion from string 'CreatedOn' to type 'Date' is not valid.
I have tried using SSRS expressions on the CreatedOn field, such as CDate() and FormateDateTime(). That does get the report to run, but the chart shows up blank. I would not think that the field should have to be formatted or converted in SSRS since it is already a datetime field at the database level. I have tried about 10 different combinations of groupings and sorts, but my chart always shows up blank. I have even used CDate() and so forth in the grouping and sort expressions which are a part of the chart.
How can I get my bar chart to work (a.k.a. show datetimes, grouped and sorted)?
I am going to go off the assumption:
You have more data being pulled than just the ‘CreatedOn’ field.
a. Because if you don't it will be difficult to create a chart as you have nothing else to compare it too.
If this is an accurate assumption here is an example of how to create a chart and have the data formatted by dates.
When designing a chart you need to take a lot into consideration. First I like to ensure that I have as little null data as possible. This causes charts to act wonky (at least from my experience).
Second is how the data will be represented.
We have several things to consider when we think about charts and groups.
There are Category groups, Series groups, and then the Values.
According to Microsoft: charts have a direct similarity to Matrix’s. They act the same way:
The Column groups of a Matrix are similar to that of the Category Groups in a chart.
The Row groups of a Matrix are similar to that of the Series Groups in a chart.
The Data area of a Matrix is similar to that of the Values Groups in a chart.
Setting Up the chart:
If we want the number of Incidents sorted by Year and then Month it might look something like this:
Year by Category (You can use your group by Expression here) =Year(Fields!CreatedOn.Value)
Incidents will be in the Values section =Count(Fields!Incidents.Value)
Once you have that working and showing data I would then start adding in the data for the month. Again thinking about how you want to have the data displayed. You could then add a second Grouping on the category under the other one with your month function.
Another Useful link on charts (I know this is a little older but it’s details are still relevant)
*Side Note: When dealing with dates it can be a little tricky especially if SSRS is not recognizing something as a date. Here is a useful link that I have used when dealing with dates.
Dates
I hope this helped!
The issue was that the CreatedOn date field was not being defined correctly.
My data source was defined using an expression which depended upon parameters, using the expression: ="Data Source=sql01;" & "Initial Catalog=" & Parameters!ParentID.Value. Because the dataset must be defined at the runtime of the report, I was not able to refresh fields in the dataset using the "Refresh Fields" button.
I thought that I could work around this by defining the CreatedOn field manually. On the first attempt, SSRS threw an error at runtime claiming that I was trying to convert a string to a date, which basically meant that it perceived the CreatedOn field as defined within the dataset as a string. So I then tried to use the CDate() expression in the definition of the field, but that left me with a completely blank graph.
The solution was to:
Delete all of the fields I already had defined within the dataset
Temporarily hard-code my data source to use a database (Data Source=sql01;Initial Catalog=MyDatabaseName)
Click the "Refresh Fields" button within the dataset
Go back at set the data source to the original data source I wanted which depended upon parameters.
I am using custom variables to track order ids. In order to aggregate analytics data into out data warehouse, I want to select a number of metrics with the custom variable as a dimension. However, if I do so, I will not get the entries where the variable is not set (E.g. sessions that didn't result in a sale). I need to get these as well.
Can I write a filter or segment that selects only the entries that doesn't have a particular custom variable? I have tried:
segment=dynamic::ga:customVarValue1==
But that doesn't seem to work (It gives no results back).
Basically I'm looking for the equivalent to where ga:customVarValue1 is null in sql.
In short, it's not possible to get the nullset data, as explained by a Google rep:
For some dimensions, GA uses the default value of (not set).
Custom Variable do not have a default value, so if a hit does not have a
custom variable associated with it, all the other dimensions in the query
are not added to the reports.
The original answer is a little confusing, but when you read between the lines it suggests that they throw out these "empty" values when they run their aggregates.
The "correct" approach, as he explains, is to set a default value for any row you want reported:
If you need to see the (not set) value, you could try sending a default
value for custom variables.
For example if you use visitor level custom vars to track member vs
non-member, you should always set non-member as a default for everybody;
then modify to member once they register.
Details are here: http://groups.google.com/group/google-analytics-data-export-api/browse_thread/thread/cd078ddb26ca18d5?pli=1
I've just had some success solving this by using a Regex to capture users or sessions where the custom dimension has no value. In my case I want to separate logged in and logged out users.
The Regex .+ will capture any non-empty value, so can be used to get the job done.
The filters for my Returning users segment is matches regex: .+ like this:
For the Customer prospects I used: does not match regex: .+ like this:
It's early days, but this appears to be working: