I have a job runner app that runs jobs sporadically throughout the day, and I'm reporting metrics for them. They don't run continuously, so AI interpolates between the gaps and the graph looks weird. I would like the graph to either not interpolate or just start/end at zero, so for now I am going to try reporting non-pre-aggregated zero values before and after the jobs, so at least this interpolation will be at the bottom of the graph and be easier to read.
Is there a correct way to indicate the end of a segment of values and the start of another, to avoid this?
Here is a good example of what I'm talking about:
This is a single job runner from our staging env. which ran four jobs in this timeframe, and you can see where the dotted lines are between jobs. This is what I'm trying to avoid. This is exacerbated in a real environment with multiple runners or if you need to compare, e.g.:
What I would like to do is somehow avoid the dotted lines and just go to/from zero (have all the dotted lines at the bottom so they don't interfere with other series). Maybe it's not possible.
I tried sending zero values (both direct and pre-aggregated) at the beginning and end of the jobs and using Operations hoping they would "bookend" the jobs, but neither seemed to do anything.
Metrics Troubleshooting Guide explains the dashes on the line chart type. Generally, the line chart is a great option for showing the continuity and trends. It is a poor option for showing sparse data points. It is so because this type of the chart is rendered by drawing the line segments between known values but without visualizing each value. For better illustration, think about how the line would look when there is only one known value on the entire chart. The two options could be: 1 - show a horizontal line (that’s what we do, but in addition, we add the dashed visualization so that you know the other values are missing) or 2 - not show the line and switch visualization to display just a dot (we let you do it manually by picking scattered plot chart option).
Related
I have an application that dumps statistics once every minute into ElasticSearch. Using Kibana, I've created a lens to visualize the values over time. All good.
Occasionally, the application skips a dump (which is a separate issue I'll be working on), so I may have occasional missing values. In the Lens, the Line chart breaks the line when a value is missing for a certain time (see screenshot).
How can I interpolate between the available values and show a continuous line instead?
EDIT
To make it clear, I'm asking how to fix the chart in the Lens. The issue of skipped dumps is a separate one and needs to be addressed separately. I may still have some missing values (unlikely but not impossible), and I would like my Lens to show a continuous chart anyway.
UPDATE
While I found the "Missing values" setting in "Visual Options" and I set it to "Linear", the interpolation is rendered with a dashed line. How can I make it the same as the main line?
I plot a lot of graphs in gnuplot. These graphs are based onn sensor readings from around the solar power system.
Each graph has needed to be updated by typing something like
load "solar
where solar is a gnuplot program that performs the plot showing the condition of the 24 V (500Ah) battery bank and leaves it on the screen so I can do a regional screen capture for storage.
In this particular case, the numbers come in at 2-minute intervals. Unless the inverter is turned on, in which case they come in at 20 second intervals. So I end up typing that command a lot just to see how clean the signal is.
So the question came up as to whether I need to continue to tell it to load the program every time I want to see updates.
How can I actually make it automatically live?
Turns out it is as simple as can be to have it run live.
This article: Running Gnuplot as a live graph, with automatic updates
explains the process nicely.
Turns out that all you need to do is add two lines of code after the plot command. In my case, I want it to update the graph every 15 seconds, so the last two lines of the program are simply
pause 15
reread
Here is an excerpt from the article:
Gnuplot has some useful commands we can use:
pause
reread
These are fairly self-explanatory, so let’s make a Gnuplot file, liveplot.gnu, that refreshes itself once every second.
set xrange [0:20]
set yrange [0:400]
plot "plot.dat" using 1:2 with lines
pause 1
reread
We set the bounds of our graph, then plot the data from the file. using 1:2 means plot columns 1 and 2 as x and y, respectively. with lines means that the points are joined together rather than plotted separately. We pause for 1 second and then reread, meaning that the command file is re-executed.
It turned out to be so simple that I am going to add those two lines to all my graphs that I monitor on the xterminals of the individual Rpi3s that monitor the sensors.
Collected together on the big screen it gives me a great overview of the entire system, including temperatures and voltages and such.
The best part is that there is no need to specify the X range to be fixed. It is much better to let it recalculate every time it rereads.
Results: A true live graph, monitoring conditions of the sensors from which it is receiving near-real-time data.
(You can see how hot the panels get even on a relatively cool day, and how the MPPT charge controller works to maintain the voltage)
https://www.SDsolarBlog.com/montage
I'm working with the ciplot graphing module for Stata and am encountering a problem with the alignment of bars when I use the by() option. Here's a trivial example demonstrating the issue:
webuse citytemp, clear
ciplot heatdd cooldd, by(region) horizontal recast(conn)
So, the graph shows means and confidence intervals for two variables across categories of the region variable. The bars for the different variables do not align horizontally, though. For each region, the point and bar for heatdd is one line above, and the point and bar for cooldd is one line below, the category label. I would like these to be on the same line, but I can't figure out how to achieve it.
I'm open to solutions that do not involve ciplot, but I have found it to be useful for the specific task I'm working on.
This is my program (in Stata terms, downloadable via ssc install ciplot) so I can speak confidently. (On Statalist, it's expected that you explain the exact provenance of user-written programs; that would be good practice here too.)
It's not a bug; it's a feature (supposedly).
The offsets are entirely deliberate, to avoid messes when two or more intervals would just overlap and occlude each other, which is entirely likely when groups or comparable variables have similar values, which in turn is common when you do this. Even in your example, intervals for heating and cooling degree-days for the South would overlap otherwise, so the graph makes the point for me.
I can see that it's not what you want, but
There is no option in ciplot to remove the offset. I can see a case for one, but
My advice is now to use statsby to get a reduced dataset containing the confidence interval information, and then the graphics are typically a couple of command lines and you get to choose what you want. This approach is documented in a paper easily accessible from the Stata Journal.
You are always welcome to clone the program and modify the code using a different program name, with notional mention of the original.
I use Jenkins and its Plot Plugin to generate statistics.
Example:
For every Build I plot the line of codes. Lets say I have like 500.000 lines and it changes about +-100 lines per build.
Since the plot y-axis always starts with 0 the whole plot is meaningless. It looks like 1 straight line.
Same goes for other metrics, if the value is too high.
Question: is there any configuration to set minimum y-axes to the minimum value?
Unfortunately, I think the answer to your question is currently 'no'.
First of all, the documentation does not give any hints towards such an option.
Secondly, Given that the Jenkins Plot Plugin uses JFreeChart for the plotting, the setting of a range for the Y-axis should be done using the 'getRangeAxis' function. A search in the source-code does not reveal a call to this function.
Lastly, you do not seem to be the only one that has this problem. Issue JENKINS-2841 asks for the same functionality, but appears not be solved yet.
I'm getting familiar with Graphviz and wonder if it's doable to generate a diagram/graph like the one below (not sure what you call it). If not, does anyone know what's a good open source framework that does it? (pref, C++, Java or Python).
According to Many Eyes, this is a bubble chart. They say:
It is especially useful for data sets with dozens to hundreds of values, or with values that differ by several orders of magnitude.
...
To see the exact value of a circle on the chart, move your mouse over it. If you are charting more than one dimension, use the menu to choose which dimension to show. If your data set has multiple numeric columns, you can choose which column to base the circle sizes on by using the menu at the bottom of the chart.
Thus, any presentation with a lot of bubbles in it (especially with many small bubbles) would have to be dynamic to respond to the mouse.
My usual practice with bubble charts is to show three or four variables (x, y and another variable through the size of the bubble, and perhaps another variable with the color or shading of the bubble). With animation, you can show development over time too - see GapMinder. FlowingData provides a good example with a tutorial on how to make static bubble charts in R.
In the example shown in the question, though, the bubbles appear to be located somewhat to have similar companies close together. Even then, the exact design criteria are unclear to me. For example, I'd have expected Volkswagen to be closer to General Motors than Pfizer is (if some measure of company similarity is used to place the bubbles), but that isn't so in this diagram.
You could use Graphviz to produce a static version of a bubble chart, but there would be quite a lot of work involved to do so. You would have to preprocess the data to calculate a similarity matrix, obtain edge weights from that matrix, assign colours and sizes to each bubble and then have the preprocessing script write the Graphviz file with all edges hidden and run the Graphviz file through neato to draw it.