I am trying to plot and compare two sets of decimal numbers, between 0 and 1 using the R package, ggplot2. When I plotted using geom="density" in qplot, I noticed that the density curve goes past 1.0. I would like to have a density plot for the data that does not exceed the value range of the set, ie, all the area stays between 0 and 1.
Is it possible to plot the density between the values 0 and 1, without going past 1 or 0? If so, how would I accomplish this? I need the area of the two plots to be equal between 0 and 1, the range of the data.
Here is the code I used to generate the plots.
Right: qplot(precision,data = compare, fill=factor(dataset),binwidth = .05,geom="density", alpha=I(0.5))+ xlim(-1,2)
Left:qplot(precision,data = compare, fill=factor(dataset),binwidth = .05,geom="density", alpha=I(0.5))
You might consider using a different tool to estimate the density (the built in density functions do not consider bounds), then use ggplot2 to plot the estimated densities. The logspline package has tools that will estimate densities (useing a different algorythm than density does) and you can tell the functions that your density is bounded between 0 and 1 and it will take that into consideration in estimating the densities. Then use ggplot2 (or other code) to compare the estimated densities.
Related
uppose that i have a poisson distribution with mean of 6 i would like to plot a probability mass function which includes an overlay of the approximating normal density.
This is what i have tried
plot( dpois( x=0:10, lambda=6 ))
this produces
which is wrong since it doesnt contain an overlay of approxiamating noral density
How do i go about this?
Something like what you seem to be asking for (I'm outlining the commands and the basic ideas, but checking the help on the functions and trying should fill in the remaining details):
taking a wider range of x-values (out to at least 13 or so) and use xlim to extend the plot slightly into the negatives (maybe to -1.5) and
plotting the pmf of the Poisson with solid dots (similar to your command but with pch=16 as an argument to plot) with a suitable color, then
call points with the same x and y arguments as above and have type=h and lty=3 to get vertical dotted lines (to give a clear impression of the relative heights, somewhat akin to the appearance of a Cleveland dot-chart); I'd use the same colour as the dots or a slightly lighter/greyer version of the dot-colour
use curve to draw the normal curve with the same mean and standard deviation as the Poisson with mean 6 (see details at the Wikipedia page for the Poisson which gives the mean and variance), but across the wider range we plotted; I'd use a slightly contrasting colour for that.
I'd draw a light x-axis in (e.g. using abline with the h argument)
Putting all those suggestions together:
(However, while it's what you're asking for it's not strictly a suitable way to compare discrete and continuous variables since density and pmf are not on the same scale, since density is not probability -- the "right" comparison between a Poisson and an approximating normal would be on the scale of the cdfs so you compare like with like -- they'd both be on the scale of probabilities then)
I have an algorithm that uses an x,y plot of sorted y data to produce an ogive.
I then derive the area under the curve to derive %'s.
I'd like to do something similar using kernel density estimation. I like how the upper/lower bounds are smoothed out using kernel densities (i.e. the min and max will extend slightly beyond my hard coded input).
Either way... I was wondering if there is a way to treat an ogive as a type of cumulative distribution function and/or use kernel density estimation to derive a cumulative distribution function given y data?
I apologize if this is a confusing question. I know there is a way to derive a cumulative frequency graph (i.e. ogive). However, I can't determine how to derive a % given this cumulative frequency graph.
What I don't want is an ecdf. I know how to do that, and I am not quite trying to capture an ecdf. But, rather integration of an ogive given two intervals.
I'm not exactly sure what you have in mind, but here's a way to calculate the area under the curve for a kernel density estimate (or more generally for any case where you have the y values at equally spaced x-values (though you can, of course, generalize to variable x intervals as well)):
library(zoo)
# Kernel density estimate
# Set n to higher value to get a finer grid
set.seed(67839)
dens = density(c(rnorm(500,5,2),rnorm(200,20,3)), n=2^5)
# How to extract the x and y values of the density estimate
#dens$y
#dens$x
# x interval
dx = median(diff(dens$x))
# mean height for each pair of y values
h = rollmean(dens$y, 2)
# Area under curve
sum(h*dx) # 1.000943
# Cumulative area
# cumsum(h*dx)
# Plot density, showing points at which density is calculated
plot(dens)
abline(v=dens$x, col="#FF000060", lty="11")
# Plot cumulative area under curve, showing mid-point of each x-interval
plot(dens$x[-length(dens$x)] + 0.5*dx, cumsum(h*dx), type="l")
abline(v=dens$x[-length(dens$x)] + 0.5*dx, col="#FF000060", lty="11")
UPDATE to include ecdf function
To address your comments, look at the two plots below. The first is the empirical cumulative distribution function (ECDF) of the mixture of normal distributions that I used above. Note that the plot of this data looks the same below as it does above. The second is a plot of the ECDF of a plain vanilla normal distribution, mean=0, sd=1.
set.seed(67839)
x = c(rnorm(500,5,2),rnorm(200,20,3))
plot(ecdf(x), do.points=FALSE)
plot(ecdf(rnorm(1000)))
I have used sm.density.compare to plot 3 density functions for data with values between -90 and +10. The Y axis is labeled "density" and has the range 0 - 1.0 as for proportions or probability.
I then plot 4 density functions for data with values between 0 and 1.0. I get a useful plot and the Y axis still reads "density" but the values are apparently counts and range between 0 and 12 or so.
The function sm.options does not seem to offer control of which you get. I'd like both to be probability or proportions.
I'm new to R but have a substantial history with other software.
I have two data sets that I am comparing using a ked2d contour plot on a log10 scale,
Here I will use an example of the following data sets,
b<-log10(rgamma(1000,6,3))
a<-log10((rweibull(1000,8,2)))
density<-kde2d(a,b,n=100)
filled.contour(density,color.palette=colorRampPalette(c('white','blue','yellow','red','darkred')))
This produces the following plot,
Now my question is what does the z values on the legend actually mean? I know it represents where most the data lies but 0-15 confuses me. I thought it could be a percentage but without the log10 scale I have values ranging from 0-1? And I have also produced plots with scales 1-1.2, 1-2 using my real data.
The colors represent the the values of the estimated density function ranging from 0 to 15 apparently. Just like with your other question about the odd looking linear regression I can relate to your confusion.
You just have to understand that a density's integral over the full domain has to be 1, so you can use it to calculate the probability of an observation falling into a specific region.
I'm trying to plot an histogram of the Cauchy distribution in R using the following code:
X = rcauchy(10^5)
hist(X)
and no matter what options I try in the hist() function, I can never see more than two bars on my histogram (basically one for negative values and one for positive values).
It works fine, however, when I use the normal distribution (or others).
This results from the properties of the distribution.
Most values are relatively close to zero, but very large absolute values are much more probable than for the normal distribution. There are about 1 % values with an absolute value greater than 50, and 0.1 % greater than 500.
Try plotting only part of the values:
hist(X[abs(X)<1])
hist(X[abs(X)<5])
hist(X[abs(X)<50])
hist(X)
You can also look at the cumulative distribution function:
plot(ecdf(X))
And check the boxplot:
boxplot(X)