Maybe a dumb question, but I just started, so thanks for any help.
I created 99% confidence interval for a proportion, but I'm not sure if it is correct, how can I make sure, (when we're calculating confidence interval for mean, we're using t-score, and we can test the results by using t.test function and degrees of freedom)
Is there any similar function to do the same thing for z, proportions? or I can do the same thing by using t.test?
There are a number of functions in R for computing confidence intervals for proportions. Personally, I like to use the binomCI function from the MKinfer package:
install.packages("MKinfer")
library(MKinfer)
x <- 50 # The number of "successes" in your data
n <- 100 # The number of observations in your data
binomCI(x, n, conf.level = 0.99, method = "wald")
Note however that the so-called Wald interval (which is presented in most introductory texts on statistics) that you probably computed usually is a poor choice. See this link for some other alternatives available through binomCI.
Related
I'm wondering how to use replication weights and the confint implementation of the survey package to construct bootstrap confidence intervals/standard errors.
Looking at the survey package's implementation of confint, it seems as though it's simply taking the standard error of the theta list generated after replication, and multiplying it by the statistic corresponding to a given alpha range.
But that doesn't really correspond with any bootstrap implementation I'm aware of. Typically, you'd instead be using a percentile of the distribution of theta to get the sample mean's distribution's confidence interval. The T and BCa intervals are another matter.
Here is my R Code. I have not used the provided weights, instead letting repweights weights be generated as "sample with replacement" weights with equal probability.
data(api)
d <- apiclus1 %>% select(fpc, dnum,api99)
dclus1<-svydesign(id=~dnum, data=d, fpc=~fpc)
rclus1<-as.svrepdesign(dclus1,type="bootstrap", replicates=100)
To test the confidence intervals, we can use:
test_mean <- svymean(~api99, rclus1)
confint(test_mean, df=degf(rclus1))
confint(test_mean, df=degf(rclus1)) - mean(d$api99)
Which results:
2.5 % 97.5 %
api99 554.2971 659.6592
2.5 % 97.5 %
api99 -52.68107 52.68107
So clearly the interval is symmetric, which defeats some of the purpose of using the bootstrap.
So let's try this:
test_bs <- withReplicates(rclus1, function(w, data) weighted.mean(data$api99, w), return.replicates=T)
This will bootstrap the replicates, where the weights are the repweights (which I assume are with replacement weights). Here are the intervals using BCa intervals on the replications:
bca(test$replicates) - mean(d$api99)
-43.2878 49.1148
Clearly not symmetric.
Using percentile intervals:
c(quantile(test$replicates, 0.025),quantile(test$replicates, 0.975)) - mean(d$api99)
2.5% 97.5%
-45.50944 48.06085
Valliant implements percentile intervals this way, which should be equivalent to my percentile intervals:
smho.boot.a <- as.svrepdesign(design = smho.dsgn,
type = "subbootstrap",
replicates = 500)
# total & CI for EOYCNT based on RW bootstrap
a1 <- svytotal( ̃EOYCNT, design = smho.boot.a,
na.rm=TRUE,
return.replicates = TRUE)
# Compute CI based on bootstrap percentile method.
ta1 <- quantile(a1$replicates, c(0.025, 0.975))
I'm looking for clarifications on
a. How to construct bootstrap CIs with survey package for statistics of interest
b. If my withReplication implementation of the bootstrap is correct
The percentile stuff doesn't actually work with multistage survey data (or, at least, it isn't known to work (or, at least, it isn't known to me to work)). Survey bootstraps just estimate the variance. You don't get the higher order accuracy for smooth functions that way, but you do get asymptotically the right variance. To get asymmetric intervals you need to bootstrap some suitable function of the parameter, as svyciprop and svyquantile (and in a sense svyglm) do.
If you assume simple random sampling with replacement of clusters then you could make the percentile bootstrap and extensions work, but that's not a common structure for real-world surveys (and it doesn't really need the survey package)
I'm sure the maintainer of the package would be happy to implement a bootstrap that gave better asymmetric intervals and worked for multistage samples, if someone pointed out suitable references to him.
I am trying to understand bootstrapping in R using the Boot package. I am trying to do a simple spearman rank correlation. I have some code based on a tutorial I found online but am having some issues interpreting the output. The code is below:
*Note: these data are just random numbers I used to help me learn how to run the boot function. They do not represent anything.
test_a=data.frame(v1 = c(1,5,8,3,2,9,5,10,3,5), v2 = c(3,4,7,2,1,10,3,8,8,2))
attach(test_a)
cor.test(v1, v2, method = "spearman")
function_2 = function(test_a, i) {
d2 = test_a[i,]
return(cor(d2$v1, d2$v2, method="spearman"))
}
set.seed(1)
test_boot = boot(test_a, function_2, R=1000)
test_boot
I get the following output:
boot(data = test_a, statistic = function_2, R = 1000)
Bootstrap Statistics :
original bias std. error
t1* 0.6397639 -0.04253283 0.2547429
Which all makes sense to me. But I guess my confusion is with the boot.ci function
ci = boot.ci(test_boot, conf=0.95)
I get the following output:
> ci
BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
Based on 1000 bootstrap replicates
CALL :
boot.ci(boot.out = test_boot, conf = 0.95)
Intervals :
Level Normal Basic
95% ( 0.1830, 1.1816 ) ( 0.3173, 1.2987 )
Level Percentile BCa
95% (-0.0192, 0.9622 ) (-0.1440, 0.9497 )
Calculations and Intervals on Original Scale
And this is where I am a bit lost. I can't really find a source that explains in layman's terms in the context of a correlation coefficient, because obviously you cannot have a correlation > 1.0, yet this spits out a confidence interval (at least with two methods) that goes above 1. The sources that discuss these different confidence intervals frankly have been a bit confusing. Is there any one of these that is better for certain parameters than others? It is also possible I am completely misinterpreting what I am doing as well.
I also include the results of plot(test_boot) for your reference.
The eventual goal (with actual data) once I am more confident in running and interpreting results of bootstrapping would be to run tests for trends with time (Mann-Kendall test for trends and Thiel-Sen Slope Estimator, I cannot use parametric statistics with my data :/) and compare my observed dataset with bootstrapped samples.
Any help would really be appreciated. Thank you in advance!
The two top intervals are normal theory intervals. They use the bootstrap to calculate the standard error and then make symmetric intervals that may or may not respect the bounds of the statistic. The bottom two intervals are percentile intervals (the first is a raw percentile interval and the second is a bias-corrected, accelerated interval). These identify particular values of the bootstrap statistics that define the CI. As such, they will always respect the theoretical bounds of the statistic being bootstrapped.
Is it possible to/how can I generate a beta-binomial distribution from an existing vector?
My ultimate goal is to generate a beta-binomial distribution from the below data and then obtain the 95% confidence interval for this distribution.
My data are body condition scores recorded by a veterinarian. The values of body condition range from 0-5 in increments of 0.5. It has been suggested to me here that my data follow a beta-binomial distribution, discrete values with a restricted range.
set1 <- as.data.frame(c(3,3,2.5,2.5,4.5,3,2,4,3,3.5,3.5,2.5,3,3,3.5,3,3,4,3.5,3.5,4,3.5,3.5,4,3.5))
colnames(set1) <- "numbers"
I see that there are multiple functions which appear to be able to do this, betabinomial() in VGAM and rbetabinom() in emdbook, but my stats and coding knowledge is not yet sufficient to be able to understand and implement the instructions provided on the function help pages, at least not in a way that has been helpful for my intended purpose yet.
We can look at the distribution of your variables, y-axis is the probability:
x1 = set1$numbers*2
h = hist(x1,breaks=seq(0,10))
bp = barplot(h$counts/length(x1),names.arg=(h$mids+0.5)/2,ylim=c(0,0.35))
You can try to fit it, but you have too little data points to estimate the 3 parameters need for a beta binomial. Hence I fix the probability so that the mean is the mean of your scores, and looking at the distribution above it seems ok:
library(bbmle)
library(emdbook)
library(MASS)
mtmp <- function(prob,size,theta) {
-sum(dbetabinom(x1,prob,size,theta,log=TRUE))
}
m0 <- mle2(mtmp,start=list(theta=100),
data=list(size=10,prob=mean(x1)/10),control=list(maxit=1000))
THETA=coef(m0)[1]
We can also use a normal distribution:
normal_fit = fitdistr(x1,"normal")
MEAN=normal_fit$estimate[1]
SD=normal_fit$estimate[2]
Plot both of them:
lines(bp[,1],dbetabinom(1:10,size=10,prob=mean(x1)/10,theta=THETA),
col="blue",lwd=2)
lines(bp[,1],dnorm(1:10,MEAN,SD),col="orange",lwd=2)
legend("topleft",c("normal","betabinomial"),fill=c("orange","blue"))
I think you are actually ok with using a normal estimation and in this case it will be:
normal_fit$estimate
mean sd
6.560000 1.134196
I should like to calculate - by bootstrapping Krippendorff's Alpha outcomes - a 95% confidence interval for Krippendorff's Alpha coefficient of Raters Reliability using R package irr.
Let's use "C data from Krippendorff" in the package irr and the R script for calculating Krippendorff's Alpha once:
# the "C" data from Krippendorff
#rater per row; rated subject per column; NAs allowed
library(irr)
nmm<-matrix(c(1,1,NA,1,2,2,3,2,3,3,3,3,3,3,3,3,2,2,2,2,1,2,3,4,4,4,4,4,
1,1,2,1,2,2,2,2,NA,5,5,5,NA,NA,1,1,NA,NA,3,NA),nrow=4)
kripp.alpha(nmm,"ordinal")
You can use the boot function from the boot package to bootstrap values. Here I'll bootstrap the set of subjects but keep the raters fixed:
library(boot)
library(irr)
ka <- function(data, indices) kripp.alpha(nmm[,indices], "ordinal")$value
b <- boot(seq(ncol(nmm)), ka, 1000)
Now you can use the boot.ci function to compute a 95% confidence interval for the bootstrapped value; I'll use the percentile confidence interval, but others are available (check out ?boot.ci):
boot.ci(b, type="perc")
# BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
# Based on 1000 bootstrap replicates
#
# CALL :
# boot.ci(boot.out = b, type = "perc")
#
# Intervals :
# Level Percentile
# 95% ( 0.4297, 1.0000 )
# Calculations and Intervals on Original Scale
Unfortunately, the bootstrap solution given by josliber doesn't do what you think it does. The problem is that boot() expects data in an nXm matrix while kripp.alpha() expects data in an mXn matrix. The solution given will run, as shown, but the resampling being done is not by subjects, but by raters, so with 4 raters in the example data set we have a small number of possible samples, with the possibility that the resampled set will come from a single rater (hence the conf interval includes 1.0).
One solution is to keep your data in the nXm form that boot uses, and add a matrix transpose before giving it to kripp.alpha().
alpha.boot <- function(data,x) {
d <- t(data[x,])
kripp.alpha(d,method="nominal")$value
}
Probably too late to be helpful for Paul, but for future reference note that none of the proposed methods is consistent with the bootstrap algorithm described by Klaus Krippendorff (http://web.asc.upenn.edu/usr/krippendorff/boot.c-Alpha.pdf).
The repeated samples do neither draw raters nor units, but "hypothetical reliability data from the matrix of observed coincidences among pairs of values assigned to units in independent replications" (Krippendorff 2016). Thus, conventional bootstrap implementations will not give the answer intended by Krippendorff.
Best
Daniel
I'm trying to do something like what has mentioned in "VSUMM" article.
There are 5 approaches which i want to compare their results with mine. i do compare all of them in a table by calculating the CUS(a) i.e mean accuracy rate and CUS(e) i.e mean error rate for each of them.(it's a video summarization problem and each approaches has got their summeries for 50 videos. this summeries has been compared with five user summeries and then the CUS(a) and CUS(e) are calculated by these reults)
now i should calculate the confidence intervals(with the confidence of 98%) of each of those approaches with mine
i surf the web and find long articles about confidence intervals but i did not really understand any thing!?
what should i do?
finally find the answer:
For calculating the confidence intervals between your approach and other approaches you should have the results of your algorithm and those approaches for sure. then you can calculate mean of these data and their Standard deviation. then you could use the confidence intervals formula:
(amean-bmean) +/- z(1-a/2)*sqrt((stda^2/n1)+(stdb^2/n2)
which stda is Standard deviation of a
n1 the number of data in first group
amean=the mean value of the first group, i.e the mean of results of first approach