I recently came across a question about the statistical inference of an estimator, but I am not sure about how to do the inference part. Let me explain my question first:
Say I obtained a coefficient from a regression, say a_1. And I ran a second regression, and obtain a_2. Two regressions are using different samples. Then, I take the difference between the two estiamtes: D = a_1 - a_2. Now, I need to know whether D is statistically different from 0.
My question is how to do this. Is it the same as the comparison of two means as it stated in this link (http://www.stat.yale.edu/Courses/1997-98/101/meancomp.htm)? From my understanding, the point estimates are mean of the distribution of the coefficient, but I am not sure how to specify the number of observations in the formula shown in the above link.
Also the above step is a parametric method, from my understanding. Should I use bootstrap instead?
Could someone please help with my understanding? or maybe guide me to some good reference to follow?
Best
Related
Which are the best metrics to evaluate the fit of a GBM algorithm in R (metrics, graphs, ratios)? And how interpret them?
I think maybe you are overthinking this one! Take a step back and think about what matters... the error. You have forecasted values and you have observed values. the difference tells you most of what you need to know when comparing across models. Basic measures like MSE, MPE, etc. should do fine. If you are looking to refine within a given model, I would recommend taking a look at the gbm documentation. For example, you can pass your gbm model object to summary(), to get the relative influence of each of your variables. Additionally, you can find a lot of information in the documentation, so if you haven't taken a look, I would recommend doing so! I have posted the link at the bottom.
-Carmine
gbm_documentation
I am running an analysis of variances on a large distance matrix using adonis2 as described here: https://www.rdocumentation.org/packages/vegan/versions/2.4-2/topics/adonis
That method is frequently used in microbiome analysis to calculate beta diversity. That's also what I would like to do, i.e. to find out whether my community composition differs in response to an environmental variable (continuous)
Permanova returns one p value and there is no "official" post hoc test yet. That's where my question comes in:
I've come across publications saying they adjusted their permanova result using FDR/BH method. I cannot wrap my head around this. I'm confident I understand how FDR correction is calculated, I just don't see how that would be done for PERMANOVA, or, even more, how I would code it.
Can anyone help me out here?
Would be clearer if you provide an example of so-called publication. You are right that for each variable, permanova returns 1 p-value. However, if the model includes many variables, you would have 1 p-value for each variable and you need to correct for FDR.
For example in this publication looking at variation in gut microbiome, they wrote:
To calculate the variation explained by each of our collected host
factors, we performed an Adonis test implemented in QIIME. Each host
factor was calculated according to its explanation rate, and P values
were generated based on 1,000 permutations. All P values were then
adjusted using the Benjamini–Hochberg method.
You can also see an example of this in Table S2, I attached a screenshot here:
I have very little programming experience, but I'm working on a statistics project and would like to generate an unequal probability sample where the inclusion probability of a unit is based on its size (PPS).
Basically, I have two datasets:
ds1 lists US states and the parameter I'm trying to estimate
ds2 has the population size of each state.
My questions:
I want to use R to select a random sample from the first dataset using inclusion probabilities based on the population of each state (second dataset).
Also is there any way to use R to calculate these Generalized Unequal Probability Estimator formulas?
Also just a note on the formulas: pi_i is inclusion probability and pi_ij is joint inclusion probability.
There is a package for the same in R - pps and the documentation is here.
Also, there is another package called survey with a bit of documentation here.
I'm not sure of the difference between the two and haven't used them myself. Hope this is what you're looking for.
Yes, that's called weighted sampling. Simply set the weight to the size of the state, strictly you don't even need to normalize them by 1/sum(sizes) although it's always good practice to. There are tons of duplicate posts on SO showing how to do weighted sampling.
The only tiny complication is that you need to do a join() of the datasets ds1, ds2. Show us what code you've tried if it's causing problems. Recommend you use either dplyr or data.table.
Your second question should be asked as a separate question, and is offtopic on SO, or at least won't get a great response - best to ask statistical questions at sister site CrossValidated
I asked the following question over on stackexchange https://stats.stackexchange.com/questions/272657/determining-the-direction-of-a-significant-spearmans-rho-correlation - someone pointed me in the direction of this site as I am using spss, so if anyone had any advice that would be much appreciated.
I have conducted Spearman's Rho tests with two ordinal variables (one with 4 possible answers and the other with 6). I have obtained a statistically significant correlation between the two. My question is, how can I graphically (or some other way) determine which answer of each correlate together - as a scatterplot would not work with my data (since it is not scale).
A fluctuation plot is often a good way to look at the distribution of pairs of categorical variables. There is a custom dialog available for this if you don't want to figure out the GPL code. It is available from the Community site, but if you can't find it, send me an email (jkpeck#gmail.com), and I'll send it to you.
I have two histograms.
int Hist1[10] = {1,4,3,5,2,5,4,6,3,2};
int Hist1[10] = {1,4,3,15,12,15,4,6,3,2};
Hist1's distribution is of type multi-modal;
Hist2's distribution is of type uni-modal with single prominent peak.
My questions are
Is there any way that i could determine the type of distribution programmatically?
How to quantify whether these two histograms are similar/dissimilar?
Thanks
Raj,
I posted a C function in your other question ( automatically compare two series -Dissimilarity test ) that will compute divergence between two sets of similar data. It's actually intended to tell you how closely real data matches predicted data but I suspect you could use it for your purpose.
Basically, the smaller the error, the more similar the two sets are.
These are just guesses, but I would try fitting each distribution as a gaussian distribution and use something like the R-squared value to determine if the distribution is uni-modal or not.
As to the similarity between the two distributions, I would try doing an autocorrelation and using the peak positive value in the autocorrelation as a similarity measure. These ideas are pretty rough, but hopefully they give you some ideas.
For #2, you could calculate their cross-correlation (so long as the buckets themselves can be sorted). That would give you a rough estimation of what "similarity".
Comparison of Histograms (For Use in Cloud Modeling).
(That's an MS .doc file.)
There are a variety of software packages that will "fit" your distributions to known discrete distributions for you - Minitab, STATA, R, etc. A reference to fitting distributions in R is here. I wouldn't advise programming this from scratch.
Regarding distribution comparisons, if neither distribution fits a known distribution (Poisson, Binomial, etc.), then you need to use non-parametric methods described here.