I'm using the plottol function in the tolerance package of R and getting an error / warning after my plot is generated that say "NOTE: A regression through the origin is fitted!"
I've googled it and come up with nothing and I don't know where to start with trying to figure out what it's warning me about.
It in general indicates that a model is fitted whereby the intercept term is forced to be zero (in other words the model has no intercept). The intercept in a model of y ~ x is the expectation of y when x is equal to 0. By setting the intercept to 0 we are explicitly stating that the expectation of y is 0 when x is 0. In other words, the straight line fitted to y ~ x passes through the point (0,0), i.e. the origin.
If you want a less general explanation (more tailored to your specific example) you'll need to provide more details on which of the many functions in tolerance you are using, e.g. via a reproducible example.
Related
For my class we have to create a model to predict the credit balance of each individuals. Based on observations, many results are zero where the lm tries to calculate them.
To overcome this I created a new variable that results in zero if X and Y are true.
CB$Balzero = ifelse(CB$Rating<=230 & CB$Income<90,0,1)
This resulted in getting 90% of the zero results right. The problem is:
How can I place this variable in the lm so it correctly results in zeros when the proposition is true and the calculation when it is false?
Something like: lm=Balzero*(Balance~.)
I think that
y ~ -1 + Balzero:Balance
might work (you haven't given us a reproducible example to try).
-1 tells R to omit the intercept
: specifies an interaction. If both variables are numeric, then A:B includes the product of A and B as a term in the model.
The second term could also be specified as I(Balzero*Balance) (I means "as is", i.e. interpret * in the usual numerical sense, not in its formula-construction context.)
These specifications should fit the model
Y = beta1*Balzero*Balance + eps
where eps is an error term.
If Balzero == 0, the predicted value will be zero. If Balzero==1 the predicted value will be beta1*Balance.
You might want to look into random forest models, which naturally incorporate the kind of qualitative splitting that you're doing by hand in your example.
I am using h2o.deeplearning to train a neural network on a classification task.
What I have
Y ~ x1 + x2... where all x variables are continuous and Y is binary.
What I want
To be able to train a deeplearning object to predict the probability of a given row of being true or false. That is, a predicted(Y) restricted to between 0 and 1.
What I've tried
When Y is inputted as a numeric (i.e. 0 or 1), h2o deeplearning automatically treats it as a regression problem. This is fine, except the final layer of the NN is linear, not tanh, and the predicted values can be greater than 1 or less than 0. I've not been able to find a way to get the final layer to be a tanh.
When Y is inputted as categorical (i.e. TRUE or FALSE), h2o deeplearning automatically treats it as a classification problem. Instead of giving me the desired probability of Y being 1 or 0, it gives me its best guess of what Y is.
Is there a way around this? A trick, tweak or an overlooked parameter? I have noticed in the h2o.deeplearning documentation a 'distribution' parameter, but no further information on what that's for. My best guess is that it is some kind of link function in the same vein as GLM, but I'm not sure.
If you treat the problem as a binary classification problem then you not only get the “prediction” of 0 or 1, but also the p0 and p1 probabilities that add up to 1. These are the probabilies that the predicted value is the negative and positive class, respectively.
Then just use p1 directly.
I am attempting to understand how the predict.loess function is able to compute new predicted values (y_hat) at points x that do not exist in the original data. For example (this is a simple example and I realize loess is obviously not needed for an example of this sort but it illustrates the point):
x <- 1:10
y <- x^2
mdl <- loess(y ~ x)
predict(mdl, 1.5)
[1] 2.25
loess regression works by using polynomials at each x and thus it creates a predicted y_hat at each y. However, because there are no coefficients being stored, the "model" in this case is simply the details of what was used to predict each y_hat, for example, the span or degree. When I do predict(mdl, 1.5), how is predict able to produce a value at this new x? Is it interpolating between two nearest existing x values and their associated y_hat? If so, what are the details behind how it is doing this?
I have read the cloess documentation online but am unable to find where it discusses this.
However, because there are no coefficients being stored, the "model" in this case is simply the details of what was used to predict each y_hat
Maybe you have used print(mdl) command or simply mdl to see what the model mdl contains, but this is not the case. The model is really complicated and stores a big number of parameters.
To have an idea what's inside, you may use unlist(mdl) and see the big list of parameters in it.
This is a part of the manual of the command describing how it really works:
Fitting is done locally. That is, for the fit at point x, the fit is made using points in a neighbourhood of x, weighted by their distance from x (with differences in ‘parametric’ variables being ignored when computing the distance). The size of the neighbourhood is controlled by α (set by span or enp.target). For α < 1, the neighbourhood includes proportion α of the points, and these have tricubic weighting (proportional to (1 - (dist/maxdist)^3)^3). For α > 1, all points are used, with the ‘maximum distance’ assumed to be α^(1/p) times the actual maximum distance for p explanatory variables.
For the default family, fitting is by (weighted) least squares. For
family="symmetric" a few iterations of an M-estimation procedure with
Tukey's biweight are used. Be aware that as the initial value is the
least-squares fit, this need not be a very resistant fit.
What I believe is that it tries to fit a polynomial model in the neighborhood of every point (not just a single polynomial for the whole set). But the neighborhood does not mean only one point before and one point after, if I was implementing such a function I put a big weight on the nearest points to the point x, and lower weights to distal points, and tried to fit a polynomial that fits the highest total weight.
Then if the given x' for which height should be predicted is closest to point x, I tried to use the polynomial fitted on the neighborhoods of the point x - say P(x) - and applied it over x' - say P(x') - and that would be the prediction.
Let me know if you are looking for anything special.
To better understand what is happening in a loess fit try running the loess.demo function from the TeachingDemos package. This lets you interactively click on the plot (even between points) and it then shows the set of points and their weights used in the prediction and the predicted line/curve for that point.
Note also that the default for loess is to do a second smoothing/interpolating on the loess fit, so what you see in the fitted object is probably not the true loess fitting information, but the secondary smoothing.
Found the answer on page 42 of the manual:
In this algorithm a set of points typically small in number is selected for direct
computation using the loess fitting method and a surface is evaluated using an interpolation
method that is based on blending functions. The space of the factors is divided into
rectangular cells using an algorithm based on k-d trees. The loess fit is evaluated at
the cell vertices and then blending functions do the interpolation. The output data
structure stores the k-d trees and the fits at the vertices. This information
is used by predict() to carry out the interpolation.
I geuss that for predict at x, predict.loess make a regression with some points near x, and calculate the y-value at x.
Visit https://stats.stackexchange.com/questions/223469/how-does-a-loess-model-do-its-prediction
I am currently trying to fit an adaBoost model in R using the gbm.fit model. I have tried everything I could but in the end my model keeps giving me prediction values outside of [0,1]. I understand that type = "response" only works for bernoulli but I keep getting values just outside of 0,1. Any thoughts? Thanks!
GBMODEL <- gbm.fit(
x=training.set,
y=training.responses,
distribution="adaboost",
n.trees=5000,
interaction.depth=1,
shrinkage=0.005,
train.fraction=1,
)
predictionvalues = predict(GBMODEL,
newdata=test.predictors,
n.trees=5000,
type="response")
it is correct to obtain y range outside [0,1] by gbm package choosing "adaboost" as your loss function.
After training, adaboost predicts category by the sign of output.
For instance, for binary class problem, y{-1,1}, the class lable will be signed to the sign of output y. So if you got y=0.9 or y=1.9 will give you the same result-observation belongs to y=1 class. However, y=1.9 simply suggests a more confident conclusion than y=0.9. (if you want to know why, I would suggest you to read margin-based explanation of adaboost, you will find very similar result with SVM).
Hope this can help you.
This may not be completely accurate mathematically, but I just did pnorm( predicted values) and you get values from 0 to 1, because the adaboost predicted values appear to be scaled on a Normal(0,1).
Using predict() one can obtain the predicted value of the dependent variable (y) for a certain value of the independent variable (x) for a given model. Is there any function that predicts x for a given y?
For example:
kalythos <- data.frame(x = c(20,35,45,55,70),
n = rep(50,5), y = c(6,17,26,37,44))
kalythos$Ymat <- cbind(kalythos$y, kalythos$n - kalythos$y)
model <- glm(Ymat ~ x, family = binomial, data = kalythos)
If we want to know the predicted value of the model for x=50:
predict(model, data.frame(x=50), type = "response")
I want to know which x makes y=30, for example.
Saw the previous answer is deleted. In your case, given n=50 and the model is binomial, you would calculate x given y using:
f <- function (y,m) {
(logit(y/50) - coef(m)[["(Intercept)"]]) / coef(m)[["x"]]
}
> f(30,model)
[1] 48.59833
But when doing so, you better consult a statistician to show you how to calculate the inverse prediction interval. And please, take VitoshKa's considerations into account.
Came across this old thread but thought I would add some other info. Package MASS has function dose.p for logit/probit models. SE is via delta method.
> dose.p(model,p=.6)
Dose SE
p = 0.6: 48.59833 1.944772
Fitting the inverse model (x~y) would not makes sense here because, as #VitoshKa says, we assume x is fixed and y (the 0/1 response) is random. Besides, if the data weren’t grouped you’d have only 2 values of the explanatory variable: 0 and 1. But even though we assume x is fixed it still makes sense to calculate a confidence interval for the dose x for a given p, contrary to what #VitoshKa says. Just as we can reparameterize the model in terms of ED50, we can do so for ED60 or any other quantile. Parameters are fixed, but we still calculate CI's for them.
The chemcal package has an inverse.predict() function, which works for fits of the form y ~ x and y ~ x - 1
You just have to rearrange the regression equation, but as the comments above state this may prove tricky and not necessarily have a meaningful interpretation.
However, for the case you presented you can use:
(1/coef(model)[2])*(model$family$linkfun(30/50)-coef(model)[1])
Note I did the division by the x coefficient first to allow the name attribute to be correct.
For just a quick view (without intervals and considering additional issues) you could use the TkPredict function in the TeachingDemos package. It does not do this directly, but allows you to dynamically change the x value(s) and see what the predicted y-value is, so it would be fairly simple to move x until the desired Y is found (for given values of additional x's), this will also show possibly problems with multiple x's that would work for the same y.