I am fitting a non-linear least squares model in R. I wish to minimize $(Y - f(Xb))^2$ where $f$ is a nonlinear monotone differentiable function, $X$ is a set of features and $b$ is the parameter vector. Is there a way of doing this with constraints on $b$? I want to constrain $b$ to be greater than 0 and I want L1-style shrinkage of some of the elements to 0. Is there a way of doing this in R? nls() doesn't allow for constraints.
You can convert $\|\boldsymbol{x}\|_1$ in your objective by putting a constraint on each element of $\boldsymbol{x}$, into a simple sum and then use quadprog to solve the problem.
Related
I wanted a package that had a function that calculated the angular, linear, coefficient of determination R^2, standard deviation... and I am having trouble finding it.
I've seen the examples here https://juliastats.org/GLM.jl/stable/examples/#Linear-regression-1 but I didn't like it much because I also wanted the coefficient of determination. Could you tell me a specific function to do that or will I need to implement my own function?
StatsBase has coefficient of determination. StatsBase.jl includes many functions that were felt to be too little used for the base installation's Statistics.jl
From the docs of that package:
StatsAPI.r2
—
Function
r2(model::StatisticalModel)
r²(model::StatisticalModel)
Coefficient of determination (R-squared).
For a linear model, the R² is defined as ESS/TSS, with ESS the explained sum of squares and TSS the total sum of squares.
Particularly, instances of StatisticalModel implement the following methods.
StatsAPI.adjr2
—
Function
adjr2(model::StatisticalModel)
adjr²(model::StatisticalModel)
Adjusted coefficient of determination (adjusted R-squared).
For linear models, the adjusted R² is defined as 1 - (1 - (1-R^2)(n-1)/(n-p)), with R^2 the coefficient of determination, n the number of observations, and p the number of coefficients (including the intercept). This definition is generally known as the Wherry Formula I.
My question is on how to correctly interpret (and use) the 'weights' input variable in the nls function of R for non-linear weighted least squares regression.
The solution for solving the unknown parameters in weighted least squares theory is:
From this the variable P is the weight square matrix of size (NxN) where N is the number of data observations.
However, when I look at the nls documentation in R found here, it says the 'weights' to be input is a vector.
This has me puzzled since based on my understanding, the weights should be a square matrix. Some insights with those who have a better understanding is appreciated.
Weight variable in regression, is a measure of how important an observation is to your model due to different reasons (eg. may be in terms of reliability of measurement or inverse of variance estimate). Therefore, some observations may be more important/ weigh higher than others.
Weight vector , in matrix notation converts to a diagonal matrix for i in {1,2,3...n,} both represents the same thing (i.e. weight of ith observation). For nls package in R you need to supply weights in vector form.
Also, it should be noted that, weighted least squares is a special variant of generalized least squares in which we use weights to counter the heteroskedasticity. If the residuals are correlated for observations, perhaps a general model might be suitable.
PS: Cross validated would be the right place to get better detailed answer. Also, It seems to be memory efficient to store a vector rather than a matrix as the number of observations grows
I am currently trying to fit a linear model to count data, where the errors are following a poisson distribution. Namely I would like to minimize the following
where I have i samples. β is a vector with m coefficients and x is consisting of m independent (explanatory) variables. β should sum up to 1 and each coefficient should be larger than 0.
I am using R and I tried the package glmc without much success. The only example in the documentation is only confusing me, as I don't get how the constraint matrix Amat is enforcing a constraint on the coefficients. Is there any other example I could have a look at or another package?
I also tried solving this analytically with medium success.
Any help is appreciated!
kind regards, Lena
I am using the Orange canvas with its regression methods to make some estimations about my data set. The regression coefficients r^2 must be inside of the interval [-1,1] for being meaningful according to statistics field. But sometimes, I've got the regression coefficients -50,.. or 26,.. etc. So, I am confused about that. How can I interprete such the coefficients ? Thank you all already.
From Wikipedia:
Important cases where the computational definition of R2 can yield negative values, depending on the definition used, arise where the predictions that are being compared to the corresponding outcomes have not been derived from a model-fitting procedure using those data, and where linear regression is conducted without including an intercept. Additionally, negative values of R2 may occur when fitting non-linear functions to data.
There is nothing in the definition of R² that theoretically prevents it from having arbitrarily negative values. I guess you can interpret -50 as even worse than -1. But with regard to R² = 26, I'm clueless.
I am using the kernlab package for svm in R.I am using the linear kernel so that I can directly check the importance of the feature vectors, that is my variables.Using the coefficients of these feature vectors,I am required to calculate the weight of the various factors in the model,so that the linear separating plane that the svm will draw in my feature space can we evaluated. Basically I want to calculate the w in transpose(w)*x + b. Could someone please suggest what is to be done. I used the fields alpha and b and apha index and tried to logically calculate the weight vector, but to verify if I was calculating correctly I tried to predict on a test sample its correct predicted score, and this did not match the value predicted by the inbuilt predict function. How to calculate the weights?