I would like to find the 4-dimensional vector that minimises some function f which depends on 4 variables. The first three variables take on strictly positive values; the fourth one is unconstrained.
To do this, I would like to use R. I have tried to apply the nlminb function with lower=c(0.001, 0.001, 0.001, -Inf) as one of its optional argument. The procedure does converge but it turns out that the proposed solution does not satisfy the constraint !
I have an alternative solution that consists of using an exponential transformation. However, I would appreciate to figure out why R returns a solution that does not meet my requirements.
Any comment will be appreciated,
Thanks,
Marco
It would be very difficult for me to provide that function here. The reason is that it depends on a number of pre-defined stuff.
Anyway, I am not sure to understand why this occurs but I have realized that my function sometimes returns NaN due to very very large numbers. Actually, I have some doubt about convergence.
On the other hand, I have made some modifications and the alternative solution seems to work well.
As a conclusion, I think that the problem came from my function, not from nlminb.
Best,
Marco
Related
This has been a recurring problem for me, so I'll have to address it here.
I'm trying to build a non-linear optimization problem using optim or fsolve, and I need certain variables as fixed parameters.
So by f(a;b), I mean a function that defines b as given, and immovable for the optimization.
Evidently matlab has a way to define b as just a parameter set & only allow a to be moved in the optimization; in R this doesn't seem to apply in the same way as matlab. Both optim & fsolve require that I specify the initial values of both a & b, and they calculate the results shifting b as well as a.
If anyone could let me know of a good way out of this that would be much appreciated.
Thank you!
You can pass b via the ... argument of optim.
As far as I understand R's nonlinear equation solver nleqslv(x, fn) finds only one solution of the nonlinear equation.
However (as Bhas commented) searchZeros function (the same package) can find my solutions depending on the starting points.
Question: are there some function in R which can help choosing the set of initial points for searchZeros ,which will help me to find all the solutions ?
I am interested in the case of function with several variables.
I undestand that solution to be found pretty much depends on the initial approximation. So the brute force way is to check some reasonable grid of intial approximations. However there might be some more intelligent way to get all the solutions ?
I really love R but from time to time it really gives me a headache...
I have the following simple quadratic minimization problem which can be formulated and solved within no time in Excel (click on picture to enlarge):
and
The problem itself is pretty straightforward: I want to minimize (w1^2+w2^2)/2 by finding the best combination of w1,w2 and b under the constraints that for all Y*(w1*X1+w2*X2+b) >= 1
I know that there is the quadprog package for solving these kinds of problems but I find it so unintuitive that I am not able to specify the problem correctly :-( I hate to say it but Excel seems to be better suited for specifying optimization problems like these :-(((
My question
How to formulate the above problem correctly so that it can be solved with R (no matter which package) and the program arrives at the correct values for w1, w2 and b (as can be seen in the picture above). Please don't just post links but please give actual code that works. It would be great if you could comment your code so it becomes clear why you do the things you do. Thank you!
The necessary data is here:
data <- matrix(c(2.947814,6.626878, 1,
2.530388,7.785050, 1,
3.566991,5.651046, 1,
3.156983,5.467077, 1,
2.582346,4.457777,-1,
2.155826,6.222343,-1,
3.273418,3.520687,-1),ncol=3,byrow=T)
colnames(data) <- c("X1","X2","y")
Addendum
Some people took offense at my request to provide code (and not simply links). I apologized for that and gave my reasons that I did not find any good approaches in the answers so far on SO. The deeper reason for that is that the problem is unusual in the sense that b is only in the constraint and not in the objective function. So I still think that this question is a good fit for SO.
Actually, the problem is a little tricky because b is only present in the inequality constraint matrix but not in the objective function. Therefore the matrix in the quadratic programming problem is only positive semidefinite but not positive definite.
My approach is therefore to set the matrix entry corresponding to b to a very small value - in my case 1e-9. Someone else more familiar with such optimization problems might know how to solve the problem properly...
Calculate solve.QP input
c1=data[,"X1"]*data[,"y"]
c2=data[,"X2"]*data[,"y"]
#I use 1e-9 for the b entry
Dmat=matrix(`[<-`(numeric(9),c(1,5,9),c(1,1,1e-9)),3,3)
dvec=rep(0,3)
Amat=cbind(c1,c2,data[,"y"])
bvec=rep(1,nrow(Amat))
Solve with solve.QP
library(quadprog)
sol=solve.QP(Dmat=Dmat,dvec=dvec,Amat=t(Amat),bvec=bvec)$solution
sol
#[1] 2.903910 1.201258 -14.734964
Same as excel.
I want to use the winsorize function provided in the "robustHD" Package but it does not seem to work with NA's as can be seen in the example
## generate data
set.seed(1234) # for reproducibility
x <- rnorm(10) # standard normal
x[1] <- x[1] * 10 # introduce outlier
x[11]<- NA ## adding NA
## winsorize data
x
winsorize(x)
I googled the problem but didn't find a solution or even anyone with a similar problem. Is winsorizing might considered as a "bad" technique or how can you explain this lack of information?
If you only have a vector to winsorize, the winsor2 function defined here can be easily modified by setting na.rm = TRUE for the median and mad functions in the code. That provides the same functionality as winsorize{robustHD} with 1 difference: winsorize calls robStandardize, which includes some adjustment for very small values. I don't understand what it's doing, so caveat emptor if you forgo it.
If you want to winsorize the individual columns of a matrix (as opposed to the multivariate winsorization using a tolerance ellipse available as another option in winsorize) you should be able to poach the necessary code from winsorize.default and standardize. They do the same thing as winsor2 but in matrix form. Again, you need to add your own na.rm = TRUE settings into the functions as needed.
Some maybe useful thoughts:
Stack Overflow is a programming board, where programming related questions are asked and answers are given. For question whether or not certain statistical procedures are appropriate or considered "bad", you are more likely to find knowledgable people on crossvalidated.
A statistical method and the implementation of a statistical method into a certain software environment are often rather independent. That is to say that if the developer of a package has not included certain features (e.g NA handling) into his package, this does not mean much for the method per se. Having said that, of course it can. The only way to be sure whether the omission of a package feature was intentional is to actually ask the developer of the package. If the question is more geared towards statistics and the validity of the method in the presence of missing values, crossvalidated is likely to be more helpful.
I don't know why you can't find any information on this topic. I can confidently say though that this is the very first time I have heard the term "winsorized". I actually had to look it up, and I can surely say that I have never encountered this approach, and I would personally never use it.
A simple solution to your problem from a computational point of view would be to omit all incomplete cases before you start working with the function. It also makes intuitive sense that cases with missing values cannot be easily winsorized. First, the computation of the mean and standard deviation would have to be done on the complete cases anyway, and then it is unclear which value to assign to those with missing values since they may not necessarily be outliers, even though they could be.
If omitting incomplete cases is not an option for you, you may want to look for imputation methods (on CV).
Can anyone give me a good explanation for what the parameter "algorithm" does in the nls function in R?
Also, how does the formula work? I know it uses a tilda, but I can't really find a down-to-earth explanation of it.
Also, how important are the start values? Do I need to try multiple start values, or can I still have a guarantee that nls will find the correct parameters regardless of the start values I use?
In brief:
nls() is going to vary parameters to try to minimize the square error between your model and your data. There's several good methods it can try to find the minimum. Reading the details about "method" in ?optim will provide some good info and references.
In general, for nonlinear models, your results can be sensitive to initial guess. You should try several different guesses to make sure that the outputs are close. If your results are very sensitive to your guess, you can try re-parameterizing, using a different algorithm, or rethinking your model.
As for the formula, I'd echo the previous answer. Work through the examples in the bottom of ?nls and then try to ask a more specific question.