More flexible objective definitions with the nloptr package - r

I'm using the nloptr package and everything works well. But I need a way to define the objective function and the constraints in a faster way. I can't write all the settings by hand each time.
For example, I want to solve this problem:
library(nloptr)
eval_f <- function(x){
return(x[4]^2+x[7]^2+x[9]^2)
}
x0 = c(1,1,1,1,0.5,0,0.5,1,0)
hin <- function(x){
h <- numeric(6)
h[1] = x[1]+x[4]-x[2]-x[5]-0.01
h[2] = x[1]+x[4]-x[3]-x[6]-0.01
h[3] = x[2]+x[5]-x[3]-x[6]-0.01
h[4] = x[2]+x[8]-x[1]-x[7]-0.01
h[5] = x[2]+x[8]-x[3]-x[9]-0.01
h[6] = x[1]+x[7]-x[3]-x[9]-0.01
return(h)
}
heq <- function(x){
h <- numeric(1)
h[1] <- x[1]+x[2]+x[3]-3
return(h)
}
res <- slsqp(x0=x0,fn=eval_f,hin = hin,heq = heq)
Everything works.
But I want to define the objective function in a faster way. Can I pass another argument (the indices) to the function in an automatic way? For example:
eval_f <- function(x,indices){
return(x[indices]^2)
}
I tried but I have an error.

The ... argument to slsqp allows you to pass arbitrary arguments through to the objective function. So define a new objective function that takes indices as an argument:
eval_f2 <- function(x,indices){
return(sum(x[indices]^2))
}
... and include indices=c(4,7,9) (to match your previous objective function's definition):
res2 <- slsqp(x0=x0,fn=eval_f2, hin = hin,heq = heq, indices=c(4,7,9))
Check the solution:
all.equal(res$par,res2$par) ## TRUE
factories
More generally, you can define a factory - a function that returns a function. This works because functions have associated environments in which variables (such as the indices) can be stored. This will work even in cases where the top-level function doesn't allow arbitrary arguments to be passed through (and may e.g. be important if you want to use different sets of indices for your objective and constraint functions ...)
eval_factory <- function(indices) {
fun <- function(x) {
return(sum(x[indices]^2))
}
return(fun)
}
res3 <- slsqp(x0=x0, fn=eval_factory(indices=c(4,7,9)),
hin = hin,heq = heq)
all.equal(res$par,res3$par) ## TRUE
factory for hin
hin_factory <- function(A,b) {
fun <- function(x) {
return((A %*% x) + b)
}
return(fun)
}
A0 <- matrix(c(1, -1, 0, 1,-1, 0, 0, 0, 0,
1, 0, -1, 1, 0, -1, 0, 0, 0,
0, 1, -1, 0, 1, -1, 0, 0, 0,
-1, 1, 0, 0, 0, 0,-1, 1, 0,
0, 1, -1, 0, 0, 0, 0, 1, -1,
1, 0, -1, 0, 0, 0, 1, 0, -1),
byrow=TRUE,ncol=9)
all.equal(c(hin_factory(A0,-0.01)(x0)),hin(x0))
res4 <- slsqp(x0=x0, fn=eval_factory(indices=c(4,7,9)),
hin = hin_factory(A0,b=-0.01), heq = heq)
all.equal(res$par, res4$par)

Related

replacing specific positional value in each matrix within a list, with sequential values from a vector in r

I am attempting to replace a specific value in my list of matrices with each sequential value in a vector called one.to.two.s. This vector comprises a sequence of numbers running from 0.4 to 0.89 with steps of 0.01. From the code below, I would like to replace the value 2 in all matrices in the list by each consecutive value of one.to.two.s: the value 2 in the first matrix is replaced by the first value of one.to.two.s, the value 2 in the second matrix is replaced by the second value of one.to.two.s and so forth.
As an extension, I would like to be able repeat the one.to.two.s sequence if the vector had say length 50 and the list was say length 100. Below, I have a for loop which doesn't work, but I believe this could be handled with lapply somehow.
A <- lapply(1:50, function(x) # construct list of matrices
matrix(c(0, 0, 0, 0,
2, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 1), nrow = 4,ncol=4, byrow = TRUE))
Anew <-A
one.to.two.s <- c(seq(from = 0.40, to = 0.89,by=0.01))
for(t in 1:length(Anew)) {
Anew[[t]][2,1] <- one.to.two.s
}
Using an example one.to.two.s which is shorter than length(A), you could use rep with length.out to make it the correct length, and then Map over that vector and A to create Anew
one.to.two.s <- seq(from = 0.4, to = 0.8, by = 0.01)
Anew <- Map(function(A, x) {
A[2, 1] <- x
A
}, A, rep(one.to.two.s, length.out = length(A)))
Created on 2022-01-27 by the reprex package (v2.0.1)
You can try the following for loop if you have longer list than the vector
for(t in 1:length(Anew)) {
Anew[[t]][2,1] <- one.to.two.s[(t-1)%%length(one.to.two.s)+1]
}
I forgot to add [t] to the end of my replacement as well. Also can repeat a vector ahead of time.
for(t in 1:length(Anew)) {
Anew[[t]][2,1] <- one.to.two.s
}
instead becomes
for(t in 1:length(Anew)) {
Anew[[t]][2,1] <- one.to.two.s[t]
}
I believe this is what you are looking for. In this example, the list consists of 105 matrices.
# use replicate() instead of lapply()
B <- 50L
A <- replicate(B*2.1,
matrix(c(0, 0, 0, 0,
2, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 1), nrow = 4,ncol=4, byrow = TRUE),
simplify = FALSE)
Anew <- A
one.to.two.s <- seq(from = 0.40, to = 0.89, by = 0.01)
# loop over all elements in Anew
for (t in seq_along(Anew)) {
Anew[[t]][2,1] <- one.to.two.s[
seq_len(length(Anew) + 2L) %% (length(one.to.two.s) + 1L)
][t]
}
# > head(sapply(Anew, '[', 2))
# [1] 0.40 0.41 0.42 0.43 0.44 0.45
# > tail(sapply(Anew, '[', 2))
# [1] 0.89 0.40 0.41 0.42 0.43 0.44

How to implement special cases of state space models in dlm? Or how to obtain a Kalman-smoother from the FKF package?

I am trying to estimate a state-space model to obtain the potential output (y_p) from data on output (y) and the unemployment rate (u) using R. The model is already programmed in EViews and I simply want to reproduce its results. The model is described by the following eqations (with time indizes):
signal equations:
(i) y_t = y_p_t + eps_y_t
(ii) u_t = beta_0 + beta_1(y_t-y_p_t) + eps_u_t
state equations:
(iii) y_p_t = y_p_(t-1) + g_(t-1)
(iv) g_t = g_(t-1) + eps_g_t
I have tried different packages. But there are different problems: Either there are no intercepts allowed (dlm package) or there is no smoother function (FKF package). So I do have two questions, either of them answered would solve my problem. The first (Questions 1a and 1b) relates to the specification of an appropriate state-space model in the dlm-package; the second (Question 2) relates to a smoothing function that could be used with the FKF package.
Question 1a. In the dlm-package no intercepts are allowed. So I put beta_0 and the output gap (gap_t = y_t-y_p_t) into the state vector using the JGG-matrix to reference to the y_t-data and tried to estimate beta_1 subsequently via maximum likelihood. However, I didn't obtain reasonable results.
# States: x(1) y_pot, x(2) growth, x(3) y_gap, x(4) beta_0
# Signal: y(1) y, y(2) u
beta_1 <- -0.2
beta_0 <- 0.03
# Measurement
FF <- matrix(c(1, 0, 0, 0,
0, beta_1, 0, 1), 2, 4)
# Transition
GG <- matrix(c(1, 0, -1, 0,
1, 1, -1, 0,
0, 0, 1, 0,
0, 0, 0, beta_0), 4, 4)
JGG <- matrix(c(0, 0, 0, 0,
0, 0, 0, 0,
0, 0, 1, 0,
0, 0, 0, 0), 4, 4)
# Covariance Transition
W <- diag(1e-2, 4)
# Covariance Measurement
V <- matrix(c(1e-2, 0,
0, 1e-2), 2, 2)
m0 <- c(11.4, 0.04, 0, 0.03)
C0 <- diag(1, 4) # 1e-7
C0[3,3] <- 0.1
C0[4,4] <- 0.1
# Now bring them into the dlm-object
myMod <- dlm(FF = FF,
GG = GG,
JGG = JGG,
X = dataMLE,
W = W,
V = V,
m0 = m0,
C0 = C0)
buildFun <- function(theta) {
V(myMod)[1,1] <- lambda_ss*exp(theta[1])
V(myMod)[2,2] <- exp(theta[2])
W(myMod)[2,2] <- exp(theta[1])
FF(myMod)[2,3] <- theta[3]
return(myMod)
}
myMod.mle <- dlmMLE(y = dataMLE, parm = c(-10, -10, -.2),
build = buildFun,
lower = c(rep(-1e6, 3)),
upper = c(rep(1e6, 3)),
control = list(trace = 1, REPORT = 5, maxit = 1000))
Question 1b. I've also tried to use the state vector x(1) y_pot, x(2) growth, x(3) beta_1, x(4) beta_0, and to use JFF to get the y_t-data for the output-gap-calculation... but this approach was not sucessfull either.
Question 1: Do you know of a way in which this rather simple model could be implemented within the dlm-package? The problems are the incercepts on the one hand and on the other the interaction of the beta_1-estimation with the ouput-gap, which consists itself of one state-variable and one external signal.
A more promising approach seemed to be to use the FKF-package. However, no smoother function is provided within this package.
Question 2: Is there a way to obtain the smoothed output instead of the Kalman-filtered output usind the FKF-package?
I deepely appreciate any help on this problem!
Thank you a lot!
Samuel

R minimize portfolio function with gradient

I want to minimize function
f <- function(u){
return(-(1+u[1]+u[2]+u[3]+u[4]))
}
with gradient grad
And I have constraints:
1) u[1]+u[2]+u[3]+u[4] = 1
2) 0<=u[1]<=1, 0<=u[2]<=1, 0<=u[3]<=1, 0<=u[4]<=1
How to make it correctly? I can make it only for 2 constraint
optim(par=c(0,0,0,0), fn=f,lower=c(0, 0, 0, 0), upper=c(1, 1, 1, 1),method="L-BFGS-B")
But 1 constraint is not true in this case
Maybe you can try fmincon from package pracma like below
pracma::fmincon(c(0,0,0,0),
f,
gr = grad,
Aeq = cbind(1,1,1,1),
beq = 1,
lb = c(0,0,0,0),
ub = c(1,1,1,1))

Developing a Continuous-time Markov Chain model to simulate the distribution (counts) of parasites on a fish in R

I am developing a (complex) CTMC model in R (as a beginner in R) to simulate the distribution of parasite load (counts) at 8 different body parts of a fish; assuming a parasite can move from one body part to another randomly. To start with a simple block of codes as below, it is able to simulate CTMC but returns errors below when trying to repeat it a number of times (for some runs).
Error in sample.int(x, size, replace, prob) : too few positive
probabilities
Warning message:
In rexp(1, Qt) : NAs produced
I realized, most of the rate values in the Q (matrix) are zero and thus, sampling at such instances, returns such as error. I would like to know if there is anyway to correct this error so as to run the model a number of times without such an error.
To make it simple, I started with these piece of codes:
Fishsim_model <- function(b,d,m,X0,Ti){
#b=birth rate; d=death rate; m=movement rate; Ti=finishing time
#X0=initial distribution; X= states
X <- X0
Ti <- floor(Ti)
ti <- 0 # (initial) time
day <- 1
saved <- matrix(0, Ti+1, 8) #Matrix of zeros to save final results
saved[day,] <- X0
Q <- rep(0, 36) # vector of rates
Qt <- 0 # Qt = sum(Q) is departure rate from current state
while (ti < Ti){
#Calculate rates
Q[1]<-X[1]*b
Q[2]<-X[2]*b
Q[3]<-X[3]*b
Q[4]<-X[4]*b
Q[5]<-X[5]*b
Q[6]<-X[6]*b
Q[7]<-X[7]*b
Q[8]<-X[8]*b
Q[9]<-X[1]*d
Q[10]<-X[2]*d
Q[11]<-X[3]*d
Q[12]<-X[4]*d
Q[13]<-X[5]*d
Q[14]<-X[6]*d
Q[15]<-X[7]*d
Q[16]<-X[8]*d
Q[17]<-X[1]*m
Q[18]<-X[3]*m/3
Q[19]<-X[4]*m/5
Q[20]<-X[6]*m/2
Q[21]<-X[4]*m/5
Q[22]<-X[5]*m/2
Q[23]<-X[2]*m/2
Q[24]<-X[5]*m/2
Q[25]<-X[3]*m/2
Q[26]<-X[2]*m/2
Q[27]<-X[3]*m/3
Q[28]<-X[7]*m/2
Q[29]<-X[8]*m/2
Q[30]<-X[4]*m/5
Q[31]<-X[4]*m/4
Q[32]<-X[7]*m/2
Q[33]<-X[6]*m/2
Q[34]<-X[8]*m/2
Q[35]<-X[3]*m/4
Q[36]<-X[4]*m/5
Qt <- sum(Q)
# time for next jump
ti <- ti + rexp(1, Qt)
# new state
j <- sample(36, 1, prob = Q)
if (j == 1) {
X[1] <- X[1] + 1
} else if (j==2){
X[2]<- X[2]+1
} else if (j==3){
X[3]<-X[3]+1
} else if (j==4){
X[4]<-X[4]+1
} else if (j==5){
X[5]<-X[5]+1
} else if (j==6){
X[6]<-X[6]+1
} else if (j==7){
X[7]<-X[7]+1
} else if (j==8){
X[8]<-X[8]+1
} else if (j==9){
X[1]<-X[1]-1
} else if (j==10){
X[2]<-X[2]-1
} else if (j==11){
X[3]<-X[3]-1
} else if (j==12){
X[4]<-X[4]-1
} else if (j==13){
X[5]<-X[5]-1
} else if (j==14){
X[6]<-X[6]-1
} else if (j==15){
X[7]<-X[7]-1
}else if (j==16){
X[8]=X[8]-1
} else if (j==17){
X[1]=X[1]-1
X[3]=X[3]+1
} else if (j==18){
X[1]=X[1]+1
X[3]=X[3]-1
} else if (j==19){
X[4]=X[4]-1
X[6]=X[6]+1
} else if (j==20){
X[4]=X[4]+1
X[6]=X[6]-1
} else if (j==21){
X[4]=X[4]-1
X[5]=X[5]+1
} else if (j==22){
X[4]=X[4]+1
X[5]=X[5]-1
} else if (j==23){
X[2]=X[2]-1
X[5]=X[5]+1
} else if (j==24){
X[2]=X[2]+1
X[5]=X[5]-1
} else if (j==25){
X[3]=X[3]-1
X[2]=X[2]+1
} else if (j==26){
X[3]=X[3]+1
X[2]=X[2]-1
} else if (j==27){
X[3]=X[3]-1
X[7]=X[7]+1
} else if (j==28){
X[3]=X[3]+1
X[7]=X[7]-1
} else if (j==29){
X[8]=X[8]-1
X[4]=X[4]+1
} else if (j==30){
X[8]=X[8]+1
X[4]=X[4]-1
} else if (j==31){
X[4]=X[4]-1
X[7]=X[7]+1
} else if (j==32){
X[4]=X[4]+1
X[7]=X[7]-1
} else if (j==33){
X[6]=X[6]-1
X[8]=X[8]+1
} else if (j==34){
X[6]=X[6]+1
X[8]=X[8]-1
} else if (j==35){
X[3]=X[3]-1
X[4]=X[4]+1
} else if (j==36){
X[3]=X[3]+1
X[4]=X[4]-1
}
day.old <- day #Keep track of previous days
day=ceiling(ti)
if (day > day.old){
saved[(day.old+1):day,] <-
matrix(saved[day.old,], (day - day.old), 8, byrow=TRUE) # What was this intended to achieve?
saved[day,] <- X
cat("day =", day, X, "\n")
#cat('day:', sprintf('%7.4f',day.old), ' tail:', X[1], ' Anal:', X[2], ' LB:', X[3],' UB:',
# X[4],' Pelvic:', X[5],' Pectoral:', X[6],' dorsal:', X[7],' Head:', X[8], '\n')
}
}
return(saved)
}
#Suppose parasite prefer tail
b <- 0.5 #birth rate per day
d <- 0.14 #death rate
m <- 0.3 #movement rate
X0 <- c(2,0,0,0,0,0,0,0)# initial condition of gyro that prefers the tail
Ti <- 17 #finishing time
#set.seed(12)
Results <- Fishsim_model(b, d, m, X0, Ti)
Results
Both error messages suggest that at some point all values in the Q vector are 0, which causes the first error. Example : sample(3,1, prob = c(0,0,0)).
Consequently, the rate (Qt), which u are passing to the exponential distribution random generator is also 0 and NaN is returned, which causes the second error. Example : rexp(1,0)
Unfortunately, your code was hard for me to read, so I refactored it. You can find an augmented version below, which works with the example input. My guess is that there is an error somewhere causing Q to take the 0 state, you can trace it with some print statements and debug functionalities. You can further refactor this piece of code to make it even more readable and performant.
In general, you can investigate the mathematical conditions for the initial inputs to guarantee that the Q vector never falls in the 0 state. I am not sure if you are looking for pointers on how to do this, as well.
HTH
CHANGE_MATRIX <- matrix(
c(-1, 0, 1, 0, 0, 0, 0, 0
, 1, 0, -1, 0, 0, 0, 0, 0
, 0, 0, 0, -1, 0, 1, 0, 0
, 0, 0, 0, 1, 0, -1, 0, 0
, 0, 0, 0, -1, 1, 0, 0, 0
, 0, 0, 0, 1, -1, 0, 0, 0
, 0, -1, 0, 0, 1, 0, 0, 0
, 0, 1, 0, 0, -1, 0, 0, 0
, 0, 1, -1, 0, 0, 0, 0, 0
, 0, -1, 1, 0, 0, 0, 0, 0
, 0, 0, -1, 0, 0, 0, 1, 0
, 0, 0, 1, 0, 0, 0, -1, 0
, 0, 0, 0, 1, 0, 0, 0, -1
, 0, 0, 0, -1, 0, 0, 0, 1
, 0, 0, 0, -1, 0, 0, 1, 0
, 0, 0, 0, 1, 0, 0, -1, 0
, 0, 0, 0, 0, 0, -1, 0, 1
, 0, 0, 0, 0, 0, 1, 0, -1
, 0, 0, -1, 1, 0, 0, 0, 0
, 0, 0, 1, -1, 0, 0, 0, 0)
, ncol = 8
, byrow = T
)
UPDATE_LOCATION <- c(1, 3, 4, 6, 4, 5
, 2, 5, 3, 2, 3, 7
, 8, 4, 4, 7, 6, 8
, 3, 4)
UPDATE_WEIGHT <- c(1, 3, 5, 2, 5, 2
, 2, 2, 2, 2, 3, 2
, 2, 5, 4, 2, 2, 2
, 4, 5)
UPDATE_INDEX <- seq(17, 36)
BODY_PARTS <- c(' Tail'
,' Anal'
,' LB'
,' UB'
,' Pelvic'
,' Pectoral'
,' dorsal'
,' Head')
Fishsim_model <- function(b,d,m,X0,Ti){
#b=birth rate; d=death rate; m=movement rate; Ti=finishing time
#X0=initial distribution; X= states
X <- X0
Ti <- floor(Ti)
ti <- 0 # (initial) time
day <- 1
saved <- matrix(0, Ti+1, 8) #Matrix of zeros to save final results
saved[day,] <- X0
Q <- vector('numeric', 36)
Qt <- 0 # Qt = sum(Q) is departure rate from current state
while (ti < Ti){
#Calculate rates
Q[1:8] <- X*b
Q[9:16] <- X*d
Q[UPDATE_INDEX]<-X[UPDATE_LOCATION[seq_along(UPDATE_INDEX)]]*
(m*(1/UPDATE_WEIGHT[seq_along(UPDATE_INDEX)]))
Qt <- sum(Q)
# time for next jump
ti <- ti + rexp(1, Qt)
# new state
j <- sample(36, 1, prob = Q)
if (j <= 8) {
X[j] <- X[j] + 1
} else if (j <= 16){
X[j-8] <- X[j-8] - 1
} else{
X <- X + CHANGE_MATRIX[j-16, ]
}
day.old <- day #Keep track of previous days
day <- ceiling(ti)
if (day > day.old){
# What was this intended to achieve?
# saved[(day.old+1):day,] <- matrix(saved[day.old,]
# , (day - day.old)
# , 8
# , byrow=TRUE)
saved[day, ] <- X
cat(
paste('day:', day)
, '\n'
, paste(BODY_PARTS, ':', X)
, '\n'
)
}
}
return(saved)
}
#Suppose parasite prefer tail
b <- 0.5 #birth rate per day
d <- 0.14 #death rate
m <- 0.3 #movement rate
X0 <- c(2,0,0,0,0,0,0,0)# initial condition of gyro that prefers the tail
Ti <- 17 #finishing time
#set.seed(12)
Results <- Fishsim_model(b, d, m, X0, Ti)
Results
I have been able to figure the way out to prevent the errors based previous recommendation received here. This will help me run the model a number of times without any error message. I just needed to break the loop from running when the sum of rates equals 0.
Below is the single code of line I needed to include in my codes;
enter code here
Qt=sum(Q)
if (Qt == 0) break #Just this line code to help break the loop and return to the next
ti <- ti + rexp(1,Qt)
j=sample(152,1,prob=Q)

R linear programming set up in linprog package ignores constraint (less than or equal to) using solveLP

I am using solveLP in the linprog R package to solve a simple linear programming problem:
minimize -x1-x2
subject to 2*x1+x2+x3 =12
x1+2*x2 +x4 = 9
x1,x2,x3,x4 >=0
which has dual equivalent:
maximize 12*y1+9*y2
subject to 2*y1+y2 <= -1
y1+2*y2 <= -1
y1,y2 <=0
If I state the problem in the primal form I get the right results (5,2,0,0). But when stating the problem in the dual form, the first two constraints simply get ignored. I get the result (0,0) which clearly violates (2*y1+y2 <= -1 and y1+2*y2 <= -1), is there an extra setting or parameter I am missing ? Please have a look at the code underneath and let me know what you think:
require(linprog)
objVec <- c(-1,-1,0,0)
rhsConstr <- c(12, 9,0,0,0,0)
Amat <- rbind( c( 2, 1, 1, 0 ),
c( 1, 2, 0, 1 ),
c( 1, 0, 0, 0 ),
c( 0, 1, 0, 0 ),
c( 0, 0, 1, 0 ),
c( 0, 0, 0, 1 ))
res <- solveLP( objVec, rhsConstr, Amat, maximum=FALSE, const.dir = c("==","==",">=",">=",">=",">=") , lpSolve=TRUE)
res$solution
# dual problem - this is where the problem is
objVec <- c(12,9)
rhsConstr <- c(-1.0,-1.0,0,0)
Amat <- rbind( c( 2, 1),
c( 1, 2),
c( 1, 0),
c( 0, 1))
res <- solveLP( objVec, rhsConstr, Amat, maximum=TRUE, const.dir = rep("<=",length(rhsConstr)))
res$solution
In positive space the dual problem does give the right answer (1/3,1/3):
objVec <- c(12,9);
rhsConstr <- c(1,1,0,0);
Amat <- rbind( c( 2, 1), c( 1, 2), c( 1, 0), c( 0, 1));
res <- solveLP( objVec, rhsConstr, Amat, maximum=FALSE, const.dir = rep(">=",length(rhsConstr)) , lpSolve=TRUE);
res$solution;
As with many linear programming libraries,
there are implicit non-negative constraints, y>=0:
there are no feasible solutions
(but I would expect res$status to indicate this).
solveLP does not seem to allow negative solutions:
you can either transform the problem to have only non-negative values
(replace y1 with u1-v1, y2 with u2-v2)
or use another package, that allows negative values.
library(Rglpk)
objVec <- c(12,9)
rhsConstr <- c(-1.0,-1.0,0,0)
Amat <- rbind( c( 2, 1),
c( 1, 2),
c( 1, 0),
c( 0, 1))
Rglpk_solve_LP(
objVec, Amat, rep("<=",4), rhsConstr,
bounds = list( lower = list( ind=c(1L,2L), val=c(-Inf,-Inf) ),
upper = list( ind=c(1L,2L), val=c( Inf, Inf) ) ),
max=TRUE
)

Resources