get MLE of gamma distribution parameters(especially location parameter) in R - r
Hi, I want to estimate gamma distribution parameters hand by hand! I know a lot of R functions to estimate shape and scale parameters, but it seems hard to find code about estimating location parameter.
x <- c(108,91,62,59,84,60,71,105,70,69,66,65,78,83,82,68,107,68,68,69,80,
75,89,68,64,68,70,57,62,87,51,55,56,57,75,98,60,68,81,47,76,48,63,
58,40,62,61,58,38,40,45,68,56,64,49,53,50,39,54,47,37,50,54,70,49,
57,52,47,43,52,57,46,63,56,50,51,50,42,46,56,52,59,45,50,59,44,52,
54,53,63,45,56,55,53,56,46,45,49,63,50,41,42,53,50,58,50,37,53,58,
49,53,51,64,44,53,53,55,43,50,60,51,55,56,52,51,45,49,51,63,48,51,
60,45,40,50,66,62,69,53,54,49,47,63,55,62,57,58,51,50,57,62,45,47,
52,35,41,53,48,59,45,41,52,36,84,62,31,41,48,47,50,50,57,53,37,46,
41,56,51,39,59,53,51,49,45,42,32,55,34,43,35,48,33,41,38,57,37,40,
34,44,43,62,36,41,51,48,31,28,33,35,48,31)
# estimate shape and scale parameter
gamma_likelihood <- function(para){
sum ( (para[2] -1)*log(x) - para[2]*log(para[1]) - log(gamma(para[2])) - x/para[1] + 1/para[1])
}
MLE = optim(c(10,10),
fn = gamma_likelihood,
method = "L-BFGS-B",
lower = 0.00001,
control = list(fnscale = -1),
hessian = T
)
MLE$par
# estimate location, shape and scale parameter
gamma_likelihood <- function(para){
x = x[x > para[1]]
sum ( (para[3] -1)*log(x - para[1]) - para[3]*log(para[2]) -
log(gamma(para[3])) - x/para[2] + para[1]/para[2] )
}
MLE = optim(c(23,6,7),
fn = gamma_likelihood,
method = 'L-BFGS-B',
lower = 0.00000001,
control = list(fnscale = -1)
)
MLE$par
This is my code, I can estimate shape and scale parameters.
However, when it comes to add location parameters into log likelihood. The result seems incorrect.The TRUE parameters are c(21.4, 5.47, 6.0).
If you have any observed value less or equal than your location parameter, your whole likelihood for that value of lambda must be 0 (remember it's a function of parameters, not observations).
x = x[x > para[1]] is cutting observations that don't make sense for a specific location parameter, making your function return a valid number, when it should return -Inf if any of the x is "invalid", since you'd have 0 likelihood.
Here's a corrected version of your log-likelihood function:
# estimate location, shape and scale parameter
gamma_likelihood <- function(para){
if(min(x) < para[1]) return(-Inf)
sum ( (para[3] -1)*log(x - para[1]) - para[3]*log(para[2]) -
log(gamma(para[3])) - x/para[2] + para[1]/para[2] )
}
MLE = optim(c(23,6,7),
fn = gamma_likelihood,
method = 'L-BFGS-B',
lower = 0.00000001,
control = list(fnscale = -1)
)
MLE$par
retults in: [1] 21.161109 5.394343 6.136862
Related
objective function in optim evaluates to length 2 not 1
I do not even know if any of my equations are right, but I did this and its saying the objective function in optim evaluates to length 2 not 1. I am trying to estimate lambda and alfa using mle and log likelihood. set.seed(1001) q <- 1:5000 mean(q) sd(q) ll <- function(lambda = 0.5, alpha = 3.5) { n <- 5000 (-n) * alpha * log(lambda) + n * log(gamma(alpha)) - ((alpha - 1) * sum(log(q))) + lambda * sum(q) } MLE <- optim(c(0.5, 3.5), fn = ll, method = "L-BFGS-B", lower = 0.00001, control = list(fnscale = -1), hessian = T )
There are several places to make your optim fly: you should define your ll as a function of one argument (instead of two arguments, otherwise you will have the error "objective function in optim evaluates to length 2 not 1"), since within optim, your initial value x0 will be pass to objective function fn ll<-function(x){ n<-5000 lambda <- x[1] alfa <- x[2] (-n)*alfa*log(lambda)+n*log(gamma(alfa))-((alfa-1)*sum(log(q)))+lambda*sum(q) } In optim, the dimension of lower bound lower should be the same as x0, e.g., MLE = optim(c(0.5,3.5), fn = ll, method = "L-BFGS-B", lower = c(1e-5,1e-5), control = list(fnscale = -1), hessian = FALSE )
Fitting a local level Poisson (State Space Model)
I am working through "Forecasting with Exponential Smoothing". I am stuck on exercise 16.4 on the part that states: The data set partx contains a history of monthly sales of an automobile part. Apply a local Poisson model. Parameters should be estimated by either maximizing the likelihood or minimizing the sum of squared errors. The local Poisson model is defined as: where and I have the following code, but it seems to be stuck. The optimization always returns something close to the starting values. Am I fitting the local Poisson model correctly? library(expsmooth) data("partx") S <- function(x) { a <- x[1] if(a < 0 | a > 1) return(Inf) n <- length(partx) lambda <- numeric(n+1) error <- numeric(n) lambda[1] <- x[2] for(i in 1:n) { error[i] <- partx[i]-rpois(1,lambda[i]) lambda[i+1] <- (1-a)*lambda[i] + a*partx[i] } return(sum(error^2)) } # returns a = 0.5153971 and lambda = 5.9282414 op1 <- optim(c(0.5,5L),S, control = list(trace = 1)) # returns a = 0.5999655 and lambda = 2.1000131 op2 <- optim(c(0.5,2L),S, control = list(trace = 1))
I know the book says you could use sum of squared errors or MLE but the first option seems wired due too the fact that you have to sample a poison distribution so event if you fix the parameters you would get the different sum of squared errors every time. As you don't say that you have tried the MLE approach I program it. The math is as follows: And the code that implements it is MLELocalPoisson = function(par,y){ alpha = par[1] lambda_ini = par[2] n = length(y) vec_lambda = rep(NA, n) for(i in 1:n){ if(i==1){ vec_lambda[i] = (1-alpha)*lambda_ini+alpha*y[i] }else{ vec_lambda[i] = (1-alpha)*vec_lambda[i-1]+alpha*y[i] } } vec_lambda = c(lambda_ini,vec_lambda[-n]) sum_factorial = sum(sapply(y,function(x)log(factorial(x)))) sum_lambda = sum(vec_lambda) sum_prod = sum(log(vec_lambda)*y) loglike = -sum_prod+sum_lambda+sum_factorial return(loglike) } optim(par = c(0.1,1),fn = MLELocalPoisson,y = partx, method = "L-BFGS-B", lower=c(1e-10,1e-10),upper = c(1,Inf),control = list(maxit = 10000)) the lower values set a 1e-10 is done so the optimization do not try c(0,0) and thus generating a loglikelihood of NaN. EDIT Taking a look at the poisson regression literature the usually define $\lambda = exp(x*\beta)$ and calculate the residuals as $y-exp(x*\beta)$ (have a look at). So it might be possible to do the same in this problem using the formula given by the author for $\lambda$ like this: LocalPoisson = function(par,y,optim){ alpha = par[1] lambda_ini = par[2] n = length(y) vec_lambda = rep(NA, n) y_hat = rep(NA, n) for(i in 1:n){ if(i==1){ vec_lambda[i] = (1-alpha)*lambda_ini+alpha*y[i] }else{ vec_lambda[i] = (1-alpha)*vec_lambda[i-1]+alpha*y[i] } } if(optim){ y_hat = c(lambda_ini,vec_lambda[-n]) return(sum((y_hat-y)^2)) } else { return(data.frame(y_hat = y_hat,y=y, lambda = vec_lambda)) } } optim(par = c(0.1,1),fn = LocalPoisson,y = partx, optim =T,method = "L-BFGS-B", lower=c(1e-10,1e-10),upper = c(1,Inf),control = list(maxit = 10000)) It does not yields the same results as the MLE (and I feel more comfortable with that option but it might be a possible way to estimate the parameters).
Maximum Likelihood Estimation three-parameter Weibull for right censored data
I am trying to estimate the parameters of the three-parametric Weibull distribution with ML for censored data. I've worked it out by using the package flexsurv where I've defined an "own" density function. I've also followed the instructions given in the documentation of the function flexsurv::flexsurvregto build the list with all required information to do the MLE with a customer density function. In the following you can see what I've done so far. library(FAdist) library(flexsurv) set.seed(1) thres <- 3500 data <- rweibull(n = 1000, shape = 2.2, scale = 25000) + thres y <- sample(c(0, 1), size = 1000, replace = TRUE) df1 <- data.frame(x = data, status = y) dweib3 <- function(x, shape, scale, thres, log = FALSE) { dweibull(x - thres, shape, scale, log = log) } pweib3 <- function(q, shape, scale, thres, log_p = FALSE) { pweibull(q - thres, shape, scale, log.p = log_p) } # Not required #qweib3 <- function(p, shape, scale, thres, log.p = FALSE) { # if (log.p == TRUE) { # p <- exp(p) # } # qwei3 <- thres + qweibull(p, shape, scale) # return(qwei3) #} dweib3 <- Vectorize(dweib3) pweib3 <- Vectorize(pweib3) custom.weibull <- list(name = "weib3", pars = c('shape', 'scale', 'thres'), location = 'scale', transforms = c(log, log, log), inv.transforms = c(exp, exp, exp), inits = function(t) { c(1.2 / sqrt((var(log(t)))), exp(mean(log(t)) + (.572 / (1.2 / sqrt((var(log(t))))))), .5 * min(t)) } ) ml <- flexsurvreg(Surv(df1$x, df1$status) ~ 1, data = df1, dist = custom.weibull) The variable y should represent the status of a unit where 1 is a failure and 0 is an unfailed unit until censoring. The initial values for shape and scale are taken from the moments which are also defined in the fitdistrpluspackage. For the threshold parameter there must be a constraining since the threshold must be really smaller than the minimum of the data. Therefore a constraint of threshold is at its max .99 * t_min would be satisfactory (this is something which I haven't implement until now). The output of the above MLE is the following: > ml Call: flexsurvreg(formula = Surv(df1$x, df1$status) ~ 1, data = df1, dist = custom.weibull) Estimates: est L95% U95% se shape 2.37e+00 2.12e+00 2.65e+00 1.33e-01 scale 3.52e+04 3.32e+04 3.74e+04 1.08e+03 thres 2.75e+03 1.51e+03 5.02e+03 8.44e+02 N = 1000, Events: 481, Censored: 519 Total time at risk: 25558684 Log-likelihood = -5462.027, df = 3 AIC = 10930.05 The estimated parameters aren't fine even if there is censoring. I've did this procedure a few times with other randomly generated data... the estimates are always pretty far away from the "truth". Therefore I need an improvement of my code or another possibility to estimate the parameters of a three-parameter Weibull with MLE.
optim in r :non finite finite difference error
I have a simple likelihood function (from a normal dist with mean=0) that I want to maximize. optim keeps giving me this error: Error in optim(par = phi, fn = loglike, estimates = estimates, NULL, hessian = TRUE, : non-finite finite-difference value [1] Here is my data and likelihood function: y = [ -0.01472 0.03942 0.03592 0.02776 -0.00090 ] C = a varcov matrix: 1.66e-03 -0.000120 -6.780e-06 0.000102 -4.000e-05 -1.20e-04 0.001387 7.900e-05 -0.000140 -8.000e-05 -6.78e-06 0.000079 1.416e-03 -0.000070 8.761e-06 1.02e-04 -0.000140 -7.000e-05 0.001339 -6.000e-05 -4.00e-05 -0.000080 8.761e-06 -0.000060 1.291e-03 my log likelihood function is: lglkl = -.5*(log(det(v)) + (t(y)%%vi%%y))` where v = phi*I + C and vi=inverse(v) and I= 5*5 Identity matrix. I am trying to get the mle estimate for "phi". I thought this would be a simple optimization problem but am struggling. Would really appreciate any help. Thanks in advance. My code is below: loglike <- function(phi,y) { v = phi*I + C vi = solve(v) loglike = -.5*(log(det(v)) + (t(y)%*%vi%*%y)) return(-loglike) } phi = 0 parm <- optim(par=phi,fn=loglike,y=y,NULL,hessian = TRUE, method="L-BFGS-B",lower=0,upper=1000)
The error you ran into is because ϕ becomes negative beyond a certain number of iterations (which indicates that the constraints are not being applied correctly by the algorithm). Also, the solution does not converge to a single value but jumps between a few small values before reaching a situation where the updated covariance matrix is no-longer positive definite. At that stage you get det(v) < 0 and log[det(v)] is undefined. The optim algorithm bails out at that stage. To see what's happening, play with the maxit and ndeps parameters in the code below. require("matrixcalc") #------------------------------------------------- # Log-likelihood function #------------------------------------------------- loglike <- function(phi, y) { # Shift the covariance matrix print(paste("phi = ", phi)) #v = phi*I + (1 - phi)*C v = phi*I + C stopifnot(is.positive.definite(v)) # Invert shifted matrix vi = solve(v) # Compute log likelihood loglike = -.5*(log(det(v)) + (t(y) %*% vi %*% y)) print(paste("L = ", loglike)) return(-loglike) } #------------------------------------------------- # Data #------------------------------------------------- y = c(-0.01472, 0.03942, 0.03592, 0.02776, -9e-04) C = structure(c(0.00166, -0.00012, -6.78e-06, 0.000102, -4e-05, -0.00012, 0.001387, 7.9e-05, -0.00014, -8e-05, -6.78e-06, 7.9e-05, 0.001416, -7e-05, 8.761e-06, 0.000102, -0.00014, -7e-05, 0.001339, -6e-05, -4e-05, -8e-05, 8.761e-06, -6e-05, 0.001291), .Dim = c(5L, 5L )) #-------- # Initial parameter #-------- I = diag(5) phi = 50 #-------- # Minimize #-------- parm <- optim(par = phi, fn = loglike, y = y, NULL, hessian = TRUE, method = "L-BFGS-B", lower = 0.0001, upper = 1000, control = list(trace = 3, maxit = 1000, ndeps = 1e-4) )
Error: min(p, na.rm = TRUE) >= 0 is not TRUE
I came across an interesting presentation on page 32, and I started out to replicate and understand a code presented The code from the presentation is as follows: #Unicredit banks code library(evir) library(fExtremes) # Quantile function of lognormal-GPD severity distribution qlnorm.gpd = function(p, theta, theta.gpd, u) { Fu = plnorm(u, meanlog=theta[1], sdlog=theta[2]) x = ifelse(p<Fu, qlnorm( p=p, meanlog=theta[1], sdlog=theta[2] ), qgpd( p=(p - Fu) / (1 - Fu) , xi=theta.gpd[1], mu=theta.gpd[2], beta=theta.gpd[3]) ) return(x) } # Random sampling function of lognormal-GPD severity distribution rlnorm.gpd = function(n, theta, theta.gpd, u) { r = qlnorm.gpd(runif(n), theta, theta.gpd, u) } set.seed(1000) nSim = 1000000 # Number of simulated annual losses H = 1500 # Threshold body-tail lambda = 791.7354 # Parameter of Poisson body theta1 = 2.5 # Parameter mu of lognormal (body) theta2 = 2 # Parameter sigma of lognormal (body) theta1.tail = 0.5 # Shape parameter of GPD (tail) theta2.tail = H # Location parameter of GPD (tail) theta3.tail = 1000 # Scale parameter of GPD (tail) sj = rep(0,nSim) # Annual loss distribution inizialization freq = rpois(nSim, lambda) # Random sampling from Poisson for(i in 1:nSim) # Convolution with Monte Carlo method sj[i] = sum(rlnorm.gpd(n=freq[i], theta=c(theta1,theta2), theta.gpd=c(theta1.tail, theta2.tail, theta3.tail), u=H)) However I get this error which I cannot resolve: Error: min(p, na.rm = TRUE) >= 0 is not TRUE APPENDED Question Many thanks to Shadow. I dont know how to change function reference. Is it as easy as qgpd.fExtremes to qgpd.evir? Thanks to Shadow again to pointing this out. For anyone who wishes to change reference to function from different package (In the above example from fExtremes to evir its as simple as adding evir:::(function). Example: evir:::qgpd( p=(p - Fu) / (1 - Fu) , xi=theta.gpd[1], mu=theta.gpd[2], beta=theta.gpd[3]) )
The reason you get an error here is that the packages fExtremes and evir both implement different versions of the function qgpd. In the evir version, p can be less than 0, while the fExtremes package only implements qgpd for p>=0. The easiest solution to this is to change the qgpd function call to evir:::qgpd.