I would like to estimate the 4-parameters that minimises the log-likelihood function LogL(\theta). The first two parameters (a and sigma) are positive; the third one is unconstrained and the last one should be leas than the minimum value in the data. So, I tried to use nlminb function and write it as:
nlminb(start=c(a=0.13,mu=1,sigma=31,xi=0.01),f,lower=c(0.0001,0,0.123,0.01210),
upper=c(2,18.21,50,0.95),control=list(eval.max=100, iter.max=100))
I got a good result, but there is still problem that the estimator of mu take the same value of the upper limit for all the values I try to use and for xi, it take the same lower value even If I change the starting values.
The log likelihood function take form:
-loglik=-n*log(a)+n*log(1-exp(-1))+n*log(sigma) -(a-1)*sum(log(x-mu))+(1/xi+1)*sum(log(1+xi*((x-mu)**a)/sigma))+ sum((1+xi*((x-mu)**a)/sigma)*(-1/xi))
I have another condition that x-mu should be positive and 1+xi*((x-mu)*a)/sigma as well
Any suggestion will be appreciated
Related
I am working with big and complex function. I am using optim to estimate the model parameters. I see from the iteration values of optim, it does not converge even if the current and last values are very close.
For example,
iteration 10 400.0091
iteration 20 400.0092
iteration 30 400.0093
:
:
keep going, say for iteration 1200.
So how can I change the convergence round of the optim that is, if the current iteration is very close to previous iteration then converge.
You are looking for abstol or reltol which are components control argument.
See ?optim for more details. I can't recommend one without an example/context for your question but your call will look something like:
optim(par, fn, [other vars?], control = list(reltol = 1e-5))
I have the following equation I'd need to solve for η:
where the parameter θ is unknown (the general problem is to estimate it), and the vector h depends on that parameter, and a vector of known values y.
What ideally I would like to get is the vector η=η(θ) as a function of the parameter θ so that I can easily use it later.
This question already has answers here:
modify glm function to adopt user-specified link function in R
(2 answers)
Closed 7 years ago.
This question has already been somewhat addressed already in the past on this site, but the answers provided are not fully helpful to me. Here are the details of my questions that are actually somewhat different from what has already been discussed here:
After working hard on this, I remained unable to understand how I can define my own user-specified link function in R for glm. I have several questions on this.
First of all, I understand I have to write my own function (likely modifying one that already exists), and - in it - I need to define the following elements:
linkfun: the link function.
linkinv: the inverse of the link function, as a function of "eta".
mu.eta: the first derivative of the invlink respect to eta.
valideta: that must return TRUE if the value of eta are in the correct interval
And return all of this in a list element.
So far, so good.
Here is the first set of my questions:
The link function is sometimes defined as a function of "y" and sometimes as a function of "mu". What must be done in this respect?
Let's take an example, and type make.link("sqrt"). We then indeed discover that linkfun is sqrt(mu), linkinv is eta^2, mu.eta is 2*eta. So far, so good. However, if you look at make.link("log"), mu.eta is not simply exp(eta) as it should, but pmax(exp(eta), .Machine$double.eps) (i.e., the maximal values of the first derivative for all the eta vector). Why? I remained unable to understand this.
Just for my curiosity, why the algorithm needs the first derivative of the invlink respect to eta? This is not fully clear to me.
In my specific case, I need a quasi-logistic regression for binomial data. Instead of having a standard logit function log(p/(1-p)), I need to have the slightly modified link function (if p is defined as Y/N): log((Y+0.5)/(N-Y+0.5)).
My other question in this case is:
I remained unable to built this.. Can someone give me some hints?
Where can I find a detailed explanation of all of this? I have looked at the good old Chambers & Hastie book (1992), but the explanation is not sufficient. Are there any detailed courses available on the web, etc.?
Not sure whether I can answer all of your questions, but I give it a try:
Can you specify a linkfun which takes mu and y? Up to my knowledge, the link function should only tkae mu as the GLM (as opposed to the LM) models a function of the expecetd value mu (aka linkfunction) instead of the expecetd value itself. Hence, there should be only mu as an argument.
This has to do with vectorization. pmax returns the parallel maxima and we want to assure that we do not report values smaller than Machine$double.eps. So the linkfun does not return the maximum of all exp(eta) (that would be max(exp(eta), .Machine$double.eps)). Imagine now that eta is now a vector of all eta for which you want to calculate then linkinv, with pmax(.) you make sure that you return exp(eta) only in these cases where it is indeed larger than .Machine$double.eps. So you return also a vector of maxima. (try pmax(1:6, 4) you will get [1] 4 4 4 4 5 6)
You need the first derivative in order to calcuate the estimator of the score function of dL / dbeta[j] = sum_i^n((y[i] - mu[i])/(a(phi[i] * V(mu[i])x[ij]/g'(mu[i]) = 0. That is the derivative of the likelihood function w.r.t. to beta[j] (i.e. dL/dbeta[j]) depends on:
a(phi[i]) is a (known) function of the dispersion parameter coming from the respective distribution (e.g. a(phi) = phi = sigma^2 for the normal distribution)
V(mu[i]) for distributions of the exponential family (for which the GLM was designed) you can derive that var(Y) can be written as a(phi) * V(mu) indicating that the variance is indeed a function of the mean.
g'(mu[i]) is finally the derivative of the link function. So in order to solve the score function (thus to get estimates for beta[j], you will need the derivative of the link function
So in your case you need to define:
the linkfun
the inverse
the derivative
function to validate eta
I see your problem that you link function would also need to take y as an parameter, however, I am not sure whether the glm can deal with it, because in its fitting mechanism it will call linkfun at some point and looking at the pre-defined linkfuns, all of these require just one parameter. You could get around with that if you twist the code of glm but this will be quite some work to do (all things untested and just as food for thoughts without any guarantee that it will work):
Provide your linkfun/linkinvers etc as something like function(mu, y) [whatever you want to have here]
Create a copy of glm.fit (glm.fit2 say)
Change calls fo linkfun(mu), linkinv(eta) etc to linkfun(mu, y), linkinv(eta, y) and so forth
when you call your glm provide method = "glm.fit2" to tell glm that it should use your own fitting procedure.
The refernce book for that is McCullagh, Nelder: Generalized Linear Models. Where you find all the explanations about the exponential family of distributions, score functions etc.
You can look into function powerVarianceFamily of package EQL which also uses parameterized families for extended quasi likelihood estaimation.
Update
As just learned from the excellent answer in the previous post, no need to redefine the glm.fit as long as you use y in you linkfun, as by the time linkfun is called y should be known in the encapsualting function. So you should define linkfun like this
function(mu) [a function which uses mu and y -
as y is known within the context where this function is called]
I wish to find an optimisation tool in R that lets me determine the value of an input parameter (say, a specific value between 0.001 and 0.1) that results in my function producing a desired output value.
My function takes an input parameter and computes a value. I want this output value to exactly match a predetermined number, so the function outputs the absolute of the difference between these two values; when they are identical, the output of the function is zero.
I've tried optimize(), but it seems to be set up to minimise the input parameter, not the output value. I also tried uniroot(), but it produces the error f() values at end points not of opposite sign, suggesting that it doesn't like the fact that increasing/decreasing the input parameter reduces the output value up to a point, but going beyond that point then increases it again.
Apologies if I'm missing something obvious here—I'm completely new to optimising functions.
Indeed you are missing something obvious:-) It's very obvious how you should/could formulate your problem.
Assuming the function that must equal a desired output value is f.
Define a function g satisfying
g <- function(x) f(x) - output_value
Now you can use uniroot to find a zero of g. But you must provide endpoints that satisfy the requirements of uniroot. I.e. the value of g for one endpoint must be positive and the value of g for the other endpoint must be negative (or the other way around).
Example:
f <- function(x) x - 10
g <- function(x) f(x) - 8
then
uniroot(g,c(0,20))
will do what you want but
uniroot(g,c(0,2))
will issue the error message values at end points not of opposite sign.
You could also use an optimization function but then you want to minimize the function g. To set you straight: optimize does not minimize the input paramater. Read the help thoroughly.
I want to program the maximum likelihood of a gamma distribution in R; until now I have done the following:
library(stats4)
x<-scan("http://www.cmc.edu/pages/faculty/MONeill/Math152/Handouts/gamma-arrivals.txt")
loglike2<-function(LL){
alpha<-LL$a
beta<-LL$b
(alpha-1)*sum(log(x))-n*alpha*log(beta)-n*lgamma(alpha)}
mle(loglike2,start=list(a=0.5,b=0.5))
but when I want to run it, the following message appear:
Error in mle(loglike2, start = list(a = 0.5, b = 0.5)) :
some named arguments in 'start' are not arguments to the supplied log-likelihood
What am I doing wrong?
From the error message it sounds like mle needs to be able to see the variable names listed in start= in the function call itself.
loglike2<-function(a, b){
alpha<-a
beta<-b
(alpha-1)*sum(log(x))-n*alpha*log(beta)-n*lgamma(alpha)
}
mle(loglike2,start=list(a=0.5,b=0.5))
If that doesn't work you should post a reproducible example with all variables defined and also explicitly indicate which package the mle function is coming from.
The error message is unfortunately criptic because it indicates mising
values owing to the fact that alpha and gamma have to be positive and mle optimizes over the real numbers. Hence, you need to transfomt the vector over which the function is being optimized, like so:
library(stats4)
x<-scan("http://www.cmc.edu/pages/faculty/MONeill/Math152/Handouts/gamma-arrivals.txt")
loglike<-function(alpha,beta){
(alpha-1)*sum(log(x))-n*alpha*log(beta)-n*lgamma(alpha)
}
fit <- mle(function(alpha,beta)
# transfrom the parameters so they are positive
loglike(exp(alpha),exp(beta)),
start=list(alpha=log(.5),beta=log(.5)))
# of course you would have to exponentiate the estimates too.
exp(coef(fit1))
note that the error now is that you are using n in loglike()
which you have not defined. If you define n, then you get an error stating
Lapack routine dgesv: system is exactly singular: U[1,1] = 0. which is
caused either by a not very good guess for the start value of alpha and
beta or (more likely) that loglike() does not have a minima (I think your
deleted post from last night had a slightly different formula which I was
able to get working, but not able to respond to b/c the post was deleted...)
FYI, if you want to inspect the alpha and beta parameters that cause the
errors, you can use scoping assignment to post the most recently called
parameters to the environment in which loglike() is defined as in:
loglike<-function(alpha,beta){
g <<- c(alpha,beta)
(alpha-1)*sum(log(x))-n*alpha*log(beta)-n*lgamma(alpha)
}