I'm trying to reproduce the following example from David Ruppert's "Statistics and Data Analysis for Financial Engineering", which fits Students t-distribution to historical risk free rate:
library(MASS)
data(Capm, package = "Ecdat")
x <- Capm$rf
fitt <- fitdistr(x,"t", start = list(m=mean(x),s=sd(x)), df=3)
as.numeric(fitt$estimate)
0.437310595161651 0.152205764779349
The output is accompanied by the following Warnings message:
Warning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs producedWarning message:
In log(s): NaNs produced
It appears from the R's help file that MASS::fitdistr uses maximum-likelihood for finding optimal parameters. However, when I do optimization manually (same book), all goes smoothly and there is no warnings:
library(fGarch)
loglik_t <- function(beta) {sum( - dt((x - beta[1]) / beta[2],
beta[3], log = TRUE) + log(beta[2]) )}
start <- c(mean(x), sd(x), 5)
lower <- c(-1, 0.001, 1)
fit_t <- optim(start, loglik_t, hessian = T, method = "L-BFGS-B", lower = lower)
fit_t$par
0.44232633269102 0.163306955396773 4.12343777572566
The fitted parameters are within acceptable standard errors, and, in addition to mean and sd I have gotten df.
Can somebody advise me please:
Why MASS::fitdistr produces warnings whereas optimization via fGarch::optim succeeds without a warning?
Why there is no df in MASS::fitdistr output?
Is there a way to run MASS:fitdistr on this data without a warning and get df?
Disclaimer:
a similar question was asked couple of times without an answer here and here
You are not passing the lower argument to the function fitdistr which leads it to make a search in positive and negative domain. By passing the lower argument to function
fitt <- fitdistr(x,"t", start = list(m=mean(x),s=sd(x)), df=3, lower=c(-1, 0.001))
you get no NaNs -as you did in your manual optimisation.
EDIT:
fitt <- fitdistr(x,"t", start = list(m=mean(x),s=sd(x),df=3),lower=c(-1, 0.001,1))
returns non-integer degrees of freedom result. However, I guess, the rounded value of it, which is round(fitt$estimate['df'],0) can be used for fitted degrees of freedom parameter.
Related
I am new to programming in R and I am needing to use a zero inflated mixed effects generalized linear model for my data. I have found online that using the package glmmTMB is best for this. I have code and it runs, but I am missing some of my output- it says "NA". What can I fix so it gives me values instead of "NA"? I also keep getting a warning that says "Model convergence problem; non-positive-definite Hessian matrix" and do not understand what that means, any help?
I have looked up the error code online on a troubleshooting page (https://cran.r-project.org/web/packages/glmmTMB/vignettes/troubleshooting.html), but the explanation was way over my head.
Here is what my code looks like (there are variables commented out so they we can just look at the specific ones for now):
No.WavesModel2 <- glmmTMB(No.WavesScaled~(MudTemp+I(MudTemp^2)+
#Ba+I(Ba^2)+
#Cd+I(Cd^2)+
Pb+I(Pb^2)+
#Sr+I(Sr^2)+
#Zn+I(Zn^2)+
#Fe+I(Fe^2)+
Mn+I(Mn^2)+
#Cu+I(Cu^2)+
CarIndex+I(CarIndex^2)+
No.Pedestrians)+I(No.Pedestrians^2)
(1|Site),
data=MaleWavingR,
ziformula=~1,
family=poisson)
summary(No.WavesModel2)
These are the warning messages that show up in my output:
1: In nlminb(start = par, objective = fn, gradient = gr, control = control$optCtrl) :
NA/NaN function evaluation
2: In nlminb(start = par, objective = fn, gradient = gr, control = control$optCtrl) :
NA/NaN function evaluation
3: In fitTMB(TMBStruc) :
Model convergence problem; non-positive-definite Hessian matrix. See vignette('troubleshooting')
I have been trying to find the glasso matrix for a covariance matrix input link:
SP_glasso_matrix= Glasso(SP_covar_matrix, rho=0)
Warning message returned is:
Warning message: In glasso(SP_covar_matrix, rho = 0) : With rho=0,
there may be convergence problems if the input matrix is not of full
rank
Is there something wrong with my covariance matrix? What is rho and how do I set it?
I cannot debug the following Tweedie error in R. The tweedie function code has worked in other applications and I have checked the data. Any help would be appreciated.
glm1a <- glm(Incurred_Loss ~ operator_age_group + credit_group +
motor_length_group +stated_amount_group + Marine_Age_group,
family=tweedie(var.power=tweedie.p, link.power=0),
data=Tnwater1.df, offset=log(Exposure),
weights=Exposure^(tweedie.p - 1))
Error in if (!(validmu(mu) && valideta(eta))) stop("cannot find valid starting values: please specify some", :
missing value where TRUE/FALSE needed
In addition: Warning message:
In log(mu) : NaNs produced
I'm trying to run a factor analysis on a set of 80 dichotomous variables (1440 cases) using the hector function from the polycor package and the instructions I found here: http://researchsupport.unt.edu/class/Jon/Benchmarks/BinaryFA_L_JDS_Sep2014.pdf
Sadly, after I select just the variables interest from the rest of my dataset and run the factor analysis on them, I seem to consistently get the following error and warnings
Error in optim(0, f, control = control, hessian = TRUE, method = "BFGS") :
non-finite finite-difference value [1]
In addition: Warning messages:
1: In log(P) : NaNs produced
2: In log(P) : NaNs produced
This is with the command/when I hit the step described in the above PDF:
testMat <- hetcor(data)$cor
No idea what this means or how to proceed... Your thoughts are appreciated. Thank you!
Reproducable example which will give the mentioned error code every time is:
(Note that even without set.seed, the error comes up every time)
library(MASS)
set.seed(seed = 1)
data<-rnorm(n = 10000,mean = 0.0002,sd = 0.001)
fitdistr(x = data,densfun = "t")
The error message is:
Error in stats::optim(x = c(-0.000426453810742332, 0.000383643324222082, :
non-finite finite-difference value [2]
In addition: Warning message:
In log(s) : NaNs produced
The problem is the "non-finite finite-difference value". Fitdistr does not give me a result.
My knowledge:
I researched and apparently this could mean that a parameter is negative during the iteration. And that the solution could be to provide a better or at least different starting value. But I could not figure out how to do this and I am not sure if this is the issue.
MY QUESTION:
a) Why do I get this error message
and
b)how can I fix it in R, so that I can fit the student-t distribution to my normally distributed data?