exponential fit singularity in r - r

I'm trying to fit a bi-exponential function on this dataset but I can't seem to get it to converge. I've tried using nls2 grid search for the best starting point, I've also tried using nlsLM but neither method works. Does anyone have any suggestions?
function: y = a1*exp(-n1*t) + a2*exp(-n2*t) + c
here is the code:
y <- c(1324,1115,1140,934,1013,982,1048,1143,754,906,895,900,765,808,680,731,728,794,706,531,629,629,519,514,516,454,465,630,415,347,257,363,275,379,329,263,301,315,283,354,230,257,196,268,262,236,220,239,255,213,275,273,294,169,257,178,207,169,169,297,
227,189,214,168,263,227,185,220,169,229,174,231,178,141,195,223,258,206,181,200,150,200,169,194,230,162,174,194,225,216,196,213,150,235,231,224,244,161,219,222,210,
186,188,197,177,251,248,223,273,145,257,236,214,194,211,213,175,168,223,192,318,
263,234,163,202,239,189,216,206,185,185,191,340,145,188,305,112,252,213,245,240,196,196,179,235,241,177,196,191,181,240,164,202,201,306,214,212,185,192,178,203,203,239,141,203,190,216,174,219,153,177,223,207,186,213,173,210,191,258,277)
t <- seq(1,length(y),1)
mydata <- data.frame(t=t,y=y)
library(nls2)
fo <- y~a1*exp(-n1*t)+a2*exp(-n2*t)+c
grd <- expand.grid(a1=seq(-12030,1100,by=3000),
a2=seq(-22110,1900,by=2000),
n1=seq(0.01,.95,by=0.4),
n2=seq(0.02,.9,by=0.25),
c=seq(100,400,by=50))
fit <- nls2(fo, data=allout, start=grd, algorithm='brute-force', control=list(maxiter=100))
fit2 <- nls(fo, data=allout, start=as.list(coef(fit)), control=list(minFactor=1e-12, maxiter=200),trace=F)
error: maximum iteration exceeded
However, if I use nlsLM then I get singularity gradient matrix at initial parameter estimate.

Related

Problem in finding starting values for shifted exponential distribution

I am trying to fit shifted exponential distribution to my data but fitdist function giving error of 100 and failing to estimate starting values. I also used plotdist function to find starting or initial values in order to fit the distribution and I have obtained the followings plots with parameters rate = 0.155 shift = 0.00001 after iteration process and even I used these values in fitdist as well.
I used mledist function to calculate starting values of distribution parameters as well but it is also not working. I used fitdist function as well it gives the following error:
Error in fitdist(x, "sexp", start = list(rate = 0.155, shift = 1e-05)) :
the function mle failed to estimate the parameters,
with the error code 100
The code is as below:
library(fitdistrplus)
library(readxl)
library(tidyverse)
library(here)
library(janitor)
# Load data-------------------------------
pvr <- read_excel(here("data", "pvr.xlsx"))
pvr <- pvr %>%
select(-starts_with("...")) %>%
clean_names(case = "snake")
x <- pvr$headway
rate <- 0.155
shift <- 0.00001
dsexp <- function(x, rate, shift)
dexp(x-shift, rate=rate)
psexp <- function(x, rate, shift)
pexp(x-shift, rate=rate)
qsexp <- function(x, rate, shift)
qexp(x-shift, rate=rate)
f12 <- fitdist(x, "sexp", start = list(rate=0.155, shift=0.00001), lower = c(0, -min(x)))
The data may be downloaded from the following link below:
https://ptagovsa-my.sharepoint.com/:x:/g/personal/kkhan_tga_gov_sa/EfzCE5h0jexCkVw0Ak2S2_MBWf3WUywMd1izw41r0EsLeQ?e=EiqWDc
Can anyone help me in this regard?
The fitdist function uses mle method as default. The code works just by changing the method from mle to mse or mge.

Is it possible to adapt standard prediction interval code for dlm in R with other distribution?

Using the dlm package in R I fit a dynamic linear model to a time series data set, consisting of 20 observations. I then use the dlmForecast function to predict future values (which I can validate against the genuine data for said period).
I use the following code to create a prediction interval;
ciTheory <- (outer(sapply(fut1$Q, FUN=function(x) sqrt(diag(x))), qnorm(c(0.05,0.95))) +
as.vector(t(fut1$f)))
However my data does not follow a normal distribution and I wondered whether it would be possible to
adapt the qnorm function for other distributions. I have tried qt, but am unable to apply qgamma.......
Just wondered if anyone knew how you would go about sorting this.....
Below is a reproduced version of my code...
library(dlm)
data <- c(20.68502, 17.28549, 12.18363, 13.53479, 15.38779, 16.14770, 20.17536, 43.39321, 42.91027, 49.41402, 59.22262, 55.42043)
mod.build <- function(par) {
dlmModPoly(1, dV = exp(par[1]), dW = exp(par[2]))
}
# Returns most likely estimate of relevant values for parameters
mle <- dlmMLE(a2, rep(0,2), mod.build); #nileMLE$conv
if(mle$convergence==0) print("converged") else print("did not converge")
mod1 <- dlmModPoly(dV = v, dW = c(0, w))
mod1Filt <- dlmFilter(a1, mod1)
fut1 <- dlmForecast(mod1Filt, n = 7)
Cheers

Fixed coefficient/Offset in Fine&Gray competing-risk adjusted model (FGR)

I want to fit a Fine&Gray competing risk adjusted model including an offset. In other types of models, I am used to being able to simply put in >offset(x), which will add an offset with coefficient 1.
I tried to do the same using the FGR function from the package riskRegression. I didn't get a warning message, but I then noticed that the coefficients for the model with and without offset(x) were exactly the same for the other variables
Example:
#install.packages(riskRegression")
library(riskRegression)
matrix <- matrix(c(3,6,3,2,5,4,7,2,8,2,
0.8,0.6,0.4,0.25,0.16,0.67,0.48,0.7,0.8,0.78,
60,55,61,62,70,49,59,63,62,64,
15,16,18,12,16,13,19,12,15,14,
0,2,1,0,1,1,0,1,2,0,
345,118,225,90,250,894,128,81,530,268),
nrow=10,ncol=6)
df <- data.frame(matrix)
colnames(df) <- c("x","y","z", "a","event","time")
fit <- FGR(Hist(time,event)~ offset(x)+a+y+z, data=df, cause=1)
fit
fit2 <- FGR(Hist(time,event)~ a+y+z, data=df, cause=1)
fit2
If you run this script, you can see that the coefficients of a, y and z do not change, while you are not getting a warning that offset cannot be used (so apparantly it just simply ignored offset(x)).
Does anybody know of a way to include x as an offset (i.e. with coefficient fixed at 1) in FGR? (Edit: Or another way to calculate the correct coefficents for a, y and z with fixed x?)
You can use the survival package for Fine-Gray models with offsets. Just wrap the variable you would like to have the offset with offset(var). I set the model below to model event 1. See code below:
library(survival)
matrix <- matrix(c(3,6,3,2,5,4,7,2,8,2,
0.8,0.6,0.4,0.25,0.16,0.67,0.48,0.7,0.8,0.78,
60,55,61,62,70,49,59,63,62,64,
15,16,18,12,16,13,19,12,15,14,
0,2,1,0,1,1,0,1,2,0,
345,118,225,90,250,894,128,81,530,268),
nrow=10,ncol=6)
df <- data.frame(matrix)
colnames(df) <- c("x","y","z", "a","event","time")
coxph(Surv(time,event==1)~ offset(x)+a+y+z, data=df)

R nls2 "invalid model formula" fitting gamma

Working in R 3.1.3 and Rstudio.
I want to fit gamma distributions that include a location parameter to data in order to 'shift' the x values to a new origin.
I am trying to use nls2 with the following code:
library(nls2)
theVals <- data.frame(c(26.76,24.3,34.63,38.05,25.56,21.98,20.62,34,26.75,27.79,28.4,33.31,29.26,18.65,22.77,25.72,25.86,25.32,24.08,27.68,26.2,26.16,25.34,26.91,22.6,23.94,23.3,22.34,41.25,24.83,21.66,30.47,26.53,27.74,29.41,25.65,36.05,18.29,27.2,22.99,25.8,21.9,25.27,30.29,22.72,26.49,18.75,33.57,20.87,21.82,20.73,28.59,19.64,33.21,28.94,27.98,22.2,25.95,30.64,26.56,32.11,26.05,20.66,28.64,22.4,22.4,31.91,21.82,26.82,20.77,24.12,28.83,23.07,26.5,21.14,27.29,19.61,25.28,28.6,27.16,22.46,18.19,22.35,23.79,26.32,26.5,27.39,23.29,25.79,26.35,26.38,24.98,20,37.15,25.61,21.39,21.63,24.12,24.4,27.72,42.74,25.33,17.79,21.33,38.65,25.22,28.39,21.61,23.38,25.25,24.88,23.34,26.26,21.96,22.18,24.78,21.15,24.65,21.23,31.9,28.66,27.66,18.08,22.99,22.46,21.69,28.21,29.8,25.72,27.09,20.02,21.26,21.34,27.18,25.48,20.51,20.96,20.07,20.89,27.56,24.43,21.35,24.3,28.1,26.53,29.03,30.08,19.19,21.27,26.18,23.79,36.52,24.81,26.36,24.44,20.99,19.84,23.32,18.21,26.6,21.48,23.21,29.93,23.4,30.9,23.58,21.58,18.38,25.13,23.03,22.73,24.42,22.89,43.44,23.47,27.09,29.96,23.94,28.51,25.74,28.54,30.41,22.7,29.19,25.66,23.89,21.9,36.26,22.61,19.68,27.85,28.83,28.6,22.68,19.07,20.22,24.35,19.09,37.66,22.55,24.25,22.61,26.09,24.42,26.11,32.15,25.78,21.94,23.93,30.19,23.53,26.49,30.48,25.02,28.14,23.43,20.22,17.57,21.68,36.07,24.92,32.48,32.04,25.86,26.69,22.41,26.4,22.72,28.32,22.82,32.73,28.08,29.16,36.18,21.61,23.9,28.8,23.24,24.89,22.17,27.7,34.75,26.74,29.62,17.46,20.06,22.23,22.09,24.05,22.37,24.98,33.26,30.95,26.24,22.16,30.97,27.22,23.81,42.16,28.2,28.37,26.1,26.28,27.44,20.52,35.02,21.43,23.14,18.37,28.86,25.18,28.15,19.97,24.2,25.91,28.92,23.95,19.48,28.57,21.77,23.46,37.51,22.13,37.18,21.83,23.8,18.93,27.43,26.51,25.64,22.15,22.27,29.21,24.45,18.81,22.62,25.16,24.62,30.53,28.77,27.11,22.07,28.95,26.54,39.23,31.9,33,29.93,24.37,26.4,21.33,25.37,25.9,21.25,19.06,25.69,26.44,26.09,23.24,27.04,20.09,28.73,37.06,32.45,22.93,22.7,24.82,31.23,23.25,22.94,20.47,25.7,23.92,34.71,26.5,20.28,21.78,26.54,30.34,21.97,27.38,27.64,34.08,22.05,27.21,20.11,25.79,33.22,31.24,29.93,21.81,30.68,32.46,30.45,22.62,28.83,33.95,27.12,45.51,25.23,29.61,29.09))
colnames(theVals) <- c("theGamma")
fo <- theGamma ~ dgamma(theX-location, shape=theShape, scale=theScale )
startList <- list(location=5, theShape=3, theScale=3)
theGamma=NULL
theX <- 0:50
mo1 <- nls2(fo, start=startList, data=theVals)
I get an error "invalid model formula in ExtractVars".
Curiosly dgamma works fine:
location<- 5
theShape <- 3
theScale <- 3
dgamma(theX-location, shape=theShape, scale=theScale )
I have search stackoverflow and other sites, but can't find an answer to this one.
Any ideas?

User-specified Z matrix in lme

I have been looking forever about how to do this in R and cannot find anything! Basically, I am wanting to shrink predictors using LMM. So I have a set of fixed effects, X, and I have a set of predictors, Z, that I want to put a random effect on so the model is
Y=X*beta+Z*u+e
where u~N(0,sigma_u^2 * I) and e ~ N(0,sigma_e^2 * I). I thought I could do this in lme with
fit <- lme(Y~X,random=pdIdent(~-1+Z))
but I only get the error:
Error in getGroups.data.frame(dataMix, groups) :
invalid formula for groups
Any help on this issue is much appreciated.
Have you tried:
N = sample size
group <- rep(1, N)
fit <- lme(Y~X, random=list(group=pdIdent(~-1+Z)))

Resources