I am using R version 3.3.2 and the package copula version 0.999-15 to evaluate the fitting of the normal copula to my data. My data and code are:
Data: https://www.dropbox.com/s/tdg8bfzmy4nd1dd/jumps.dat?dl=0
library(copula)
data <- read.csv(file="jumps.dat", head=F, sep="")
cop_model <- ellipCopula("normal", dim = 2)
m <- pobs(as.matrix(data))
fitCopula(cop_model, m, method = 'mpl')
After I run the code I receive the following error:
Error in `freeParam<-`(`*tmp*`, value = estimate) : the length of 'value' is not equal to the number of free parameters
Calls: fitCopula ... fitCopula.ml -> fitCopStart -> fitCopula.icor -> freeParam<-
Execution halted
I have no idea what is happening here. The fitting for Clayton and Gumbel is pretty fine. Searching for similar errors in the web, I have found nothing. Reading the documentation (https://www.rdocumentation.org/packages/copula/versions/0.999-15/topics/fitCopula?) for some specificity for ellipCopula, I have found an specific option for posDef, but it did not returns any solution at all.
Old question, but I found this, so will share my solution.
Try to run the following, this is a minimum working example:
library(copula)
print("-----------")
mycop <- ellipCopula("normal", dim=4)
data <- matrix(runif(400), nrow=4)
fitCopula(mycop, t(data))
print("-----------")
For me, this works fine if I open R and type in the lines one by one, but fails if I run as a script with Rscript. The solution is that you need library(methods) as well.
For some reason this worked with copula v0.999-v14, but was broken by v0.999-v16. Alas.
Related
I am crossposting this from https://stats.stackexchange.com/questions/488201/alternatives-to-or-fix-for-lmerconveniencefunctions-for-use-with-lme4 as it was suggested that people might have more relevant knowledge here.
Recently I updated from R version 3.6 to version 4.0 for my analyses, and noticed that LMERConvenienceFunctions stopped working. Specifically, I use it in conjunction with LME4.
Whenever I try to use the bfFixefLMER_F.fnc (backfitting of fixed effects in LMER models) or pamer.fnc (compute upper- and lower-bound p-values for the analysis of variance or deviance) for a LMER model fitted through LME4, regardless of dataset, I am met with the error "Error in pf(anova.table[term, "F value"], anova.table[term, "Df"], nrow(model#frame) - : Non-numeric argument to mathematical function". I have tried this on two separate computers with the same result. Now, as far as I can tell, LMERConvenienceFunctions hasn't been updated since 2015, so I'm not holding out hope that a fix is forthcoming.
I tried reverting to R 3.6.2, but found the same error using the versions of LME4 that were out shortly before R 4.0 came out. I have finally found the previous version I was using, so this will (hopefully) fix it for my current analysis, but doesn't help if I want to use the most recent version of R and LME4 going forwards.
Other functions of LMERConvenience (namely fitLMER.fnc and mcp.fnc) seem to be working properly, so it doesn't seem to be a systematic issue, but it is definitely one that significantly impedes my work.
Does anyone have any suggestions for alternative packages, or could anyone offer any advice on editing the LMERConvenienceFunctions package so that I can get the broken functions working again? I don't have any experience with changing the coding within packages, so would be starting from bare basics there.
I am also aware that there are some workarounds through adding extra code in my R script, as I did find in my searching for an answer that it had previously been a problem with the same package in 2014 (https://stat.ethz.ch/pipermail/r-sig-mixed-models/2014q2/022264.html), but I am not familiar with writing that kind of code, so would appreciate any guidance there as well.
I managed to get some help from the lovely people working with LME4 on github; for anyone who has this issue in the future, there are a few things of note:
I have emailed the listed maintainer of LMERConvenienceFunctions asking for them to update the CRAN version so that it abides by the rules of CRAN. Hopefully he does this.
For a fix for those using LME4 and LMERConvenienceFunctions, but who are NOT also using lmerTest, recently the anova table headers "Chi Df" and"Df" were updated to "Df" and "npar" respectively (reasons outlined here:https://github.com/lme4/lme4/issues/528). This is an issue for LMERConvenienceFunctions because pamer.fnc called to Df, so needs to be updated to call to npar instead. Further, bfFixefLMER_F.fnc calls to pamer.fnc, so is fixed when pamer.fnc is updated. For anyone unsure how, I found the code using getAnywhere() and modified it, so just copy the code below and paste it once right near the start of the file:
pamer.fnc <- function (model, ndigits = 4)
{
if (length(rownames(anova(model))) == 0) {
cat("nothing to evaluate: model has only an intercept.\n\n")
cat("printing model fixed effects:\n")
fixef(model)
}
else {
dims <- NULL
rank.X = qr(model#pp$X)$rank
anova.table = anova(model)
anova.table = cbind(anova.table, upper.den.df = nrow(model#frame) -
rank.X)
p.values.upper = as.numeric()
p.values.lower = as.numeric()
for (term in row.names(anova.table)) {
p.values.upper = c(p.values.upper, round(1 - pf(anova.table[term,
"F value"], anova.table[term, "npar"],
nrow(model#frame) - rank.X), ndigits))
model.ranef <- ranef(model)
lower.bound <- 0
for (i in 1:length(names(model.ranef))) {
dims <- dim(model.ranef[[i]])
lower.bound <- lower.bound + dims[1] * dims[2]
}
p.values.lower = c(p.values.lower, 1 - pf(anova.table[term,
"F value"], anova.table[term, "npar"],
nrow(model#frame) - rank.X - lower.bound))
}
dv <- gsub(" ", "", gsub("(.*)~.*",
"\\1", as.character(model#call)[2]))
ss.tot <- sum((model#frame[, dv] - mean(model#frame[,
dv]))^2)
aov.table <- as.data.frame(anova(model))
expl.dev <- vector("numeric")
for (i in rownames(aov.table)) {
expl.dev <- c(expl.dev, aov.table[i, 2]/ss.tot)
}
names(expl.dev) <- rownames(aov.table)
anova.table = round(cbind(anova.table, upper.p.val = p.values.upper,
lower.den.df = nrow(model#frame) - rank.X - lower.bound,
lower.p.val = p.values.lower, `expl.dev.(%)` = expl.dev *
100), ndigits)
return(anova.table)
}
}
(You may also need to run the function script for bfFixefLMER_F.fnc separately again, to let R know that bfFixefLMER_F.fnc should be calling from the updated version of pamer.fnc)
For a fix for those using LME4 and LMERConvenienceFunctions, but who ARE also using lmerTest, you will need to either 1) use numDF instead of npar, or 2) replace "anova.table = anova(model)" with "anova.table = anova(model, dff = "lme4") in versions of lmerTest 3.0 onwards. This appears to be due to the anova.lmerModLmerTest function (added in lmerTest 3.0) overwriting what the call to anova() does, with a Type III Satterthwaite analysis listed before lme4, leading to it being defaulted to when dff is not specified.
I'm using R Studio based on R 3.4.3. However, when I tried to call the forecast.HoltWinters function, R told me that "could not find function "forecast.HoltWinters"". Inspect the installed package (v8.2) told me that it's true, there is no forecast.HoltWinters. But the manual in https://cran.r-project.org/web/packages/forecast/ clearly stated that forecast.HoltWinters is still available.
I have also tried stats::HoldWinters, but it's working wrong. The code run fine on another computer, but it couldn't run at all on mine. Is there any solution?
Here is the code. Book2.csv has enough data to last more than 3 periods.
dltt <- read.csv("book2.csv", header = TRUE)
dltt.ts <- ts(dltt$Total, frequency=12, start=c(2014,4))
dltt.ts.hw <- HoltWinters(dltt.ts)
library(forecast)
dltt.ts.hw.fc <- forecast.HoltWinters(dltt.ts.hw) //Error as soon as I run this line
Fit a HoltWinters model using the HoltWinters function and then use forecast. Its all in the help for HoltWinters and forecast, namely "The function invokes particular _methods_ which depend on the class of the first argument". I'll copy the guts of it here:
m <- HoltWinters(co2)
forecast(m)
Note this will call the non-exported forecast.HoltWinters function, which you should never call directly using triple-colon notation as some may suggest.
I am trying to execute Random Forest algorithm on SparkR, with Spark 1.5.1 installed. I am not getting clear idea, why i am getting the error -
Error: could not find function "includePackage"
Further even if I use mapPartitions function in my code , i get the error saying -
Error: could not find function "mapPartitions"
Please find the below code:
rdd <- SparkR:::textFile(sc, "http://localhost:50070/explorer.html#/Datasets/Datasets/iris.csv",5)
includePackage(sc,randomForest)
rf <- mapPartitions(rdd, function(input) {
## my function code for RF
}
This is more of a comment and a cross question rather than an answer (not allowed to comment because of the reputation) but just to take this further, if we are using the collect method to convert the rdd back to an R dataframe, isnt that counter productive as if the data is too large, it would take too long to execute in R.
Also does it mean that we could possibly use any R package say, markovChain or a neuralnet using the same methodology.
Kindly check the functions that can be in used in sparkR http://spark.apache.org/docs/latest/api/R/index.html
This doesn't include function mapPartitions() or includePackage()
#For reading csv in sparkR
sparkRdf <- read.df(sqlContext, "./nycflights13.csv",
"com.databricks.spark.csv", header="true")
#Possible way to use `randomForest` is to convert the `sparkR` data frame to `R` data frame
Rdf <- collect(sparkRdf)
#compute as usual in `R` code
>install.packages("randomForest")
>library(rainForest)
......
#convert back to sparkRdf
sparkRdf <- createDataFrame(sqlContext, Rdf)
At first, let's create some sample categorical data with 3 levels.
y<-sample(c("A","B","C"),50,replace=TRUE)
I'm trying to formulate a Bayesian statistical model in which the y variable follows categorical distribution with parameters theta1,theta2,theta3. These parameters describe the probability a single y[i] belongs to the corresponding category. In the bayesian perspective, these parameters are also random variables and we use to assign a dirichlet prior to them with hyper-parameters alpha1,alpha2,alpha3.
I'm having some problems with the syntax as it seems.
CODE
model<-function(){
#likelihood
for( i in 1:N){
y[i]~ dcat(theta[])
}
#prior
theta[1:3]~ ddirch(alpha[])
}
library(R2OpenBUGS)
model.file <- file.path(tempdir(),"model.txt")
write.model(model, model.file)
y<-sample(c("A","B","C"),50,replace=TRUE)
N<-50
alpha<-c(1,1,1)
data<-list('y','N','alpha')
params<-c('theta')
inits<-function(){theta=c(1/3,1/3,1/3)}
We call OpenBUGS through R, with the bugs function
out<-bugs(data,inits,params,model.file,n.chains = 2
,n.iter=6000,codaPkg = TRUE,n.burnin = 1000,DIC = TRUE)
I've tried different ways to syntactically formulate the above code, dribbling through the errors and getting familiar with the log.txt file (that is the file that keeps the OpenBUGS output) until this code gave me a log.txt with no errors while R still has problems.
R output
Error in bugs.run(n.burnin, OpenBUGS.pgm, debug = debug, WINE = WINE, :
Look at the log file in /tmp/Rtmpofdk0t and
try again with 'debug=TRUE' to figure out what went wrong within OpenBUGS.
In addition: Warning message:
In FUN(X[[i]], ...) : class of 'x' was discarded
log.txt
OpenBUGS version 3.2.3 rev 1012
model is syntactically correct
data loaded
model compiled
initial values generated, model initialized
1000 updates took 0 s
monitor set
monitor set
monitor set
monitor set
deviance set
Thanks in advance for your help
I think you should rename theta1, theta2, theta3 with alpha1, alpha2, alpha3, because you use the alpha1,... in the function ddirch, but you never declare them. Instead you declare theta1 and so on, but never use them.
If there are any other issues, you might have a look at the log file, like the compiler suggests.
After numerous experiments, i figured out that for some reason OpenBUGS cant accept factor variables given as usual. So i changed my data ( format "A","B","C") to numeric (format 1,2,3) with the as.numeric R function and everything ran smoothly!
I'm trying to fit a distribution to some data using the function fdist from the package fdistrplus.
I've started learning R recently and got a hold of this package that I thought could do everything I needed.
When trying to use it to fit to a pareto distribution, I had to input my definitions of the pareto distribution like so:
ppareto = function(x,alpha,theta){
(1-(theta/(x+theta))^alpha)*(x>=0)
}
dpareto = function(x,alpha,theta){
(alpha*theta^alpha*(x+theta)^(-alpha-1))*(x>=0)
}
rpareto = function(n,alpha,theta){
l = array(as.numeric(NA),n)
ltmp = runif(n,0,1)
l = teta/ltmp^(1/alpha)-theta
return(l)
}
When calling
fitdist(data,"pareto",method = "mme")
It said it could not find the "dpareto" function. That is resolved by the code above.
Afterwards it said it needed an order parameter and a empirical moment parameter.
It's my understanding that the order parameter is the vector of orders that I want the routine to use as it's basis for the calculation.
So I tried:
fitdist(data,"pareto",method="mme",order=c(1,2))
As I saw in the examples of the package pdf.
Then it complained that:
Error in mmedist(data, distname, start = start, fix.arg = fix.arg, ...) :
The moment function must be defined.
Okay, next step:
momentfunc = function(x,order){mean(x^order)}
To define the function.
Then I tried
fitdist(data,"pareto",method="mme",order=c(1,2),memp=momentfunc)
Error in mmedist(data, distname, start = start, fix.arg = fix.arg, ...) :
The moment function must be defined.
Then I tried:
fitdist(data,"pareto",method="mme",order=c(1,2),memp="momentfunc")
Error in mmedist(data, distname, start = start, fix.arg = fix.arg, ...) :
The moment function must be defined.
I've searched the web, found examples of the pareto dist and followed them step by step all to no avail...
I've searched stackoverflow and found nothing.
So here I am. If anyone could help me I would be much obliged.
Is it my dumb mistake? Is it a bug in the package?
I've tried this on Windows 7, R 3.1.0
and on MacOS, R 3.0.1 (where I get a warning that the package was built under R 3.0.2)
Thanks in advance
you have to install and load the 'actuar' package