I'm running a generalised additive mixed model using the gamlss() function. I used the fitDist() on my data and it recommended I used a zero inflated poisson. My response variable is 'deg' and is count data but has a lot of zeros.
fitDEG <- fitDist(deg, data=node_dat, k = 2, type = "counts", try.gamlss = TRUE)
> fitDEG
Family: c("ZIP", "Poisson Zero Inflated")
Fitting method: "nlminb"
Call: gamlssML(formula = y, family = DIST[i])
Mu Coefficients:
[1] 0.3803
Sigma Coefficients:
[1] 2.81
Degrees of Freedom for the fit: 2 Residual Deg. of Freedom 82208
Global Deviance: 38484.9
AIC: 38488.9
SBC: 38507.6
I've tried running a model with a single smoothed term, one numerical explanatory variable (TL), four categorical explanatory variables and two random effects.
mDEG_zip <- gamlss(formula = deg ~ pb(SE_score) + TL + species + sex + season + year +
re(random = ~1|code)+ re(random = ~1|station),
family=ZIP(), data=node_dat)
but I get a warning after twenty iterations
Warning message:
In RS() : Algorithm RS has not yet converged
However I can create a summary output
> summary(mDEG_zip)
******************************************************************
Family: c("ZIP", "Poisson Zero Inflated")
Call: gamlss(formula = deg ~ pb(SE_score) + TL + species + sex + season + year + re(random = ~1 | code) +
re(random = ~1 | station), family = ZIP(), data = node_dat, start.from = mDEG_zip, iter = 20, n.cyc = 40)
Fitting method: RS()
------------------------------------------------------------------
Mu link function: log
Mu Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -3.1461720 0.0755180 -41.661 < 2e-16 ***
pb(SE_score) -0.5060934 0.1431689 -3.535 0.000408 ***
TL 0.0037801 0.0005586 6.767 1.32e-11 ***
speciesSilvertip Shark 2.6530209 0.0326096 81.357 < 2e-16 ***
sexM 0.1816634 0.0277136 6.555 5.60e-11 ***
seasonwet.season -0.0020792 0.0271809 -0.076 0.939026
year2015 0.0614232 0.0449014 1.368 0.171330
year2016 0.1322559 0.0390032 3.391 0.000697 ***
year2017 0.0816437 0.0397759 2.053 0.040115 *
year2018 -0.3669929 0.0557062 -6.588 4.48e-11 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
------------------------------------------------------------------
Sigma link function: logit
Sigma Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.28396 0.02217 57.91 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
------------------------------------------------------------------
NOTE: Additive smoothing terms exist in the formulas:
i) Std. Error for smoothers are for the linear effect only.
ii) Std. Error for the linear terms maybe are not accurate.
------------------------------------------------------------------
No. of observations in the fit: 82210
Degrees of Freedom for the fit: 171.7671
Residual Deg. of Freedom: 82038.23
at cycle: 40
Global Deviance: 30763.12
AIC: 31106.66
SBC: 32707.02
******************************************************************
I've tried to use the refit() function but I get the same result after another twenty iterations.
If the model doesn't converge, is this an issue when interpreting the model outputs? A reproducible example dataset is below.
> dput(head(node_dat))
structure(list(station = structure(c(17L, 49L, 23L, 25L, 25L,
9L), .Label = c("BE01", "BE02", "BEUWM01", "BL01", "BL02", "GCB01",
"GCB02", "GCB03", "NI01", "NI01b", "NI03", "PB01", "PB02", "PB03",
"PB04", "PB05", "PB06", "PB07", "PB08", "PB09", "PB10", "PB11",
"PB12", "PB13", "PB14", "PB15", "PB16", "PB17", "PB18", "PB19",
"PB20", "PB21", "PB22", "PB23", "PB24", "PB25", "PB26", "PB27",
"PB28", "PB29", "PB30", "PB4G01", "PB4G02", "PBUWM01", "PBUWM02",
"SA01", "SA02", "SA02b", "SA03", "SA04", "SA05", "SA06", "SA07",
"SA11", "SAUWM01", "SB01", "SB02/AR02", "SB03/AR05", "SB04/AR06",
"VB01", "VB02", "VB03", "VB04"), class = "factor"), monthyear = structure(c(27L,
17L, 38L, 4L, 19L, 29L), .Label = c("2014/01", "2014/02", "2014/03",
"2014/04", "2014/05", "2014/06", "2014/07", "2014/08", "2014/09",
"2014/10", "2014/11", "2014/12", "2015/01", "2015/02", "2015/03",
"2015/04", "2015/05", "2015/06", "2015/07", "2015/08", "2015/09",
"2015/10", "2015/11", "2015/12", "2016/01", "2016/02", "2016/03",
"2016/04", "2016/05", "2016/06", "2016/07", "2016/08", "2016/09",
"2016/10", "2016/11", "2016/12", "2017/01", "2017/02", "2017/03",
"2017/04", "2017/05", "2017/06", "2017/07", "2017/08", "2017/09",
"2017/10", "2017/11", "2017/12", "2018/01", "2018/02", "2018/03",
"2018/04", "2018/05", "2018/06", "2018/07", "2018/08", "2018/09",
"2018/10", "2018/11", "2018/12"), class = "factor"), code = structure(c(99L,
204L, 183L, 146L, 4L, 135L), .Label = c("2390", "13573", "13574",
"13575", "13576", "19318", "19319", "19321", "19322", "19506",
"19514", "19519", "19520", "19524", "25537", "25540", "25541",
"25543", "25546", "25549", "25552", "25553", "27583", "27585",
"27586", "27591", "27592", "27593", "27594", "27595", "27596",
"27597", "27600", "27601", "27605", "27607", "27608", "27613",
"27614", "27617", "27619", "27620", "27621", "27626", "27627",
"27629", "27630", "27631", "27632", "28608", "28611", "28612",
"28618", "28625", "28628", "28629", "28631", "28632", "28633",
"28638", "28641", "28644", "28662", "28672", "28674", "52978",
"54815", "54846", "54852", "54860", "54863", "54865", "54866",
"54868", "54877", "54882", "54883", "54884", "54886", "54890",
"54892", "54895", "54896", "54901", "54904", "54914", "54919",
"54920", "54922", "54925", "54931", "54932", "54938", "54952",
"54954", "54955", "54958", "54959", "54962", "59950", "59953",
"59954", "59955", "59957", "59958", "59959", "59960", "59961",
"59962", "59964", "59966", "59969", "59970", "59971", "59972",
"59973", "59975", "59976", "59979", "59981", "59988", "2388",
"12950", "12952", "12956", "12958", "12960", "12962", "12964",
"12966", "12968", "13577", "14203", "19320", "19523", "25534",
"25535", "25536", "25539", "25542", "25544", "25545", "25547",
"25548", "25550", "27584", "27588", "27589", "27590", "27598",
"27599", "27602", "27603", "27604", "27606", "27609", "27610",
"27611", "27615", "27616", "27618", "27622", "27624", "27625",
"28624", "28627", "28637", "28639", "28642", "28660", "28670",
"34176", "34177", "34178", "34179", "52975", "52977", "54817",
"54821", "54822", "54825", "54845", "54849", "54880", "54887",
"54889", "54893", "54898", "54899", "54905", "54911", "54912",
"54915", "54933", "54947", "54957", "54961", "59951", "59963",
"59968", "59978", "59991", "59992", "59993", "59994", "59995"
), class = "factor"), species = structure(c(1L, 2L, 2L, 2L, 1L,
2L), .Label = c("Grey Reef Shark", "Silvertip Shark"), class = "factor"),
deg = c(0, 0, 0, 0, 0, 0), gs = c(0, 0, 0, 0, 0, 0), btw = c(0,
0, 0, 0, 0, 0), ud = c(0, 0, 0, 0, 0, 0), ri = c(0, 0, 0,
0, 0, 0), SE_score = c(0.35, 0.39, 0.18, 0.23, 0.36, 0.42
), region = structure(c(5L, 6L, 5L, 5L, 5L, 4L), .Label = c("Benares",
"Blenheim", "Grand Chagos Bank", "Nelsons Island", "Peros Banhos",
"Saloman", "Speakers Bank", "Victory Bank"), class = "factor"),
date = structure(c(1456790400, 1430434800, 1485907200, 1396306800,
1435705200, 1462057200), class = c("POSIXct", "POSIXt"), tzone = ""),
month = c(3, 5, 2, 4, 7, 5), season = structure(c(2L, 1L,
2L, 1L, 1L, 1L), .Label = c("dry.season", "wet.season"), class = "factor"),
year = structure(c(3L, 2L, 4L, 1L, 2L, 3L), .Label = c("2014",
"2015", "2016", "2017", "2018"), class = "factor"), sex = structure(c(1L,
2L, 2L, 1L, 1L, 2L), .Label = c("F", "M"), class = "factor"),
TL = c(117, 157, 137, 108, 94, 137), TL_stand = c(0.353383458646617,
0.654135338345865, 0.503759398496241, 0.285714285714286,
0.180451127819549, 0.503759398496241), btw_stand = c(0, 0,
0, 0, 0, 0)), na.action = structure(c(`59` = 59L, `91` = 91L,
`119` = 119L, `144` = 144L, `715` = 715L, `754` = 754L, `780` = 780L,
`803` = 803L, `2116` = 2116L, `2452` = 2452L, `2489` = 2489L,
`2504` = 2504L, `2544` = 2544L, `3070` = 3070L, `3092` = 3092L,
`3126` = 3126L, `3151` = 3151L, `4464` = 4464L, `4800` = 4800L,
`4842` = 4842L, `4862` = 4862L, `4893` = 4893L, `6181` = 6181L,
`8221` = 8221L, `10073` = 10073L, `11232` = 11232L, `11603` = 11603L,
`11639` = 11639L, `11663` = 11663L, `11688` = 11688L, `12266` = 12266L,
`12288` = 12288L, `12322` = 12322L, `12347` = 12347L, `13660` = 13660L,
`14023` = 14023L, `14045` = 14045L, `14075` = 14075L, `14104` = 14104L,
`15417` = 15417L, `15780` = 15780L, `15795` = 15795L, `15837` = 15837L,
`15877` = 15877L, `17138` = 17138L, `17164` = 17164L, `17194` = 17194L,
`17219` = 17219L, `18532` = 18532L, `18895` = 18895L, `18917` = 18917L,
`18951` = 18951L, `18976` = 18976L, `20289` = 20289L, `20652` = 20652L,
`20674` = 20674L, `20704` = 20704L, `20729` = 20729L, `22055` = 22055L,
`22409` = 22409L, `22435` = 22435L, `22461` = 22461L, `22490` = 22490L,
`23803` = 23803L, `24166` = 24166L, `24188` = 24188L, `24218` = 24218L,
`24247` = 24247L, `25560` = 25560L, `25919` = 25919L, `25939` = 25939L,
`25976` = 25976L, `25996` = 25996L, `27308` = 27308L, `27330` = 27330L,
`27360` = 27360L, `27385` = 27385L, `28702` = 28702L, `29065` = 29065L,
`29087` = 29087L, `29121` = 29121L, `29146` = 29146L, `30459` = 30459L,
`30822` = 30822L, `30844` = 30844L, `30874` = 30874L, `30903` = 30903L,
`32216` = 32216L, `32579` = 32579L, `32605` = 32605L, `32631` = 32631L,
`32660` = 32660L, `33973` = 33973L, `34336` = 34336L, `34369` = 34369L,
`34397` = 34397L, `34415` = 34415L, `34875` = 34875L, `34901` = 34901L,
`34931` = 34931L, `34956` = 34956L, `36269` = 36269L, `36632` = 36632L,
`36658` = 36658L, `36684` = 36684L, `36709` = 36709L, `38026` = 38026L,
`38389` = 38389L, `38415` = 38415L, `38441` = 38441L, `38466` = 38466L,
`39783` = 39783L, `40146` = 40146L, `40168` = 40168L, `40198` = 40198L,
`40223` = 40223L, `41540` = 41540L, `41914` = 41914L, `41937` = 41937L,
`41960` = 41960L, `41984` = 41984L, `43297` = 43297L, `43633` = 43633L,
`43675` = 43675L, `43695` = 43695L, `43730` = 43730L, `45014` = 45014L,
`45363` = 45363L, `45400` = 45400L, `45415` = 45415L, `45455` = 45455L,
`45954` = 45954L, `45991` = 45991L, `46009` = 46009L, `46048` = 46048L,
`46541` = 46541L, `46576` = 46576L, `46602` = 46602L, `46627` = 46627L,
`46817` = 46817L, `46859` = 46859L, `46879` = 46879L, `46910` = 46910L,
`48198` = 48198L, `48547` = 48547L, `48589` = 48589L, `48609` = 48609L,
`48640` = 48640L, `49928` = 49928L, `50277` = 50277L, `50319` = 50319L,
`50345` = 50345L, `50370` = 50370L, `51658` = 51658L, `52007` = 52007L,
`52048` = 52048L, `52069` = 52069L, `52100` = 52100L, `53388` = 53388L,
`53737` = 53737L, `53778` = 53778L, `53799` = 53799L, `53830` = 53830L,
`55118` = 55118L, `55467` = 55467L, `55508` = 55508L, `55529` = 55529L,
`55560` = 55560L, `56848` = 56848L, `57197` = 57197L, `57238` = 57238L,
`57264` = 57264L, `57295` = 57295L, `58555` = 58555L, `58596` = 58596L,
`58617` = 58617L, `58648` = 58648L, `59936` = 59936L, `60285` = 60285L,
`60322` = 60322L, `60337` = 60337L, `60377` = 60377L, `60875` = 60875L,
`60905` = 60905L, `60931` = 60931L, `60972` = 60972L, `61441` = 61441L,
`61463` = 61463L, `61497` = 61497L, `61522` = 61522L, `62835` = 62835L,
`63197` = 63197L, `63236` = 63236L, `63260` = 63260L, `63276` = 63276L,
`63793` = 63793L, `64180` = 64180L, `64206` = 64206L, `64232` = 64232L,
`64261` = 64261L, `65574` = 65574L, `65937` = 65937L, `65959` = 65959L,
`65993` = 65993L, `66018` = 66018L, `67331` = 67331L, `67694` = 67694L,
`67716` = 67716L, `67746` = 67746L, `67772` = 67772L, `69088` = 69088L,
`69424` = 69424L, `69466` = 69466L, `69486` = 69486L, `69517` = 69517L,
`70805` = 70805L, `72253` = 72253L, `73419` = 73419L, `73760` = 73760L,
`73802` = 73802L, `73828` = 73828L, `73859` = 73859L, `75141` = 75141L,
`76590` = 76590L, `77752` = 77752L, `78906` = 78906L, `79486` = 79486L,
`79523` = 79523L, `79536` = 79536L, `79556` = 79556L, `80122` = 80122L,
`80159` = 80159L, `80174` = 80174L, `80214` = 80214L, `82125` = 82125L
), class = "omit"), row.names = c(31669L, 80335L, 63799L, 59674L,
1051L, 51949L), class = "data.frame")
You should adjust the number of iterations in numerical algorithm:
mDEG_zip <- gamlss(formula = deg ~ pb(SE_score) + TL + species + sex + season +
year + re(random = ~1|code) + re(random = ~1|station),
family=ZIP(), data = node_dat,
control = gamlss.control(n.cyc = 200))
The parameter n.cyc is 20 by default. I changed it to 200.
You can change the method argument, if you want Rigby and Stasinopoulos Algorithm or Cole and Green, or both, and the number of iteractions. Here is somes examples:
BCCGfixo <- gamlss(Claims1 ~1, family=BCCGo, data = dados_oc, method = RS(500))
You just need to change the argument
method = mixed(50,500)
Here the model uses 50 iteractions of RS and 500 of CG. You can use only CG too
method = CG(100)
Try changing the inicial start values of the parameters, might help. Something like that
mu.start=10, sigma.start=70, nu.start=0.5, tau.start=10
But I must warn you, work with random effects in gamlss is quite hard, and is usually that the model doesn't congerge at all, no matter what you do.
Hope this helps
Related
Alright I have checked the internet, and I am still creating a data frame of replicated lines.
I have a for loop that is creating the welch t-test results, I have saved the values like such:
gene <- biomarkers$Symbol
pval <- ttest$p.value
tstat <- ttest$statistic
I tried to iterate with the for loop to add the results to a data frame which created at the start of the chunk
df2 <- data.frame(pathol=(character()),
genes=character(),
p_value=character(),
t_stat=character(),
stringsAsFactors=FALSE)
for (gene in biomarkers$Symbol) {
print(gene)
ddat <- degs[degs$Symbol== gene & degs$pathol=="mitosis",]
ttest <- t.test(logFC ~ value, data = ddat)
print (ttest)
df2[nrow(df2) + 1,] #(this added the 10 genes only to the rows, not to the column)
then I realised I need to use lapply...which I tried this:
prova <- lapply(biomarkers$Symbol, function(gene) {
append = (gene)
#ttest <- t.test(logFC ~ value, data = ddat)
})
do.call(rbind, prova)
this created a list of the ten genes, however when I uncomment the variable 'ttest', it just adds a list of:
[1,] "logFC by value"
[2,] "logFC by value"
[3,] "logFC by value"
[4,] "logFC by value"
[5,] "logFC by value"
[6,] "logFC by value"
[7,] "logFC by value"
[8,] "logFC by value"
[9,] "logFC by value"
[10,] "logFC by value"
I would like to end up with a data frame that looks like this:
pathol
genes
p_value
t_stat
mitosis
PBK
0.05
000.4
mitosis
PLK4
0.02
000.9
#the gene will be the same for all of the rows.
any help would be great!
EDIT
dput output:
dput(head(ddat))
structure(list(experiment = c("FP001RO_15_HI", "FP001RO_15_HI",
"FP001RO_15_HI", "FP001RO_15_LOW", "FP001RO_15_LOW", "FP001RO_15_LOW"
), Human.Gene.entrezID = c("57405", "57405", "57405", "57405",
"57405", "57405"), Symbol = c("SPC25", "SPC25", "SPC25", "SPC25",
"SPC25", "SPC25"), description = c("SPC25 component of NDC80 kinetochore complex",
"SPC25 component of NDC80 kinetochore complex", "SPC25 component of NDC80 kinetochore complex",
"SPC25 component of NDC80 kinetochore complex", "SPC25 component of NDC80 kinetochore complex",
"SPC25 component of NDC80 kinetochore complex"), score = c(3.867,
3.867, 3.867, 3.867, 3.867, 3.867), type = c("WGGNC", "WGGNC",
"WGGNC", "WGGNC", "WGGNC", "WGGNC"), pathol = c("mitosis", "mitosis",
"mitosis", "mitosis", "mitosis", "mitosis"), Probeid = c("295661_at",
"295661_at", "295661_at", "295661_at", "295661_at", "295661_at"
), logFC = c(-0.0641349806730976, -0.0641349806730976, -0.0641349806730976,
-0.0324566291924587, -0.0324566291924587, -0.0324566291924587
), AveExpr = c(4.1541958195567, 4.1541958195567, 4.1541958195567,
4.17003499529702, 4.17003499529702, 4.17003499529702), t = c(-0.567682269120301,
-0.567682269120301, -0.567682269120301, -0.214562465957216, -0.214562465957216,
-0.214562465957216), P.Value = c(0.580865708246137, 0.580865708246137,
0.580865708246137, 0.833648126277364, 0.833648126277364, 0.833648126277364
), adj.P.Val = c(0.828914361133465, 0.828914361133465, 0.828914361133465,
0.999594589241814, 0.999594589241814, 0.999594589241814), B = c(-6.4683360535952,
-6.4683360535952, -6.4683360535952, -5.45944975240508, -5.45944975240508,
-5.45944975240508), cpd = c("FP001RO", "FP001RO", "FP001RO",
"FP001RO", "FP001RO", "FP001RO"), time = c(15, 15, 15, 15, 15,
15), dose = c("HI", "HI", "HI", "LOW", "LOW", "LOW"), entrezgene_rat = c(295661,
295661, 295661, 295661, 295661, 295661), external_gene_name_rat = c("Spc25",
"Spc25", "Spc25", "Spc25", "Spc25", "Spc25"), external_gene_name_human = c("SPC25",
"SPC25", "SPC25", "SPC25", "SPC25", "SPC25"), entrezGene_probes_human = c("57405_at",
"57405_at", "57405_at", "57405_at", "57405_at", "57405_at"),
inMap_human_withGrey = c(1, 1, 1, 1, 1, 1), inMap_rat_withGrey = c(1,
1, 1, 1, 1, 1), variable = structure(c(11L, 12L, 10L, 10L,
11L, 12L), .Label = c("Necrosis1", "Necrosis2", "Necrosis3",
"hyperpl1", "hyperpl2", "hyperpl3", "fibrosis", "hypertrophy1",
"hypertrophy2", "mitosis1", "mitosis2", "mitosis3", "vacuolation1",
"vacuolation2", "vacuolation3"), class = "factor"), value = structure(c(1L,
1L, 1L, 1L, 1L, 1L), .Label = c("0", "1"), class = "factor"),
pathology = c("mitosis", "mitosis", "mitosis", "mitosis",
"mitosis", "mitosis")), row.names = c(13L, 14L, 15L, 55L,
56L, 57L), class = "data.frame")
dput of gene
dput(biomarkers$Symbol)
c("PBK", "PLK4", "CDKN3", "CDCA3", "PRC1", "CDK1", "TCF19", "SHCBP1",
"CENPK", "SPC25")
We may use
prova <- lapply(biomarkers$Symbol, function(gene) {
ddat <- subset(degs, Symbol == gene & pathol =="mitosis")
fmla <- reformulate('value', response = 'logFC')
if(nlevels(droplevels(ddat$val)) >=2) {
ttest <- t.test(fmla, data = ddat)
data.frame(pathol = 'mitosis', genes = gene,
p_value = ttest$p.value, t_stat = ttest$statistic)
} else NULL
})
names(prova) <- biomarkers$Symbol
out <- do.call(rbind, prova)
Addendum to your question, remember to perform multiple hypothesis testing correction when doing lots of tests at once.
Working off of akrun's answer, here's BH correction:
ord = order(prova$p.value)
prova$fdr = prova$p.value*nrow(prova)/ord
This will reduce the number of false positives.
I have a dataframe dt like below:
dt <- structure(list(Patient = structure(1:5, .Label = c("Sample-3L-AA1B",
"Sample-4N-A93T", "Sample-4T-AA8H", "Sample-5M-AAT4", "Sample-5M-AAT6"), class = "factor"), years = c(1.3013698630137, 0.4, 1.05479452054795,
0.134246575342466, 0.794520547945205), patient_vital_status = c(0L,
0L, 0L, 1L, 1L), `5S_rRNA` = c(0.772491219057, 1.12342309804,
0.283762608812, 0.882492010705, 0.805980084005), `5_8S_rRNA` = c(0,
0, 0, 0, 0), `7SK` = c(0.075067668297, 0, 0, 0.138592705037,
0.210961230042), A1BG = c(0.0282019073358, 0.169523031145, 0.00835845927105,
0.00515484599363, 0.0470792160901)), row.names = c(NA, 5L), class = "data.frame")
I'm trying to apply following code on the required columns and append the each output into a list.
library(survminer)
library(survival)
# vector with the variables to run through
genes <- colnames(dt[4:7])
datalist = data.frame()
for(i in 1:length(genes)){
surv_rnaseq.cut <- surv_cutpoint(
dt,
time = "years",
event = "patient_vital_status",
variables = c(genes[i]))
surv_rnaseq.cat <- surv_categorize(surv_rnaseq.cut)
fit <- survfit(as.formula(paste0("Surv(years, patient_vital_status) ~", genes[i])),
data = surv_rnaseq.cat)
fr <- data.frame(surv_pvalue(fit, surv_rnaseq.cat))
datalist <- rbind(datalist, fr)
}
I got the following error:
Error in parse(text = x, keep.source = FALSE) :
<text>:1:37: unexpected symbol
1: Surv(years, patient_vital_status) ~5S_rRNA
^
I thought may be the error is due to _ symbol in the name. I even removed that and checked but it didn't work.
Below is a subsample of my data set (only 2 rows by 215 columns). I am trying to view them on RStudio but it gives me the following error:
Error in View : undefined columns selected
Do not really know what is going on. The whole set is 7786 rows by 215 columns. Viewing it works fine, however, when doing any kind of subsetting or removing one row it is no longer want to view.
structure(list(`NA` = structure(c(16343, 16344), class = "Date"),
AVON = c("615.5", "621.5"), BA. = c("471.5", "463.2"), CMRG = c("224.5",
"224.5"), COB = c("291.10000000000002", "283.5"), MGGT = c("451.2",
"444.7"), QQ. = c("224.5", "223.5"), RR. = c("953.65", "933.38"
), SNR = c("268.2", "264.7"), ULE = c("1771", "1746"), GKN = c("319.2",
"311.5"), BRAG = c("617", "603"), BVIC = c("668", "661"),
CCH = c("1333", "1327"), DGE = c("1785", "1760.5"), SAB = c("3428",
"3383"), STCK = c("291.60000000000002", "294"), ALNT = c("328",
"321"), CAR = c("125", "124.5"), CRDA = c("2053", "1990"),
ELM = c("255.5", "254.5"), JMAT = c("2919", "2825"), SYNT = c("212",
"210.8"), VCTA = c("1606", "1605"), DIA = c("901", "924"),
DNO = c("611", "611"), E2V = c("161", "160.5"), HLMA = c("612",
"598.5"), HTY = c("309.8", "308"), MGAM = c("296.8", "289.40000000000003"
), OXFD = c("1020", "1035"), RSHW = c("1630", "1625"), SXS = c("1808",
"1778"), TTG = c("166.75", "167.5"), XAR = c("376", "367"
), X = c("1527", "1520"), ABF = c("2679", "2654"), AE = c("633.5",
"640"), CARM = c("1647", "1637"), CWK = c("1328", "1320"),
DCG = c("383.75", "369"), DVO = c("237.75", "231"), GNCL = c("234",
"229.6"), HFG = c("416", "411"), FD = c("36.5", "34.75"),
TATE = c("591.5", "585"), MNDI = c("1011", "1012"), BI = c("616",
"620"), REX = c("491.8", "483.5"), RC = c("559", "540"),
SMDS = c("266.3", "257"), SMIN = c("1264", "1250"), VSVS = c("451.8",
"438.40000000000003"), AGA = c("163.25", "160.25"), BDEV = c("396.1",
"389.3"), BKG = c("2250", "2224"), BLWY = c("1567", "1558"
), BVS = c("779", "771"), CRST = c("325", "314.60000000000002"
), GLSN = c("393.5", "388.5"), MCB = c("83.53", "83.29"),
SN = c("1334", "1309"), RB. = c("5350", "5305"), RDW = c("280.7",
"273.8"), TW. = c("112.8", "111.8"), BODY = c("668.5", "647"
), FENR = c("317.60000000000002", "313.10000000000002"),
GDWN = c("3500", "3500"), HILS = c("561", "561.5"), IMI = c("1230",
"1206"), MRO = c("247.70000000000002", "246"), VAR = c("304",
"300.75"), RNO = c("56", "54.5"), RTRK = c("2765", "2736"
), SFR = c("63.5", "64"), SRX = c("2826", "2812"), TRI = c("105.75",
"105"), VTC = c("613.5", "612"), WEIR = c("2502", "2430"),
EVR = c("130", "123.60000000000001"), FXO = c("112.3", "105.10000000000001"
), BBA = c("325", "326"), BMS = c("494.38", "492"), CKN = c("2350",
"2341"), FSHR = c("1326", "1294"), RMG = c("392.2", "399.7"
), STOB = c("111", "109"), UKM = c("473.88", "467"), WIN = c("136.25",
"137.5"), GAW = c("597.5", "585"), HTM = c("131.5", "129.25"
), `NA` = c(NA_character_, NA_character_), AAL = c("1384",
"1363.5"), ABG = c("218.8", "209.1"), ANTO = c("721", "702"
), AF = c("131.5", "130.25"), AQ = c("18.5", "18.75"), ARMS = c("69",
"62.25"), BLT = c("1715", "1690.5"), CEY = c("61.15", "61"
), FRES = c("760", "747"), GEMD = c("192", "191.75"), GLEN = c("343.2",
"336.45"), HOC = c("135.30000000000001", "130.19999999999999"
), KAZ = c("263.39999999999998", "260.10000000000002"), KMRL = c("9.5",
"9.3000000000000007"), LMI = c("185.8", "176.8"), NWR = c("1.97",
"1.82"), `NA` = c(NA_character_, NA_character_), DL = c("190.20000000000002",
"190"), OG = c("22", "24"), OLY = c("516", "496.6"), RIO = c("3031.5",
"3020"), RRS = c("4209", "4154"), VED = c("998.5", "974.5"
), AFR = c("103.5", "109.4"), BG. = c("1140", "1093"), B. = c("453.45",
"452.75"), CNE = c("176.5", "171.6"), ENQ = c("109.60000000000001",
"107.8"), EXI = c("157", "150"), HDY = c("102", "99.75"),
JKX = c("48.25", "47"), OHR = c("229.3", "220.9"), MO = c("333",
"324.7"), RDSA = c("2358.5", "2331"), RDSB = c("2437", "2418.5"
), SIA = c("381", "377.90000000000003"), SMDR = c("100",
"98.5"), TLW = c("644.5", "631"), AMEC = c("1104", "1077"
), CIU = c("283.5", "275.75"), GMS = c("157", "157"), HTG = c("892.5",
"876"), LAM = c("163.25", "160"), FC = c("1037", "1011"),
WG. = c("759.5", "743"), BRBY = c("1511", "1476"), ZC = c("365.7",
"366"), SG = c("1133", "1126"), TED = c("1863", "1862"),
ULVR = c("2585", "2547"), AZN = c("4441.5", "4360.5"), BTG = c("700",
"697.5"), CIR = c("304", "300"), DH = c("758", "753"), GNS = c("1130",
"1130"), GSK = c("1413", "1414"), HIK = c("1733", "1715"),
SH = c("5340", "5310"), SK = c("329.25", "319"), VEC = c("132",
"132"), AGK = c("1548", "1528"), AHT = c("1043", "1024"),
ATK = c("1317", "1323"), BAB = c("1092", "1085"), BNZL = c("1610",
"1597"), BRAM = c("376", "374"), BRSN = c("980", "979"),
CLLN = c("304.60000000000002", "304.3"), CMS = c("59.75",
"59.5"), CNCT = c("149.25", "151"), CI = c("1164", "1165"
), CTR = c("259.5", "255"), DCC = c("3422", "3405"), DLAR = c("477",
"478"), DLM = c("689.5", "685"), ECOM = c("223", "219.8"),
ESNT = c("797.5", "792.5"), EXO = c("176.5", "180"), EXN = c("983.5",
"968"), GFS = c("250.70000000000002", "251.6"), GFTU = c("626",
"616"), HAS = c("116.3", "115.7"), HRG = c("45.75", "45.75"
), HSV = c("319.7", "319"), HWDN = c("339.1", "335"), HYC = c("749",
"748"), IRV = c("599.5", "592.5"), ITRK = c("2621", "2631"
), LVD = c("201.75", "201.5"), MER = c("435", "436.75"),
MMC = c("25.25", "25"), MNZS = c("569", "575.5"), MI = c("418.6",
"421"), MTO = c("287.90000000000003", "286.60000000000002"
), NTG = c("483.8", "481.3"), AY = c("983.5", "989"), FL = c("182",
"180.1"), RCDO = c("671", "667.5"), RENT = c("117.8", "116"
), RGU = c("169.70000000000002", "169.9"), RS = c("261",
"251.6"), RWA = c("302.5", "302.5"), SDY = c("70.5", "69.75"
), SERC = c("286.10000000000002", "279.8"), SHI = c("166.6",
"161.1"), SIV = c("199.75", "200"), SKS = c("90", "92"),
STHR = c("350.25", "358.5"), TK = c("1664", "1635"), TRB = c("170.5",
"172"), V. = c("609.5", "600"), WOS = c("3242", "3243"),
XCH = c("188", "184.75"), ARM = c("906", "887.5"), BVC = c("16.38",
"16.25"), CSR = c("758", "756"), IMG = c("188.5", "184.75"
), LRD = c("309.7", "306.7"), IC = c("298.10000000000002",
"299"), SEU = c("141", "141"), ST = c("104.60000000000001",
"99.9"), BATS = c("3482", "3480"), IMT = c("2664", "2679"
)), .Names = c("NA", "AVON", "BA.", "CMRG", "COB", "MGGT",
"QQ.", "RR.", "SNR", "ULE", "GKN", "BRAG", "BVIC", "CCH", "DGE",
"SAB", "STCK", "ALNT", "CAR", "CRDA", "ELM", "JMAT", "SYNT",
"VCTA", "DIA", "DNO", "E2V", "HLMA", "HTY", "MGAM", "OXFD", "RSHW",
"SXS", "TTG", "XAR", "X", "ABF", "AE", "CARM", "CWK", "DCG",
"DVO", "GNCL", "HFG", "FD", "TATE", "MNDI", "BI", "REX", "RC",
"SMDS", "SMIN", "VSVS", "AGA", "BDEV", "BKG", "BLWY", "BVS",
"CRST", "GLSN", "MCB", "SN", "RB.", "RDW", "TW.", "BODY", "FENR",
"GDWN", "HILS", "IMI", "MRO", "VAR", "RNO", "RTRK", "SFR", "SRX",
"TRI", "VTC", "WEIR", "EVR", "FXO", "BBA", "BMS", "CKN", "FSHR",
"RMG", "STOB", "UKM", "WIN", "GAW", "HTM", NA, "AAL", "ABG",
"ANTO", "AF", "AQ", "ARMS", "BLT", "CEY", "FRES", "GEMD", "GLEN",
"HOC", "KAZ", "KMRL", "LMI", "NWR", NA, "DL", "OG", "OLY", "RIO",
"RRS", "VED", "AFR", "BG.", "B.", "CNE", "ENQ", "EXI", "HDY",
"JKX", "OHR", "MO", "RDSA", "RDSB", "SIA", "SMDR", "TLW", "AMEC",
"CIU", "GMS", "HTG", "LAM", "FC", "WG.", "BRBY", "ZC", "SG",
"TED", "ULVR", "AZN", "BTG", "CIR", "DH", "GNS", "GSK", "HIK",
"SH", "SK", "VEC", "AGK", "AHT", "ATK", "BAB", "BNZL", "BRAM",
"BRSN", "CLLN", "CMS", "CNCT", "CI", "CTR", "DCC", "DLAR", "DLM",
"ECOM", "ESNT", "EXO", "EXN", "GFS", "GFTU", "HAS", "HRG", "HSV",
"HWDN", "HYC", "IRV", "ITRK", "LVD", "MER", "MMC", "MNZS", "MI",
"MTO", "NTG", "AY", "FL", "RCDO", "RENT", "RGU", "RS", "RWA",
"SDY", "SERC", "SHI", "SIV", "SKS", "STHR", "TK", "TRB", "V.",
"WOS", "XCH", "ARM", "BVC", "CSR", "IMG", "LRD", "IC", "SEU",
"ST", "BATS", "IMT"), row.names = 7785:7786, class = "data.frame")
I am on Mac OS 10.10, R 3.1.1 and RStudio 0.98.1060.
One of your column names is NA. If d is your data defined above, then try names(d)[92]. Try replacing with a non-missing column name.
As allready mentioned by DMC, but with a short version of your example code.
a <- structure(list(`NA` = structure(c(16343, 16344), class = "Date"),
AVON = c("615.5", "621.5"),
BA. = c("471.5", "463.2"),
`NA` = c(NA_character_, NA_character_), AAL = c("1384", "1363.5")),
.Names = c(NA, "AVON", "BA.", "NA", "AAL"), row.names = 7785:7786, class = "data.frame")
View(a)
Error in View : undefined columns selected
names(a)
[1] NA "AVON" "BA." "NA" "AAL"
a <- structure(list(`NA` = structure(c(16343, 16344), class = "Date"),
AVON = c("615.5", "621.5"),
BA. = c("471.5", "463.2"),
`NA` = c(NA_character_, NA_character_), AAL = c("1384", "1363.5")),
.Names = c("NA", "AVON", "BA.", "NA", "AAL"), row.names = 7785:7786, class = "data.frame")
View(a)
names(a)
[1] "NA" "AVON" "BA." "NA" "AAL"
You need to have proper names in the data frame to View it.
Code:
library(caret)
#adaptative control resampling method for fitting svr
ctrlada <- trainControl(method = "adaptive_cv", number = 10, returnResamp = "final",
adaptive = list(min = 5,
alpha = 0.05,
method = "gls",
complete = TRUE),
allowParallel = TRUE) #10 separate 10-fold cross-validations are used as the resampling scheme
set.seed(100)
marsFitacv <- train(R ~ ., data = trainSet,
method = "earth",
tuneGrid = expand.grid(degree = 2, nprune = 40:80),
trControl = ctrlada)
error:
x parameter filtering failed
Error in `$<-.data.frame`(`*tmp*`, "nprune", value = NA) :
replacement has 1 row, data has 0
data:
dput(head(trainSet))
structure(list(fy = c(317.913756282, 365.006253069, 392.548100067,
305.350697829, 404.999341917, 326.558279739), fu = c(538.962896683,
484.423120589, 607.974981919, 566.461909098, 580.287855801, 454.178316794
), E = c(194617.707566, 181322.455065, 206661.286272, 182492.029532,
189867.929239, 181991.379749), eu = c(0.153782620813, 0.208857408687,
0.29933255604, 0.277013319499, 0.251278125174, 0.20012525805),
imp_local = c(1555.3450957, 1595.41614044, 763.56392418,
1716.78277731, 1045.72429616, 802.742305814), imp_global = c(594.038972858,
1359.48216529, 1018.89209367, 850.887850177, 1381.3557372,
1714.66351462), teta1c = c(0.033375064111, 0.021482368218,
0.020905367537, 0.006956337817, 0.034913536977, 0.03009770223
), k1c = c(4000921.55552, 4499908.41979, 9764999.26902, 9273400.46159,
6163057.88855, 12338543.5703), k2_2L = c(98633499.5682, 53562216.5496,
51597126.6866, 79496746.0098, 54060378.6334, 88854286.5457
), k2_3L = c(53752551.0262, 125020222.794, 124021434.482,
125817803.431, 75021821.6702, 35160224.288), k2_4L = c(56725106.5978,
126865701.893, 145764489.664, 64837586.8755, 49128911.0832,
70088564.0166), bmaxc = c(3481281.32908, 4393584.00639, 2614830.02391,
3128593.72039, 3179348.29527, 4274637.35956), dfactorc = c(2.5474729895,
2.94296926288, 2.79505551368, 2.47882735165, 2.46407943564,
1.41121223341), amaxc = c(73832.9746763, 99150.5068997, 77165.4338508,
128546.996471, 53819.0447533, 54870.9707106), teta1s = c(0.015467320192,
0.013675755546, 0.031668366149, 0.028898297322, 0.019211801086,
0.013349768955), k1s = c(5049506.54552, 11250622.6842, 13852560.5089,
18813117.5726, 18362782.7372, 14720875.0829), k2_ab1s = c(276542468.441,
275768806.723, 211613299.608, 264475187.749, 162043062.526,
252936228.465), k2_ab2s = c(108971516.033, 114017918.32,
248886114.151, 213529935.615, 236891513.077, 142986118.909
), k2_ab3s = c(33306211.9166, 28220338.4744, 40462423.2281,
23450400.4429, 46044346.1128, 23695405.2598), bmaxab1 = c(4763935.86742,
4297372.01966, 3752983.00638, 4861240.46459, 4269771.8481,
4162098.23435), bmaxab2 = c(1864128.647, 1789714.6047, 2838412.50704,
2122535.96812, 2512362.60884, 1176995.61871), ab1 = c(66.4926766666,
42.7771212442, 45.4212664748, 50.3764074404, 35.4792060556,
34.1116517971), ab2 = c(21.0285105309, 23.5869838719, 18.8524808986,
10.1121885612, 10.9695055644, 12.1154127169), dfactors = c(2.47803921947,
0.874644748155, 0.749837099991, 1.96711589185, 2.5407774352,
1.28554379333), teta1f = c(0.037308451805, 0.035718600749,
0.012495093438, 0.000815957999, 0.002155991091, 0.02579104469
), k1f = c(14790480.9871, 17223538.1853, 19930679.8931, 3524230.46974,
15721827.0137, 13599317.0371), k2f = c(55614283.976, 54695745.7762,
86690362.7036, 99857853.7312, 63119072.711, 37510791.5472
), bmaxf = c(2094770.19484, 3633133.51482, 1361188.05421,
2001027.51219, 2534273.6726, 3765850.14143), dfactorf = c(0.745459795314,
2.04869176933, 0.853221909609, 1.76652410119, 0.523675021418,
1.0808768613), k2b = c(1956.92858062, 1400.78738327, 1771.23607857,
1104.05501369, 1756.6767193, 1509.9294956), amaxb = c(38588.0915097,
35158.1672213, 25711.062782, 21103.1603387, 27230.6973685,
43720.3558889999), dfactorb = c(0.822346959126, 2.34421354848,
0.79990635332, 2.99070447299, 1.76373031599, 1.38640223249
), roti = c(16.1560390049, 12.7223971386, 6.43238062144,
15.882552267, 16.0836252663, 18.2734832893), rotmaxbp = c(0.235615453341,
0.343204895932, 0.370304533553, 0.488746319999, 0.176135112774,
0.46921999001), R = c(0.022186087, 0.023768855, 0.023911029,
0.023935705, 0.023655335, 0.022402726)), .Names = c("fy",
"fu", "E", "eu", "imp_local", "imp_global", "teta1c", "k1c",
"k2_2L", "k2_3L", "k2_4L", "bmaxc", "dfactorc", "amaxc", "teta1s",
"k1s", "k2_ab1s", "k2_ab2s", "k2_ab3s", "bmaxab1", "bmaxab2",
"ab1", "ab2", "dfactors", "teta1f", "k1f", "k2f", "bmaxf", "dfactorf",
"k2b", "amaxb", "dfactorb", "roti", "rotmaxbp", "R"), row.names = c(7L,
8L, 20L, 23L, 28L, 29L), class = "data.frame")
Data has no equal rows or NaNs
Code:
library(nnet)
library(caret)
#K-folds resampling method for fitting model
ctrl <- trainControl(method = "repeatedcv", number = 10, repeats = 10,
allowParallel = TRUE) #10 separate 10-fold cross-validations
nnetGrid <- expand.grid(decay = seq(0.0002, .0008, length = 4),
size = seq(6, 10, by = 2),
bag = FALSE)
set.seed(100)
nnetFitcv <- train(R ~ .,
data = trainSet,
method = "avNNet",
tuneGrid = nnetGrid,
trControl = ctrl,
preProc = c("center", "scale"),
linout = TRUE,
## Reduce the amount of printed output
trace = FALSE,
## Expand the number of iterations to find
## parameter estimates..
maxit = 2000,
## and the number of parameters used by the model
MaxNWts = 5 * (34 + 1) + 5 + 1)
Error:
Error in train.default(x, y, weights = w, ...) :
final tuning parameters could not be determined
In addition: Warning messages:
1: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, :
There were missing values in resampled performance measures.
2: In train.default(x, y, weights = w, ...) :
missing values found in aggregated results
data:
dput(head(trainSet))
structure(list(fy = c(317.913756282, 365.006253069, 392.548100067,
305.350697829, 404.999341917, 326.558279739), fu = c(538.962896683,
484.423120589, 607.974981919, 566.461909098, 580.287855801, 454.178316794
), E = c(194617.707566, 181322.455065, 206661.286272, 182492.029532,
189867.929239, 181991.379749), eu = c(0.153782620813, 0.208857408687,
0.29933255604, 0.277013319499, 0.251278125174, 0.20012525805),
imp_local = c(1555.3450957, 1595.41614044, 763.56392418,
1716.78277731, 1045.72429616, 802.742305814), imp_global = c(594.038972858,
1359.48216529, 1018.89209367, 850.887850177, 1381.3557372,
1714.66351462), teta1c = c(0.033375064111, 0.021482368218,
0.020905367537, 0.006956337817, 0.034913536977, 0.03009770223
), k1c = c(4000921.55552, 4499908.41979, 9764999.26902, 9273400.46159,
6163057.88855, 12338543.5703), k2_2L = c(98633499.5682, 53562216.5496,
51597126.6866, 79496746.0098, 54060378.6334, 88854286.5457
), k2_3L = c(53752551.0262, 125020222.794, 124021434.482,
125817803.431, 75021821.6702, 35160224.288), k2_4L = c(56725106.5978,
126865701.893, 145764489.664, 64837586.8755, 49128911.0832,
70088564.0166), bmaxc = c(3481281.32908, 4393584.00639, 2614830.02391,
3128593.72039, 3179348.29527, 4274637.35956), dfactorc = c(2.5474729895,
2.94296926288, 2.79505551368, 2.47882735165, 2.46407943564,
1.41121223341), amaxc = c(73832.9746763, 99150.5068997, 77165.4338508,
128546.996471, 53819.0447533, 54870.9707106), teta1s = c(0.015467320192,
0.013675755546, 0.031668366149, 0.028898297322, 0.019211801086,
0.013349768955), k1s = c(5049506.54552, 11250622.6842, 13852560.5089,
18813117.5726, 18362782.7372, 14720875.0829), k2_ab1s = c(276542468.441,
275768806.723, 211613299.608, 264475187.749, 162043062.526,
252936228.465), k2_ab2s = c(108971516.033, 114017918.32,
248886114.151, 213529935.615, 236891513.077, 142986118.909
), k2_ab3s = c(33306211.9166, 28220338.4744, 40462423.2281,
23450400.4429, 46044346.1128, 23695405.2598), bmaxab1 = c(4763935.86742,
4297372.01966, 3752983.00638, 4861240.46459, 4269771.8481,
4162098.23435), bmaxab2 = c(1864128.647, 1789714.6047, 2838412.50704,
2122535.96812, 2512362.60884, 1176995.61871), ab1 = c(66.4926766666,
42.7771212442, 45.4212664748, 50.3764074404, 35.4792060556,
34.1116517971), ab2 = c(21.0285105309, 23.5869838719, 18.8524808986,
10.1121885612, 10.9695055644, 12.1154127169), dfactors = c(2.47803921947,
0.874644748155, 0.749837099991, 1.96711589185, 2.5407774352,
1.28554379333), teta1f = c(0.037308451805, 0.035718600749,
0.012495093438, 0.000815957999, 0.002155991091, 0.02579104469
), k1f = c(14790480.9871, 17223538.1853, 19930679.8931, 3524230.46974,
15721827.0137, 13599317.0371), k2f = c(55614283.976, 54695745.7762,
86690362.7036, 99857853.7312, 63119072.711, 37510791.5472
), bmaxf = c(2094770.19484, 3633133.51482, 1361188.05421,
2001027.51219, 2534273.6726, 3765850.14143), dfactorf = c(0.745459795314,
2.04869176933, 0.853221909609, 1.76652410119, 0.523675021418,
1.0808768613), k2b = c(1956.92858062, 1400.78738327, 1771.23607857,
1104.05501369, 1756.6767193, 1509.9294956), amaxb = c(38588.0915097,
35158.1672213, 25711.062782, 21103.1603387, 27230.6973685,
43720.3558889999), dfactorb = c(0.822346959126, 2.34421354848,
0.79990635332, 2.99070447299, 1.76373031599, 1.38640223249
), roti = c(16.1560390049, 12.7223971386, 6.43238062144,
15.882552267, 16.0836252663, 18.2734832893), rotmaxbp = c(0.235615453341,
0.343204895932, 0.370304533553, 0.488746319999, 0.176135112774,
0.46921999001), R = c(0.022186087, 0.023768855, 0.023911029,
0.023935705, 0.023655335, 0.022402726)), .Names = c("fy",
"fu", "E", "eu", "imp_local", "imp_global", "teta1c", "k1c",
"k2_2L", "k2_3L", "k2_4L", "bmaxc", "dfactorc", "amaxc", "teta1s",
"k1s", "k2_ab1s", "k2_ab2s", "k2_ab3s", "bmaxab1", "bmaxab2",
"ab1", "ab2", "dfactors", "teta1f", "k1f", "k2f", "bmaxf", "dfactorf",
"k2b", "amaxb", "dfactorb", "roti", "rotmaxbp", "R"), row.names = c(7L,
8L, 20L, 23L, 28L, 29L), class = "data.frame")
data has no equal rows or zero values or NaNs. Any help is appreciated.
I guess the problem is caused by MaxNWts, which is The maximum allowable number of weights. The value you gave is less than the weights for networks with size larger than 5 units. It should be at least:
MaxNWts = max(nnetGrid$size)*(ncol(trainSet) + output_neron)
+ max(nnetGrid$size) + output_neron
So, in your case, it should be at least MaxNWts = 10 * (34 + 1) + 10 + 1