Related
I'm running a generalised additive mixed model using the gamlss() function. I used the fitDist() on my data and it recommended I used a zero inflated poisson. My response variable is 'deg' and is count data but has a lot of zeros.
fitDEG <- fitDist(deg, data=node_dat, k = 2, type = "counts", try.gamlss = TRUE)
> fitDEG
Family: c("ZIP", "Poisson Zero Inflated")
Fitting method: "nlminb"
Call: gamlssML(formula = y, family = DIST[i])
Mu Coefficients:
[1] 0.3803
Sigma Coefficients:
[1] 2.81
Degrees of Freedom for the fit: 2 Residual Deg. of Freedom 82208
Global Deviance: 38484.9
AIC: 38488.9
SBC: 38507.6
I've tried running a model with a single smoothed term, one numerical explanatory variable (TL), four categorical explanatory variables and two random effects.
mDEG_zip <- gamlss(formula = deg ~ pb(SE_score) + TL + species + sex + season + year +
re(random = ~1|code)+ re(random = ~1|station),
family=ZIP(), data=node_dat)
but I get a warning after twenty iterations
Warning message:
In RS() : Algorithm RS has not yet converged
However I can create a summary output
> summary(mDEG_zip)
******************************************************************
Family: c("ZIP", "Poisson Zero Inflated")
Call: gamlss(formula = deg ~ pb(SE_score) + TL + species + sex + season + year + re(random = ~1 | code) +
re(random = ~1 | station), family = ZIP(), data = node_dat, start.from = mDEG_zip, iter = 20, n.cyc = 40)
Fitting method: RS()
------------------------------------------------------------------
Mu link function: log
Mu Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -3.1461720 0.0755180 -41.661 < 2e-16 ***
pb(SE_score) -0.5060934 0.1431689 -3.535 0.000408 ***
TL 0.0037801 0.0005586 6.767 1.32e-11 ***
speciesSilvertip Shark 2.6530209 0.0326096 81.357 < 2e-16 ***
sexM 0.1816634 0.0277136 6.555 5.60e-11 ***
seasonwet.season -0.0020792 0.0271809 -0.076 0.939026
year2015 0.0614232 0.0449014 1.368 0.171330
year2016 0.1322559 0.0390032 3.391 0.000697 ***
year2017 0.0816437 0.0397759 2.053 0.040115 *
year2018 -0.3669929 0.0557062 -6.588 4.48e-11 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
------------------------------------------------------------------
Sigma link function: logit
Sigma Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.28396 0.02217 57.91 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
------------------------------------------------------------------
NOTE: Additive smoothing terms exist in the formulas:
i) Std. Error for smoothers are for the linear effect only.
ii) Std. Error for the linear terms maybe are not accurate.
------------------------------------------------------------------
No. of observations in the fit: 82210
Degrees of Freedom for the fit: 171.7671
Residual Deg. of Freedom: 82038.23
at cycle: 40
Global Deviance: 30763.12
AIC: 31106.66
SBC: 32707.02
******************************************************************
I've tried to use the refit() function but I get the same result after another twenty iterations.
If the model doesn't converge, is this an issue when interpreting the model outputs? A reproducible example dataset is below.
> dput(head(node_dat))
structure(list(station = structure(c(17L, 49L, 23L, 25L, 25L,
9L), .Label = c("BE01", "BE02", "BEUWM01", "BL01", "BL02", "GCB01",
"GCB02", "GCB03", "NI01", "NI01b", "NI03", "PB01", "PB02", "PB03",
"PB04", "PB05", "PB06", "PB07", "PB08", "PB09", "PB10", "PB11",
"PB12", "PB13", "PB14", "PB15", "PB16", "PB17", "PB18", "PB19",
"PB20", "PB21", "PB22", "PB23", "PB24", "PB25", "PB26", "PB27",
"PB28", "PB29", "PB30", "PB4G01", "PB4G02", "PBUWM01", "PBUWM02",
"SA01", "SA02", "SA02b", "SA03", "SA04", "SA05", "SA06", "SA07",
"SA11", "SAUWM01", "SB01", "SB02/AR02", "SB03/AR05", "SB04/AR06",
"VB01", "VB02", "VB03", "VB04"), class = "factor"), monthyear = structure(c(27L,
17L, 38L, 4L, 19L, 29L), .Label = c("2014/01", "2014/02", "2014/03",
"2014/04", "2014/05", "2014/06", "2014/07", "2014/08", "2014/09",
"2014/10", "2014/11", "2014/12", "2015/01", "2015/02", "2015/03",
"2015/04", "2015/05", "2015/06", "2015/07", "2015/08", "2015/09",
"2015/10", "2015/11", "2015/12", "2016/01", "2016/02", "2016/03",
"2016/04", "2016/05", "2016/06", "2016/07", "2016/08", "2016/09",
"2016/10", "2016/11", "2016/12", "2017/01", "2017/02", "2017/03",
"2017/04", "2017/05", "2017/06", "2017/07", "2017/08", "2017/09",
"2017/10", "2017/11", "2017/12", "2018/01", "2018/02", "2018/03",
"2018/04", "2018/05", "2018/06", "2018/07", "2018/08", "2018/09",
"2018/10", "2018/11", "2018/12"), class = "factor"), code = structure(c(99L,
204L, 183L, 146L, 4L, 135L), .Label = c("2390", "13573", "13574",
"13575", "13576", "19318", "19319", "19321", "19322", "19506",
"19514", "19519", "19520", "19524", "25537", "25540", "25541",
"25543", "25546", "25549", "25552", "25553", "27583", "27585",
"27586", "27591", "27592", "27593", "27594", "27595", "27596",
"27597", "27600", "27601", "27605", "27607", "27608", "27613",
"27614", "27617", "27619", "27620", "27621", "27626", "27627",
"27629", "27630", "27631", "27632", "28608", "28611", "28612",
"28618", "28625", "28628", "28629", "28631", "28632", "28633",
"28638", "28641", "28644", "28662", "28672", "28674", "52978",
"54815", "54846", "54852", "54860", "54863", "54865", "54866",
"54868", "54877", "54882", "54883", "54884", "54886", "54890",
"54892", "54895", "54896", "54901", "54904", "54914", "54919",
"54920", "54922", "54925", "54931", "54932", "54938", "54952",
"54954", "54955", "54958", "54959", "54962", "59950", "59953",
"59954", "59955", "59957", "59958", "59959", "59960", "59961",
"59962", "59964", "59966", "59969", "59970", "59971", "59972",
"59973", "59975", "59976", "59979", "59981", "59988", "2388",
"12950", "12952", "12956", "12958", "12960", "12962", "12964",
"12966", "12968", "13577", "14203", "19320", "19523", "25534",
"25535", "25536", "25539", "25542", "25544", "25545", "25547",
"25548", "25550", "27584", "27588", "27589", "27590", "27598",
"27599", "27602", "27603", "27604", "27606", "27609", "27610",
"27611", "27615", "27616", "27618", "27622", "27624", "27625",
"28624", "28627", "28637", "28639", "28642", "28660", "28670",
"34176", "34177", "34178", "34179", "52975", "52977", "54817",
"54821", "54822", "54825", "54845", "54849", "54880", "54887",
"54889", "54893", "54898", "54899", "54905", "54911", "54912",
"54915", "54933", "54947", "54957", "54961", "59951", "59963",
"59968", "59978", "59991", "59992", "59993", "59994", "59995"
), class = "factor"), species = structure(c(1L, 2L, 2L, 2L, 1L,
2L), .Label = c("Grey Reef Shark", "Silvertip Shark"), class = "factor"),
deg = c(0, 0, 0, 0, 0, 0), gs = c(0, 0, 0, 0, 0, 0), btw = c(0,
0, 0, 0, 0, 0), ud = c(0, 0, 0, 0, 0, 0), ri = c(0, 0, 0,
0, 0, 0), SE_score = c(0.35, 0.39, 0.18, 0.23, 0.36, 0.42
), region = structure(c(5L, 6L, 5L, 5L, 5L, 4L), .Label = c("Benares",
"Blenheim", "Grand Chagos Bank", "Nelsons Island", "Peros Banhos",
"Saloman", "Speakers Bank", "Victory Bank"), class = "factor"),
date = structure(c(1456790400, 1430434800, 1485907200, 1396306800,
1435705200, 1462057200), class = c("POSIXct", "POSIXt"), tzone = ""),
month = c(3, 5, 2, 4, 7, 5), season = structure(c(2L, 1L,
2L, 1L, 1L, 1L), .Label = c("dry.season", "wet.season"), class = "factor"),
year = structure(c(3L, 2L, 4L, 1L, 2L, 3L), .Label = c("2014",
"2015", "2016", "2017", "2018"), class = "factor"), sex = structure(c(1L,
2L, 2L, 1L, 1L, 2L), .Label = c("F", "M"), class = "factor"),
TL = c(117, 157, 137, 108, 94, 137), TL_stand = c(0.353383458646617,
0.654135338345865, 0.503759398496241, 0.285714285714286,
0.180451127819549, 0.503759398496241), btw_stand = c(0, 0,
0, 0, 0, 0)), na.action = structure(c(`59` = 59L, `91` = 91L,
`119` = 119L, `144` = 144L, `715` = 715L, `754` = 754L, `780` = 780L,
`803` = 803L, `2116` = 2116L, `2452` = 2452L, `2489` = 2489L,
`2504` = 2504L, `2544` = 2544L, `3070` = 3070L, `3092` = 3092L,
`3126` = 3126L, `3151` = 3151L, `4464` = 4464L, `4800` = 4800L,
`4842` = 4842L, `4862` = 4862L, `4893` = 4893L, `6181` = 6181L,
`8221` = 8221L, `10073` = 10073L, `11232` = 11232L, `11603` = 11603L,
`11639` = 11639L, `11663` = 11663L, `11688` = 11688L, `12266` = 12266L,
`12288` = 12288L, `12322` = 12322L, `12347` = 12347L, `13660` = 13660L,
`14023` = 14023L, `14045` = 14045L, `14075` = 14075L, `14104` = 14104L,
`15417` = 15417L, `15780` = 15780L, `15795` = 15795L, `15837` = 15837L,
`15877` = 15877L, `17138` = 17138L, `17164` = 17164L, `17194` = 17194L,
`17219` = 17219L, `18532` = 18532L, `18895` = 18895L, `18917` = 18917L,
`18951` = 18951L, `18976` = 18976L, `20289` = 20289L, `20652` = 20652L,
`20674` = 20674L, `20704` = 20704L, `20729` = 20729L, `22055` = 22055L,
`22409` = 22409L, `22435` = 22435L, `22461` = 22461L, `22490` = 22490L,
`23803` = 23803L, `24166` = 24166L, `24188` = 24188L, `24218` = 24218L,
`24247` = 24247L, `25560` = 25560L, `25919` = 25919L, `25939` = 25939L,
`25976` = 25976L, `25996` = 25996L, `27308` = 27308L, `27330` = 27330L,
`27360` = 27360L, `27385` = 27385L, `28702` = 28702L, `29065` = 29065L,
`29087` = 29087L, `29121` = 29121L, `29146` = 29146L, `30459` = 30459L,
`30822` = 30822L, `30844` = 30844L, `30874` = 30874L, `30903` = 30903L,
`32216` = 32216L, `32579` = 32579L, `32605` = 32605L, `32631` = 32631L,
`32660` = 32660L, `33973` = 33973L, `34336` = 34336L, `34369` = 34369L,
`34397` = 34397L, `34415` = 34415L, `34875` = 34875L, `34901` = 34901L,
`34931` = 34931L, `34956` = 34956L, `36269` = 36269L, `36632` = 36632L,
`36658` = 36658L, `36684` = 36684L, `36709` = 36709L, `38026` = 38026L,
`38389` = 38389L, `38415` = 38415L, `38441` = 38441L, `38466` = 38466L,
`39783` = 39783L, `40146` = 40146L, `40168` = 40168L, `40198` = 40198L,
`40223` = 40223L, `41540` = 41540L, `41914` = 41914L, `41937` = 41937L,
`41960` = 41960L, `41984` = 41984L, `43297` = 43297L, `43633` = 43633L,
`43675` = 43675L, `43695` = 43695L, `43730` = 43730L, `45014` = 45014L,
`45363` = 45363L, `45400` = 45400L, `45415` = 45415L, `45455` = 45455L,
`45954` = 45954L, `45991` = 45991L, `46009` = 46009L, `46048` = 46048L,
`46541` = 46541L, `46576` = 46576L, `46602` = 46602L, `46627` = 46627L,
`46817` = 46817L, `46859` = 46859L, `46879` = 46879L, `46910` = 46910L,
`48198` = 48198L, `48547` = 48547L, `48589` = 48589L, `48609` = 48609L,
`48640` = 48640L, `49928` = 49928L, `50277` = 50277L, `50319` = 50319L,
`50345` = 50345L, `50370` = 50370L, `51658` = 51658L, `52007` = 52007L,
`52048` = 52048L, `52069` = 52069L, `52100` = 52100L, `53388` = 53388L,
`53737` = 53737L, `53778` = 53778L, `53799` = 53799L, `53830` = 53830L,
`55118` = 55118L, `55467` = 55467L, `55508` = 55508L, `55529` = 55529L,
`55560` = 55560L, `56848` = 56848L, `57197` = 57197L, `57238` = 57238L,
`57264` = 57264L, `57295` = 57295L, `58555` = 58555L, `58596` = 58596L,
`58617` = 58617L, `58648` = 58648L, `59936` = 59936L, `60285` = 60285L,
`60322` = 60322L, `60337` = 60337L, `60377` = 60377L, `60875` = 60875L,
`60905` = 60905L, `60931` = 60931L, `60972` = 60972L, `61441` = 61441L,
`61463` = 61463L, `61497` = 61497L, `61522` = 61522L, `62835` = 62835L,
`63197` = 63197L, `63236` = 63236L, `63260` = 63260L, `63276` = 63276L,
`63793` = 63793L, `64180` = 64180L, `64206` = 64206L, `64232` = 64232L,
`64261` = 64261L, `65574` = 65574L, `65937` = 65937L, `65959` = 65959L,
`65993` = 65993L, `66018` = 66018L, `67331` = 67331L, `67694` = 67694L,
`67716` = 67716L, `67746` = 67746L, `67772` = 67772L, `69088` = 69088L,
`69424` = 69424L, `69466` = 69466L, `69486` = 69486L, `69517` = 69517L,
`70805` = 70805L, `72253` = 72253L, `73419` = 73419L, `73760` = 73760L,
`73802` = 73802L, `73828` = 73828L, `73859` = 73859L, `75141` = 75141L,
`76590` = 76590L, `77752` = 77752L, `78906` = 78906L, `79486` = 79486L,
`79523` = 79523L, `79536` = 79536L, `79556` = 79556L, `80122` = 80122L,
`80159` = 80159L, `80174` = 80174L, `80214` = 80214L, `82125` = 82125L
), class = "omit"), row.names = c(31669L, 80335L, 63799L, 59674L,
1051L, 51949L), class = "data.frame")
You should adjust the number of iterations in numerical algorithm:
mDEG_zip <- gamlss(formula = deg ~ pb(SE_score) + TL + species + sex + season +
year + re(random = ~1|code) + re(random = ~1|station),
family=ZIP(), data = node_dat,
control = gamlss.control(n.cyc = 200))
The parameter n.cyc is 20 by default. I changed it to 200.
You can change the method argument, if you want Rigby and Stasinopoulos Algorithm or Cole and Green, or both, and the number of iteractions. Here is somes examples:
BCCGfixo <- gamlss(Claims1 ~1, family=BCCGo, data = dados_oc, method = RS(500))
You just need to change the argument
method = mixed(50,500)
Here the model uses 50 iteractions of RS and 500 of CG. You can use only CG too
method = CG(100)
Try changing the inicial start values of the parameters, might help. Something like that
mu.start=10, sigma.start=70, nu.start=0.5, tau.start=10
But I must warn you, work with random effects in gamlss is quite hard, and is usually that the model doesn't congerge at all, no matter what you do.
Hope this helps
I am trying to do a nice 3D plot using time series UV-Vis spectral data, like the first 3D graph below.
I used dput() function to show my data as below.
dput(head(Data))
and get:
structure(list(Wavelength = c(250, 252.5, 255, 257.5, 260, 262.5
), Date = structure(c(1365465600, 1365552000, 1365638400, 1365724800,
1365811200, 1365897600), class = c("POSIXct", "POSIXt"), tzone = "UTC"),
X250 = c(25.736217791411, 25.1182597222222, 24.8889567642957,
24.4881150070126, 24.2313916666667, 24.0346564673157), X252.5 = c(25.2168558282209,
24.6022625, 24.375429567643, 23.979541374474, 23.7248569444444,
23.5341668984701), X255 = c(24.6049539877301, 23.9896638888889,
23.7649888423989, 23.3753758765778, 23.1248263888889, 22.9342726008345
), X257.5 = c(24.0257944785276, 23.4037875, 23.1804435146443,
22.795541374474, 22.5493486111111, 22.35478581363), X260 = c(23.5917024539877,
22.9615486111111, 22.7389818688982, 22.3544109396914, 22.1091222222222,
21.9127635605007), X262.5 = c(23.1529110429448, 22.5264,
22.3051380753138, 21.924920056101, 21.6777583333333, 21.4858831710709
), X265 = c(22.5382085889571, 21.918125, 21.6982092050209,
21.3252145862553, 21.0800125, 20.8941794158554), X267.5 = c(21.8748957055215,
21.2488722222222, 21.0308619246862, 20.6643941093969, 20.4225541666667,
20.2340041724617), X270 = c(21.2791380368098, 20.6435263888889,
20.4248619246862, 20.0633674614306, 19.823075, 19.6308664812239
), X272.5 = c(20.7223159509202, 20.0815388888889, 19.8637377963738,
19.5049382889201, 19.2644375, 19.0733532684284), X275 = c(20.1307699386503,
19.4878583333333, 19.2719930264993, 18.9178920056101, 18.6778583333333,
18.487123783032), X277.5 = c(19.4758159509202, 18.8334013888889,
18.6168423988842, 18.2683702664797, 18.0308791666667, 17.8428414464534
), X280 = c(18.8002392638037, 18.1555375, 17.9391743375174,
17.5960126227209, 17.3601361111111, 17.1767204450626), X282.5 = c(18.1441809815951,
17.4953208333333, 17.2798758716876, 16.9426002805049, 16.7091777777778,
16.5276926286509), X285 = c(17.5349570552147, 16.8845097222222,
16.6697126917713, 16.337541374474, 16.1053763888889, 15.9246383866481
), X287.5 = c(16.8937300613497, 16.2423861111111, 16.0298493723849,
15.7046171107994, 15.4740472222222, 15.2938845618915), X290 = c(16.2328404907975,
15.582175, 15.3718940027894, 15.0537826086957, 14.8253291666667,
14.646668984701), X292.5 = c(15.5751472392638, 14.9258291666667,
14.7170446304045, 14.4051360448808, 14.1804194444444, 14.0051933240612
), X295 = c(14.9421717791411, 14.2904180555556, 14.0835034867504,
13.778301542777, 13.5566902777778, 13.3817329624478), X297.5 = c(14.4052392638037,
13.7537152777778, 13.5474421199442, 13.2471725105189, 13.0260375,
12.8530055632823), X300 = c(13.9050889570552, 13.2578347222222,
13.0537364016736, 12.7592973352034, 12.5405888888889, 12.3732670375522
), X302.5 = c(13.401036809816, 12.7586194444445, 12.5570264993026,
12.2684992987377, 12.0533902777778, 11.8884353268428), X305 = c(12.902717791411,
12.2623319444444, 12.0613933054393, 11.7787826086956, 11.5651972222222,
11.4043337969402), X307.5 = c(12.4249355828221, 11.7857791666667,
11.5858940027894, 11.3107840112202, 11.0999777777778, 10.9374005563282
), X310 = c(11.9935245398773, 11.3549347222222, 11.1571324965133,
10.8868260869565, 10.67955, 10.5179193324061), X312.5 = c(11.6203098159509,
10.9819680555556, 10.7849539748954, 10.516744740533, 10.3119236111111,
10.156892906815), X315 = c(11.2705674846626, 10.63485, 10.4406555090656,
10.177301542777, 9.97497638888889, 9.82023783031989), X317.5 = c(10.9098711656442,
10.2797055555556, 10.0875216178522, 9.82828471248247, 9.62720138888889,
9.47325173852573), X320 = c(10.5516625766871, 9.922075, 9.73211157601116,
9.47973772791024, 9.28055, 9.12974826147427), X322.5 = c(10.2039049079755,
9.57341805555556, 9.38568758716876, 9.13822159887798, 8.94393888888889,
8.79378581363004), X325 = c(9.86461042944785, 9.23893472222222,
9.05190516039052, 8.80583450210379, 8.61297777777778, 8.46346731571627
), X327.5 = c(9.5730490797546, 8.95366388888889, 8.76786750348675,
8.52597615708275, 8.33285972222222, 8.18865924895688), X330 = c(9.29567791411043,
8.67920416666667, 8.49494421199442, 8.25632117812062, 8.06634305555556,
7.92368984700974), X332.5 = c(9.0010981595092, 8.38594305555556,
8.20200139470014, 7.96879803646564, 7.78189166666667, 7.63967593880389
), X335 = c(8.71511349693252, 8.09972916666667, 7.92145885634589,
7.69297335203366, 7.50750694444444, 7.3643421418637), X337.5 = c(8.45265644171779,
7.84026666666667, 7.66316178521618, 7.43814305750351, 7.25333333333333,
7.11477607788595), X340 = c(8.19838036809816, 7.59187361111111,
7.41271827057183, 7.19209256661992, 7.00985416666667, 6.87590959666203
), X342.5 = c(7.95462883435583, 7.34913194444444, 7.17332914923291,
6.95607433380084, 6.77643055555556, 6.64161752433936), X345 = c(7.72929754601227,
7.12695138888889, 6.95060669456067, 6.7338920056101, 6.55681388888889,
6.42844089012517), X347.5 = c(7.50222699386503, 6.90189583333333,
6.72852022315202, 6.51498036465638, 6.33897916666667, 6.21110848400556
), X350 = c(7.2898527607362, 6.69248472222222, 6.52281171548117,
6.3138920056101, 6.13989722222222, 6.00979554937413), X352.5 = c(7.07927300613497,
6.48439166666667, 6.31443514644351, 6.10986956521739, 5.93754583333333,
5.81149652294854), X355 = c(6.86355214723926, 6.26889861111111,
6.09972524407252, 5.89783450210379, 5.72706527777778, 5.60363838664812
), X357.5 = c(6.6658527607362, 6.07161111111111, 5.90622873082287,
5.70649649368864, 5.53700833333333, 5.41251877607789), X360 = c(6.47400613496932,
5.88162777777778, 5.71633472803347, 5.5187713884993, 5.35024444444444,
5.22833796940195), X362.5 = c(6.30347852760736, 5.71433888888889,
5.54859414225942, 5.3527587657784, 5.1863375, 5.06425173852573
), X365 = c(6.16322699386503, 5.57911527777778, 5.41753835425384,
5.22367601683029, 5.05854027777778, 4.94001390820584), X367.5 = c(6.02653067484663,
5.44968333333333, 5.29124267782427, 5.10030294530154, 4.93744305555556,
4.82315020862309), X370 = c(5.86248466257669, 5.28735277777778,
5.12692747559275, 4.93981065918654, 4.77878055555556, 4.6638066759388
), X372.5 = c(5.68123006134969, 5.10715, 4.94811854951185,
4.764904628331, 4.60323333333333, 4.48933379694019), X375 = c(5.53815644171779,
4.96720416666667, 4.81097350069735, 4.62871388499299, 4.46926944444445,
4.35810987482615), X377.5 = c(5.40510429447853, 4.8370625,
4.68262761506276, 4.50320617110799, 4.34814444444444, 4.23741724617524
), X380 = c(5.30138343558282, 4.73901805555556, 4.5846889818689,
4.40751612903226, 4.25314444444444, 4.14589847009736), X382.5 = c(5.20306134969325,
4.64154722222222, 4.49053417015342, 4.31611079943899, 4.16125138888889,
4.05445757997218), X385 = c(5.07416871165644, 4.515375, 4.3646959553696,
4.19456802244039, 4.04227222222222, 3.93332127955494), X387.5 = c(4.96219325153374,
4.40868194444444, 4.25931520223152, 4.0922889200561, 3.9414,
3.83277607788595), X390 = c(4.83940797546012, 4.29276527777778,
4.14653974895398, 3.97920196353436, 3.82955277777778, 3.72600834492351
), X392.5 = c(4.71031288343558, 4.16622083333333, 4.02140446304045,
3.85572510518934, 3.70953194444445, 3.60538386648122), X395 = c(4.62059202453988,
4.07360138888889, 3.92935564853556, 3.76849228611501, 3.62277638888889,
3.51784283727399), X397.5 = c(4.49155521472393, 3.95385972222222,
3.81216178521618, 3.65344039270687, 3.50810833333333, 3.4116926286509
), X400 = c(4.37370858895706, 3.84129861111111, 3.70030125523013,
3.5432286115007, 3.39975277777778, 3.30254659248957), X402.5 = c(4.27044171779141,
3.738875, 3.59841562064156, 3.4434095371669, 3.30132638888889,
3.20369819193324), X405 = c(4.15562883435583, 3.6327375,
3.49318270571827, 3.33822159887798, 3.19675555555555, 3.10537273991655
), X407.5 = c(4.09877607361963, 3.57864305555556, 3.44027057182706,
3.28816129032258, 3.14875138888889, 3.05293602225313), X410 = c(4.01044785276074,
3.48679444444444, 3.35288842398884, 3.20463534361851, 3.06820833333333,
2.9671752433936), X412.5 = c(3.87490797546012, 3.35905694444444,
3.2244839609484, 3.07484291725105, 2.93775416666667, 2.8437983310153
), X415 = c(3.79108282208589, 3.28507361111111, 3.14952161785216,
3.00085273492286, 2.86421805555555, 2.77412378303199), X417.5 = c(3.68693558282209,
3.17824583333333, 3.04701394700139, 2.9043366058906, 2.76995972222222,
2.67318358831711), X420 = c(3.56437116564417, 3.05670972222222,
2.92747419804742, 2.786095371669, 2.65226666666667, 2.55702225312935
), X422.5 = c(3.51319018404908, 3.00990694444444, 2.87744769874477,
2.73367180925666, 2.60142361111111, 2.51227399165508), X425 = c(3.43132208588957,
2.92941805555555, 2.80041562064156, 2.65933099579243, 2.52816388888889,
2.43237969401947), X427.5 = c(3.31023926380368, 2.81629166666667,
2.68912831241283, 2.54957363253857, 2.41817777777778, 2.33177051460362
), X430 = c(3.26191717791411, 2.77505833333333, 2.64634030683403,
2.50681767180926, 2.37668055555556, 2.29549374130737), X432.5 = c(3.22984662576687,
2.74605555555556, 2.61894979079498, 2.48168022440393, 2.35282083333333,
2.26610987482615), X435 = c(3.13996932515337, 2.65807916666667,
2.5333179916318, 2.39782187938289, 2.26921388888889, 2.18432684283727
), X437.5 = c(3.08235582822086, 2.60515833333333, 2.4803249651325,
2.34626928471248, 2.21812083333333, 2.13608901251739), X440 = c(3.01686809815951,
2.54044583333333, 2.41671548117155, 2.2833842917251, 2.16062083333333,
2.07232267037552), X442.5 = c(2.93285889570552, 2.46135694444444,
2.33697489539749, 2.20531837307153, 2.08016805555556, 1.99902225312935
), X445 = c(2.8908773006135, 2.42607361111111, 2.30482426778243,
2.17530014025245, 2.04767916666667, 1.97140194714882), X447.5 = c(2.85198466257669,
2.38861111111111, 2.2694839609484, 2.14013884992987, 2.01676111111111,
1.93308901251738), X450 = c(2.8046472392638, 2.34394583333333,
2.22286750348675, 2.09689481065919, 1.97544444444445, 1.89651043115438
), X452.5 = c(2.77545705521472, 2.31733333333333, 2.20150627615063,
2.07733800841515, 1.95440277777778, 1.87571627260083), X455 = c(2.71198159509203,
2.26175277777778, 2.14609483960948, 2.02168022440393, 1.90238194444444,
1.82506536856745), X457.5 = c(2.69071779141104, 2.24759861111111,
2.1283709902371, 2.0067461430575, 1.88878055555556, 1.81217802503477
), X460 = c(2.63857055214724, 2.19480416666667, 2.07814365411437,
1.95862833099579, 1.84296111111111, 1.76875243393602), X462.5 = c(2.55233742331288,
2.10490416666667, 1.99411854951185, 1.87502664796634, 1.76141805555556,
1.68473435326843), X465 = c(2.54993251533742, 2.10804166666667,
1.99814644351464, 1.88111360448808, 1.76530277777778, 1.68772322670375
), X467.5 = c(2.54192944785276, 2.10937638888889, 1.99570990237099,
1.88144319775596, 1.76786666666667, 1.69657162726008), X470 = c(2.45424233128834,
2.02388472222222, 1.91476011157601, 1.80086255259467, 1.68827638888889,
1.61762865090403), X472.5 = c(2.38808588957055, 1.96169583333333,
1.85649232914923, 1.74286956521739, 1.63194444444444, 1.56048400556328
), X475 = c(2.40716871165644, 1.98333055555556, 1.8761589958159,
1.7647840112202, 1.65609861111111, 1.58707093184979), X477.5 = c(2.38796012269939,
1.96411527777778, 1.85740725244073, 1.74938990182328, 1.64029444444444,
1.57306258692629), X480 = c(2.32367484662577, 1.90594027777778,
1.80042677824268, 1.69331276297335, 1.58667777777778, 1.51872322670375
), X482.5 = c(2.26839877300614, 1.85602638888889, 1.75232914923291,
1.64626928471248, 1.542725, 1.47292350486787), X485 = c(2.24077914110429,
1.82308194444444, 1.72304323570432, 1.61980925666199, 1.5158,
1.44215159944367), X487.5 = c(2.19346625766871, 1.77742916666667,
1.67717712691771, 1.57596213183731, 1.46934861111111, 1.39640751043115
), X490 = c(2.1796226993865, 1.77528611111111, 1.67270432357043,
1.56773772791024, 1.46491111111111, 1.39967732962448), X492.5 = c(2.19796932515337,
1.79399861111111, 1.69254253835425, 1.58660028050491, 1.48578888888889,
1.42326842837274), X495 = c(2.16203067484663, 1.75784722222222,
1.66087866108787, 1.56017812061711, 1.45659583333333, 1.39502364394993
), X497.5 = c(2.12128220858896, 1.72433055555556, 1.62827615062762,
1.52875175315568, 1.42800833333333, 1.36483727399165), X500 = c(2.10455214723926,
1.71279027777778, 1.61463458856346, 1.51266058906031, 1.41571388888889,
1.35290403337969), X502.5 = c(2.09321779141104, 1.70464027777778,
1.60352022315202, 1.50263674614306, 1.40064722222222, 1.34636300417246
), X505 = c(2.06455828220859, 1.67302916666667, 1.57388145048815,
1.47552734922861, 1.37514166666667, 1.31759944367177), X507.5 = c(2.01607975460123,
1.62675138888889, 1.53272524407252, 1.43562692847125, 1.34284444444444,
1.27975660639777), X510 = c(1.97671165644172, 1.59260694444444,
1.50193444909344, 1.41013043478261, 1.31677916666667, 1.25430876216968
), X512.5 = c(1.96352147239264, 1.57798472222222, 1.48636262203626,
1.39764375876578, 1.30225555555556, 1.23669680111266), X515 = c(1.92503987730061,
1.54797083333333, 1.45469456066946, 1.36199298737728, 1.26766805555556,
1.21219749652295), X517.5 = c(1.90465030674847, 1.53300555555556,
1.44391352859135, 1.35238569424965, 1.2612375, 1.20831293463143
), X520 = c(1.87689570552147, 1.50217916666667, 1.4149330543933,
1.32729733520337, 1.23678194444445, 1.17577329624478), X522.5 = c(1.83759202453988,
1.46538888888889, 1.37917015341702, 1.29119775596073, 1.20174861111111,
1.14477607788595), X525 = c(1.83651533742331, 1.46525, 1.38037935843794,
1.29450490883591, 1.2079125, 1.15156189151599), X527.5 = c(1.82067791411043,
1.45758472222222, 1.36769874476987, 1.28326788218794, 1.19442916666667,
1.13923365785814), X530 = c(1.7854754601227, 1.42325833333333,
1.33740446304045, 1.25485553997195, 1.16439166666667, 1.11009457579972
), X532.5 = c(1.72706134969325, 1.36137083333333, 1.28353277545328,
1.20330014025245, 1.1164875, 1.05959248956885), X535 = c(1.73546012269939,
1.37703333333333, 1.2953570432357, 1.21220617110799, 1.12938333333333,
1.07361335187761), X537.5 = c(1.75189570552147, 1.39752083333333,
1.3135160390516, 1.23146283309958, 1.15030416666667, 1.10116133518776
), X540 = c(1.70230061349693, 1.35148888888889, 1.2665369595537,
1.18725666199158, 1.10636944444444, 1.06070792767733), X542.5 = c(1.71566564417178,
1.36359027777778, 1.28287029288703, 1.20369565217391, 1.1235375,
1.07357719054242), X545 = c(1.66589877300613, 1.31230416666667,
1.23633891213389, 1.15782889200561, 1.07775833333333, 1.01979972183588
), X547.5 = c(1.62098466257669, 1.27635555555556, 1.19730962343096,
1.11939270687237, 1.03757361111111, 0.990175243393602), X550 = c(1.64088650306749,
1.30173333333333, 1.22275592747559, 1.14718793828892, 1.06357916666667,
1.02470931849791), X552.5 = c(1.65776687116564, 1.31616944444444,
1.23976708507671, 1.16367180925666, 1.08286944444444, 1.03624200278164
), X555 = c(1.64775766871166, 1.30469861111111, 1.23003905160391,
1.15642496493689, 1.07983611111111, 1.02301529902643), X557.5 = c(1.57864417177914,
1.24234861111111, 1.16718131101813, 1.09518373071529, 1.01922638888889,
0.966821974965229), X560 = c(1.55608895705521, 1.22404166666667,
1.14994979079498, 1.07797896213184, 0.998729166666667, 0.953904033379694
), X562.5 = c(1.57327300613497, 1.24271111111111, 1.17039748953975,
1.10076437587658, 1.02429722222222, 0.982093184979138), X565 = c(1.54155828220859,
1.21247638888889, 1.14011994421199, 1.06987377279102, 0.995325,
0.950226703755216), X567.5 = c(1.5184754601227, 1.19163194444444,
1.11808926080893, 1.04673772791024, 0.969984722222222, 0.920390820584145
), X570 = c(1.52191104294479, 1.19953472222222, 1.12552022315202,
1.05369144460028, 0.979043055555555, 0.936059805285118),
X572.5 = c(1.49422699386503, 1.17113333333333, 1.09843793584379,
1.02884291725105, 0.956215277777778, 0.914119610570236),
X575 = c(1.48120552147239, 1.16091666666667, 1.08878382147838,
1.02371248246844, 0.950745833333333, 0.903524339360223),
X577.5 = c(1.49473312883436, 1.17405, 1.10468340306834, 1.04081346423562,
0.967102777777778, 0.918347705146036), X580 = c(1.45893558282209,
1.13822638888889, 1.07150488145049, 1.004904628331, 0.931488888888889,
0.888706536856745), X582.5 = c(1.40335889570552, 1.08981666666667,
1.02136262203626, 0.955345021037868, 0.882438888888889, 0.842457579972184
), X585 = c(1.41981901840491, 1.11120277777778, 1.04046164574617,
0.976112201963534, 0.902045833333333, 0.854774687065369),
X587.5 = c(1.44122085889571, 1.13389722222222, 1.06531938633194,
0.998193548387097, 0.927052777777778, 0.884666203059805),
X590 = c(1.43262883435583, 1.12188472222222, 1.05589539748954,
0.991642356241234, 0.923915277777778, 0.883214186369958),
X592.5 = c(1.39457668711656, 1.08495694444444, 1.01676150627615,
0.954368863955119, 0.886066666666667, 0.842401947148818),
X595 = c(1.39473006134969, 1.09030138888889, 1.02193584379358,
0.95813744740533, 0.889866666666667, 0.851438108484006),
X597.5 = c(1.39176380368098, 1.08874305555556, 1.02160251046025,
0.960784011220196, 0.891755555555556, 0.851367176634214),
X600 = c(1.32321472392638, 1.02342361111111, 0.958054393305439,
0.896713884992987, 0.828456944444444, 0.789112656467316),
X602.5 = c(1.3699754601227, 1.06539861111111, 1.00389818688982,
0.94194950911641, 0.875038888888889, 0.832920723226704),
X605 = c(1.39004294478528, 1.0856125, 1.02410460251046, 0.964893408134642,
0.897881944444444, 0.851397774687065), X607.5 = c(1.33887423312883,
1.04344305555556, 0.978672245467225, 0.918176718092567, 0.853443055555556,
0.810876216968011), X610 = c(1.34292944785276, 1.04911111111111,
0.98465690376569, 0.922030855539972, 0.858780555555556, 0.820618915159944
), X612.5 = c(1.32123006134969, 1.02857083333333, 0.962319386331939,
0.899249649368864, 0.834901388888889, 0.797037552155772),
X615 = c(1.30576380368098, 1.01544166666667, 0.949560669456067,
0.889232819074334, 0.824005555555556, 0.787675938803894),
X617.5 = c(1.30801840490798, 1.01950138888889, 0.959458856345886,
0.902255259467041, 0.839116666666667, 0.80459388038943),
X620 = c(1.25273619631902, 0.963565277777778, 0.903634588563459,
0.845955119214586, 0.781859722222222, 0.751739916550765),
X622.5 = c(1.24233742331288, 0.951993055555556, 0.889652719665272,
0.832720897615708, 0.765165277777778, 0.728485396383866),
X625 = c(1.3115736196319, 1.020975, 0.960577405857741, 0.902300140252454,
0.835329166666667, 0.789098748261474), X627.5 = c(1.32251840490798,
1.03869861111111, 0.977846582984658, 0.916507713884993, 0.852429166666667,
0.812655076495132), X630 = c(1.26833435582822, 0.9928125,
0.93086750348675, 0.87539270687237, 0.812751388888889, 0.775739916550765
), X632.5 = c(1.24898773006135, 0.965934722222222, 0.908059972105997,
0.85682889200561, 0.794722222222222, 0.751435326842837),
X635 = c(1.28299386503067, 0.995876388888889, 0.936375174337517,
0.881263674614306, 0.819331944444444, 0.78326842837274),
X637.5 = c(1.2784018404908, 1.00060833333333, 0.939645746164575,
0.880981767180926, 0.821463888888889, 0.792333796940195),
X640 = c(1.24037116564417, 0.960616666666667, 0.905357043235704,
0.846816269284712, 0.787144444444444, 0.744055632823366),
X642.5 = c(1.19373619631902, 0.907856944444444, 0.852075313807531,
0.797594670406732, 0.732779166666667, 0.684244784422809),
X645 = c(1.17663190184049, 0.898422222222222, 0.835518828451883,
0.782708274894811, 0.717233333333333, 0.684310152990264),
X647.5 = c(1.21073619631902, 0.943138888888889, 0.879875871687587,
0.826015427769986, 0.7630125, 0.737840055632823), X650 = c(1.20088036809816,
0.934829166666667, 0.877008368200837, 0.821608695652174,
0.760254166666667, 0.727004172461752), X652.5 = c(1.18600920245399,
0.915881944444444, 0.860887029288703, 0.80358064516129, 0.744027777777778,
0.705867872044506), X655 = c(1.22061349693252, 0.951930555555556,
0.896224546722455, 0.840730715287518, 0.784806944444444,
0.750878998609179), X657.5 = c(1.19243865030675, 0.924801388888889,
0.865479776847978, 0.81258064516129, 0.757759722222222, 0.721283727399165
), X660 = c(1.10550920245399, 0.841702777777778, 0.780730822873082,
0.731366058906031, 0.673393055555556, 0.638547983310153)), row.names = c(NA,
6L), class = "data.frame")
My code:
Data <- data.frame(Data)
library(rgl)
library(pls)
x <- as.POSIXct(Data$Date, format = "2013-01-10", tz = "Australia/Adelaide")
y <- Data$Wavelength
z <- as.matrix(Data[,3:167])
open3d()
plot3d(x, y, z,col="purple", size=3, xlab = "Date", ylab = "Wavelength", zlab = "Absorbance (/cm)")
I get a not very nice plot as shown in the second graph.
The x-axis has labels in numerical dates, but I want the actual dates in the format like "1 June 2013" or "1/6/2013".
I also don't know how to make a nice rainbow colour looking 3D graph.
Any help is welcome :)
a typical UV-Vis spectrum looks like graph 3 below
using the codes provided by Marco Sandri,
library(tidyr)
library(ggplot2)
library(plotly)
Data %>%
gather(Series, y, X250:X660, factor_key=TRUE) %>%
plot_ly(x = ~Date, y = ~Wavelength, z = ~y,
type = 'scatter3d', mode = 'lines', color = ~Series)
The graph 4 is the plot I get.
A solution based on the plotly package:
library(tidyr)
library(plotly)
Data %>%
gather(Series, y, X250:X660, factor_key=TRUE) %>%
plot_ly(x = ~Date, y = ~Wavelength, z = ~y, type = 'scatter3d', mode = 'lines', color = ~Series)
You may omit and redraw the axes using bbox3d. In bbox3d just xat= and xlab= specifications are needed, where we use x.
open3d()
plot3d(x, y, z, col="purple", size=3, xlab="Date", ylab="Wavelength",
zlab="Absorbance (/cm)", axes=FALSE, top=TRUE)
bbox3d(xat=x, xlab=x, col="black", front="line", back="line", lit=FALSE)
To calculate the Red Edge Position Index, I need to find the wavelength value (column name) corresponding to the maximum derivative of reflectance in the red edge region from 690nm to 740nm. I have included a subset of my dataframe below, it contains the correct interval...
I have 640 rows (Sample) of 2151 measurements (values) plus a few catagoricals in the first columns (e.g. plantType and plantCondition). I need to find the column of the value corresponding to the maximum of the derivative of the values in the interval specified and return the wavelength value to the REPI column.
I am trying something like this but I do not know how to calculate the maximum of the derivative in the specified interval
# find the maximum of the derivative of the values in columns x690:x740
# attempt to find for single sample first
> which( colnames(spec.data)=="X690")
[1] 352
> which( colnames(spec.data)=="X740")
[1] 402
# I want to return the values of the differential but this doesn't work
> foo.vector <- diff(spec.data[1,352:402])
>> Error in r[i1] - r[-length(r):-(length(r) - lag + 1L)] : non-numeric argument to binary operator
This makes sense because I don't have the dt in dx/dt but I am not sure how to retrieve the position of the maximum value of the derivative of this interval. once I did I think I would
> spec.data$REPI <- which( colnames(spec.data) == max(foo.vector))
Then I think I would lapply this for each row?
Can anyone point me towards a solution for this?
Thank you...
subset of data from dput
> dput(spec.data[1:2, c(1:3, 7, 300:450)])
structure(list(Sample = c("JUMO_G1 P1T9 Leaf Clip00000.asd",
"JUMO_G1 P1T9 Leaf Clip00001.asd"), plantType = c("JUMO", "JUMO"
), plantCondition = c("G", "G"), REPI = c(NA_real_, NA_real_),
X638 = c(0.0611, 0.06114), X639 = c(0.0606, 0.06064), X640 = c(0.0601,
0.06012), X641 = c(0.0595, 0.05953), X642 = c(0.0589, 0.05893
), X643 = c(0.0584, 0.05834), X644 = c(0.0577, 0.05775),
X645 = c(0.05717, 0.05717), X646 = c(0.0566, 0.05664), X647 = c(0.0562,
0.05618), X648 = c(0.0557, 0.05573), X649 = c(0.0554, 0.05536
), X650 = c(0.0551, 0.05505), X651 = c(0.0547, 0.05475),
X652 = c(0.05448, 0.05447), X653 = c(0.0542, 0.05421), X654 = c(0.054,
0.05395), X655 = c(0.0536, 0.05357), X656 = c(0.0532, 0.05319
), X657 = c(0.0528, 0.05277), X658 = c(0.0523, 0.05229),
X659 = c(0.0518, 0.05176), X660 = c(0.05128, 0.05126), X661 = c(0.0508,
0.05077), X662 = c(0.0503, 0.05024), X663 = c(0.0498, 0.04978
), X664 = c(0.0494, 0.04936), X665 = c(0.049, 0.04897), X666 = c(0.04869,
0.04866), X667 = c(0.0484, 0.04838), X668 = c(0.0482, 0.04815
), X669 = c(0.048, 0.04797), X670 = c(0.0479, 0.04782), X671 = c(0.0478,
0.04775), X672 = c(0.0478, 0.04773), X673 = c(0.0478, 0.04773
), X674 = c(0.0478, 0.04776), X675 = c(0.0479, 0.04786),
X676 = c(0.0481, 0.04802), X677 = c(0.0483, 0.0482), X678 = c(0.0486,
0.04843), X679 = c(0.0489, 0.04873), X680 = c(0.04925, 0.04911
), X681 = c(0.0498, 0.04962), X682 = c(0.0504, 0.05026),
X683 = c(0.05122, 0.05103), X684 = c(0.0522, 0.052), X685 = c(0.0533,
0.05317), X686 = c(0.0548, 0.05458), X687 = c(0.05647, 0.05627
), X688 = c(0.0584, 0.05824), X689 = c(0.0608, 0.06057),
X690 = c(0.0634, 0.06326), X691 = c(0.0664, 0.06626), X692 = c(0.0698,
0.06958), X693 = c(0.0734, 0.07317), X694 = c(0.0773, 0.07701
), X695 = c(0.0814, 0.08109), X696 = c(0.0856, 0.0854), X697 = c(0.0901,
0.08989), X698 = c(0.0947, 0.09449), X699 = c(0.0994, 0.09917
), X700 = c(0.10417, 0.10395), X701 = c(0.10899, 0.10881),
X702 = c(0.11385, 0.11366), X703 = c(0.11871, 0.11854), X704 = c(0.12356,
0.12342), X705 = c(0.1284, 0.12829), X706 = c(0.13324, 0.13312
), X707 = c(0.13803, 0.13792), X708 = c(0.14281, 0.14273),
X709 = c(0.14763, 0.14755), X710 = c(0.15243, 0.15235), X711 = c(0.15718,
0.15713), X712 = c(0.16192, 0.16189), X713 = c(0.1667, 0.16663
), X714 = c(0.17143, 0.17137), X715 = c(0.17609, 0.17605),
X716 = c(0.18069, 0.18062), X717 = c(0.18528, 0.1852), X718 = c(0.18977,
0.18968), X719 = c(0.19417, 0.19406), X720 = c(0.19851, 0.19838
), X721 = c(0.20276, 0.20263), X722 = c(0.20686, 0.20671),
X723 = c(0.2108, 0.21063), X724 = c(0.21465, 0.21449), X725 = c(0.21837,
0.21819), X726 = c(0.22194, 0.22174), X727 = c(0.22534, 0.22515
), X728 = c(0.2286, 0.22838), X729 = c(0.23164, 0.23142),
X730 = c(0.23447, 0.23427), X731 = c(0.23719, 0.23696), X732 = c(0.23984,
0.23959), X733 = c(0.24229, 0.24203), X734 = c(0.24452, 0.24426
), X735 = c(0.24668, 0.24638), X736 = c(0.24867, 0.24839),
X737 = c(0.25053, 0.25028), X738 = c(0.25229, 0.25203), X739 = c(0.25382,
0.25359), X740 = c(0.25531, 0.25508), X741 = c(0.25672, 0.25646
), X742 = c(0.25791, 0.25766), X743 = c(0.25907, 0.25884),
X744 = c(0.26014, 0.25993), X745 = c(0.2611, 0.26089), X746 = c(0.26201,
0.26178), X747 = c(0.26278, 0.26257), X748 = c(0.26347, 0.26329
), X749 = c(0.26414, 0.26397), X750 = c(0.26475, 0.26459),
X751 = c(0.26525, 0.2651), X752 = c(0.26568, 0.26554), X753 = c(0.26614,
0.266), X754 = c(0.26652, 0.26639), X755 = c(0.26682, 0.26671
), X756 = c(0.2671, 0.26701), X757 = c(0.26743, 0.26734),
X758 = c(0.26767, 0.26758), X759 = c(0.26789, 0.26781), X760 = c(0.26814,
0.26808), X761 = c(0.2682, 0.26817), X762 = c(0.26835, 0.26831
), X763 = c(0.26856, 0.26851), X764 = c(0.26872, 0.26869),
X765 = c(0.26884, 0.26881), X766 = c(0.26892, 0.2689), X767 = c(0.26896,
0.26894), X768 = c(0.26898, 0.26896), X769 = c(0.2691, 0.26909
), X770 = c(0.2692, 0.2692), X771 = c(0.26921, 0.26921),
X772 = c(0.26923, 0.26926), X773 = c(0.26927, 0.26931), X774 = c(0.26935,
0.26939), X775 = c(0.26945, 0.26947), X776 = c(0.26946, 0.26949
), X777 = c(0.26948, 0.26952), X778 = c(0.26953, 0.26958),
X779 = c(0.26958, 0.26963), X780 = c(0.26965, 0.2697), X781 = c(0.2697,
0.26975), X782 = c(0.2697, 0.26977), X783 = c(0.26972, 0.26978
), X784 = c(0.26979, 0.26982), X785 = c(0.26987, 0.2699),
X786 = c(0.26991, 0.26998), X787 = c(0.26989, 0.26997), X788 = c(0.26991,
0.26998)), .Names = c("Sample", "plantType", "plantCondition",
"REPI", "X638", "X639", "X640", "X641", "X642", "X643", "X644",
"X645", "X646", "X647", "X648", "X649", "X650", "X651", "X652",
"X653", "X654", "X655", "X656", "X657", "X658", "X659", "X660",
"X661", "X662", "X663", "X664", "X665", "X666", "X667", "X668",
"X669", "X670", "X671", "X672", "X673", "X674", "X675", "X676",
"X677", "X678", "X679", "X680", "X681", "X682", "X683", "X684",
"X685", "X686", "X687", "X688", "X689", "X690", "X691", "X692",
"X693", "X694", "X695", "X696", "X697", "X698", "X699", "X700",
"X701", "X702", "X703", "X704", "X705", "X706", "X707", "X708",
"X709", "X710", "X711", "X712", "X713", "X714", "X715", "X716",
"X717", "X718", "X719", "X720", "X721", "X722", "X723", "X724",
"X725", "X726", "X727", "X728", "X729", "X730", "X731", "X732",
"X733", "X734", "X735", "X736", "X737", "X738", "X739", "X740",
"X741", "X742", "X743", "X744", "X745", "X746", "X747", "X748",
"X749", "X750", "X751", "X752", "X753", "X754", "X755", "X756",
"X757", "X758", "X759", "X760", "X761", "X762", "X763", "X764",
"X765", "X766", "X767", "X768", "X769", "X770", "X771", "X772",
"X773", "X774", "X775", "X776", "X777", "X778", "X779", "X780",
"X781", "X782", "X783", "X784", "X785", "X786", "X787", "X788"
), row.names = 1:2, class = "data.frame")
You can try this
spec.data$REPI <- apply(spec.data[,-(1:4)], 1, function(x) which.max(diff(x)))
Or you can try using dplyr and tidyr:
library(dplyr)
library(tidyr)
spec.data %>%
gather(key, value, -Sample, -plantType, - plantCondition, -REPI) %>%
group_by(Sample) %>%
summarise(which.max(diff(value)))
They both seem to give same results.
Below is a subsample of my data set (only 2 rows by 215 columns). I am trying to view them on RStudio but it gives me the following error:
Error in View : undefined columns selected
Do not really know what is going on. The whole set is 7786 rows by 215 columns. Viewing it works fine, however, when doing any kind of subsetting or removing one row it is no longer want to view.
structure(list(`NA` = structure(c(16343, 16344), class = "Date"),
AVON = c("615.5", "621.5"), BA. = c("471.5", "463.2"), CMRG = c("224.5",
"224.5"), COB = c("291.10000000000002", "283.5"), MGGT = c("451.2",
"444.7"), QQ. = c("224.5", "223.5"), RR. = c("953.65", "933.38"
), SNR = c("268.2", "264.7"), ULE = c("1771", "1746"), GKN = c("319.2",
"311.5"), BRAG = c("617", "603"), BVIC = c("668", "661"),
CCH = c("1333", "1327"), DGE = c("1785", "1760.5"), SAB = c("3428",
"3383"), STCK = c("291.60000000000002", "294"), ALNT = c("328",
"321"), CAR = c("125", "124.5"), CRDA = c("2053", "1990"),
ELM = c("255.5", "254.5"), JMAT = c("2919", "2825"), SYNT = c("212",
"210.8"), VCTA = c("1606", "1605"), DIA = c("901", "924"),
DNO = c("611", "611"), E2V = c("161", "160.5"), HLMA = c("612",
"598.5"), HTY = c("309.8", "308"), MGAM = c("296.8", "289.40000000000003"
), OXFD = c("1020", "1035"), RSHW = c("1630", "1625"), SXS = c("1808",
"1778"), TTG = c("166.75", "167.5"), XAR = c("376", "367"
), X = c("1527", "1520"), ABF = c("2679", "2654"), AE = c("633.5",
"640"), CARM = c("1647", "1637"), CWK = c("1328", "1320"),
DCG = c("383.75", "369"), DVO = c("237.75", "231"), GNCL = c("234",
"229.6"), HFG = c("416", "411"), FD = c("36.5", "34.75"),
TATE = c("591.5", "585"), MNDI = c("1011", "1012"), BI = c("616",
"620"), REX = c("491.8", "483.5"), RC = c("559", "540"),
SMDS = c("266.3", "257"), SMIN = c("1264", "1250"), VSVS = c("451.8",
"438.40000000000003"), AGA = c("163.25", "160.25"), BDEV = c("396.1",
"389.3"), BKG = c("2250", "2224"), BLWY = c("1567", "1558"
), BVS = c("779", "771"), CRST = c("325", "314.60000000000002"
), GLSN = c("393.5", "388.5"), MCB = c("83.53", "83.29"),
SN = c("1334", "1309"), RB. = c("5350", "5305"), RDW = c("280.7",
"273.8"), TW. = c("112.8", "111.8"), BODY = c("668.5", "647"
), FENR = c("317.60000000000002", "313.10000000000002"),
GDWN = c("3500", "3500"), HILS = c("561", "561.5"), IMI = c("1230",
"1206"), MRO = c("247.70000000000002", "246"), VAR = c("304",
"300.75"), RNO = c("56", "54.5"), RTRK = c("2765", "2736"
), SFR = c("63.5", "64"), SRX = c("2826", "2812"), TRI = c("105.75",
"105"), VTC = c("613.5", "612"), WEIR = c("2502", "2430"),
EVR = c("130", "123.60000000000001"), FXO = c("112.3", "105.10000000000001"
), BBA = c("325", "326"), BMS = c("494.38", "492"), CKN = c("2350",
"2341"), FSHR = c("1326", "1294"), RMG = c("392.2", "399.7"
), STOB = c("111", "109"), UKM = c("473.88", "467"), WIN = c("136.25",
"137.5"), GAW = c("597.5", "585"), HTM = c("131.5", "129.25"
), `NA` = c(NA_character_, NA_character_), AAL = c("1384",
"1363.5"), ABG = c("218.8", "209.1"), ANTO = c("721", "702"
), AF = c("131.5", "130.25"), AQ = c("18.5", "18.75"), ARMS = c("69",
"62.25"), BLT = c("1715", "1690.5"), CEY = c("61.15", "61"
), FRES = c("760", "747"), GEMD = c("192", "191.75"), GLEN = c("343.2",
"336.45"), HOC = c("135.30000000000001", "130.19999999999999"
), KAZ = c("263.39999999999998", "260.10000000000002"), KMRL = c("9.5",
"9.3000000000000007"), LMI = c("185.8", "176.8"), NWR = c("1.97",
"1.82"), `NA` = c(NA_character_, NA_character_), DL = c("190.20000000000002",
"190"), OG = c("22", "24"), OLY = c("516", "496.6"), RIO = c("3031.5",
"3020"), RRS = c("4209", "4154"), VED = c("998.5", "974.5"
), AFR = c("103.5", "109.4"), BG. = c("1140", "1093"), B. = c("453.45",
"452.75"), CNE = c("176.5", "171.6"), ENQ = c("109.60000000000001",
"107.8"), EXI = c("157", "150"), HDY = c("102", "99.75"),
JKX = c("48.25", "47"), OHR = c("229.3", "220.9"), MO = c("333",
"324.7"), RDSA = c("2358.5", "2331"), RDSB = c("2437", "2418.5"
), SIA = c("381", "377.90000000000003"), SMDR = c("100",
"98.5"), TLW = c("644.5", "631"), AMEC = c("1104", "1077"
), CIU = c("283.5", "275.75"), GMS = c("157", "157"), HTG = c("892.5",
"876"), LAM = c("163.25", "160"), FC = c("1037", "1011"),
WG. = c("759.5", "743"), BRBY = c("1511", "1476"), ZC = c("365.7",
"366"), SG = c("1133", "1126"), TED = c("1863", "1862"),
ULVR = c("2585", "2547"), AZN = c("4441.5", "4360.5"), BTG = c("700",
"697.5"), CIR = c("304", "300"), DH = c("758", "753"), GNS = c("1130",
"1130"), GSK = c("1413", "1414"), HIK = c("1733", "1715"),
SH = c("5340", "5310"), SK = c("329.25", "319"), VEC = c("132",
"132"), AGK = c("1548", "1528"), AHT = c("1043", "1024"),
ATK = c("1317", "1323"), BAB = c("1092", "1085"), BNZL = c("1610",
"1597"), BRAM = c("376", "374"), BRSN = c("980", "979"),
CLLN = c("304.60000000000002", "304.3"), CMS = c("59.75",
"59.5"), CNCT = c("149.25", "151"), CI = c("1164", "1165"
), CTR = c("259.5", "255"), DCC = c("3422", "3405"), DLAR = c("477",
"478"), DLM = c("689.5", "685"), ECOM = c("223", "219.8"),
ESNT = c("797.5", "792.5"), EXO = c("176.5", "180"), EXN = c("983.5",
"968"), GFS = c("250.70000000000002", "251.6"), GFTU = c("626",
"616"), HAS = c("116.3", "115.7"), HRG = c("45.75", "45.75"
), HSV = c("319.7", "319"), HWDN = c("339.1", "335"), HYC = c("749",
"748"), IRV = c("599.5", "592.5"), ITRK = c("2621", "2631"
), LVD = c("201.75", "201.5"), MER = c("435", "436.75"),
MMC = c("25.25", "25"), MNZS = c("569", "575.5"), MI = c("418.6",
"421"), MTO = c("287.90000000000003", "286.60000000000002"
), NTG = c("483.8", "481.3"), AY = c("983.5", "989"), FL = c("182",
"180.1"), RCDO = c("671", "667.5"), RENT = c("117.8", "116"
), RGU = c("169.70000000000002", "169.9"), RS = c("261",
"251.6"), RWA = c("302.5", "302.5"), SDY = c("70.5", "69.75"
), SERC = c("286.10000000000002", "279.8"), SHI = c("166.6",
"161.1"), SIV = c("199.75", "200"), SKS = c("90", "92"),
STHR = c("350.25", "358.5"), TK = c("1664", "1635"), TRB = c("170.5",
"172"), V. = c("609.5", "600"), WOS = c("3242", "3243"),
XCH = c("188", "184.75"), ARM = c("906", "887.5"), BVC = c("16.38",
"16.25"), CSR = c("758", "756"), IMG = c("188.5", "184.75"
), LRD = c("309.7", "306.7"), IC = c("298.10000000000002",
"299"), SEU = c("141", "141"), ST = c("104.60000000000001",
"99.9"), BATS = c("3482", "3480"), IMT = c("2664", "2679"
)), .Names = c("NA", "AVON", "BA.", "CMRG", "COB", "MGGT",
"QQ.", "RR.", "SNR", "ULE", "GKN", "BRAG", "BVIC", "CCH", "DGE",
"SAB", "STCK", "ALNT", "CAR", "CRDA", "ELM", "JMAT", "SYNT",
"VCTA", "DIA", "DNO", "E2V", "HLMA", "HTY", "MGAM", "OXFD", "RSHW",
"SXS", "TTG", "XAR", "X", "ABF", "AE", "CARM", "CWK", "DCG",
"DVO", "GNCL", "HFG", "FD", "TATE", "MNDI", "BI", "REX", "RC",
"SMDS", "SMIN", "VSVS", "AGA", "BDEV", "BKG", "BLWY", "BVS",
"CRST", "GLSN", "MCB", "SN", "RB.", "RDW", "TW.", "BODY", "FENR",
"GDWN", "HILS", "IMI", "MRO", "VAR", "RNO", "RTRK", "SFR", "SRX",
"TRI", "VTC", "WEIR", "EVR", "FXO", "BBA", "BMS", "CKN", "FSHR",
"RMG", "STOB", "UKM", "WIN", "GAW", "HTM", NA, "AAL", "ABG",
"ANTO", "AF", "AQ", "ARMS", "BLT", "CEY", "FRES", "GEMD", "GLEN",
"HOC", "KAZ", "KMRL", "LMI", "NWR", NA, "DL", "OG", "OLY", "RIO",
"RRS", "VED", "AFR", "BG.", "B.", "CNE", "ENQ", "EXI", "HDY",
"JKX", "OHR", "MO", "RDSA", "RDSB", "SIA", "SMDR", "TLW", "AMEC",
"CIU", "GMS", "HTG", "LAM", "FC", "WG.", "BRBY", "ZC", "SG",
"TED", "ULVR", "AZN", "BTG", "CIR", "DH", "GNS", "GSK", "HIK",
"SH", "SK", "VEC", "AGK", "AHT", "ATK", "BAB", "BNZL", "BRAM",
"BRSN", "CLLN", "CMS", "CNCT", "CI", "CTR", "DCC", "DLAR", "DLM",
"ECOM", "ESNT", "EXO", "EXN", "GFS", "GFTU", "HAS", "HRG", "HSV",
"HWDN", "HYC", "IRV", "ITRK", "LVD", "MER", "MMC", "MNZS", "MI",
"MTO", "NTG", "AY", "FL", "RCDO", "RENT", "RGU", "RS", "RWA",
"SDY", "SERC", "SHI", "SIV", "SKS", "STHR", "TK", "TRB", "V.",
"WOS", "XCH", "ARM", "BVC", "CSR", "IMG", "LRD", "IC", "SEU",
"ST", "BATS", "IMT"), row.names = 7785:7786, class = "data.frame")
I am on Mac OS 10.10, R 3.1.1 and RStudio 0.98.1060.
One of your column names is NA. If d is your data defined above, then try names(d)[92]. Try replacing with a non-missing column name.
As allready mentioned by DMC, but with a short version of your example code.
a <- structure(list(`NA` = structure(c(16343, 16344), class = "Date"),
AVON = c("615.5", "621.5"),
BA. = c("471.5", "463.2"),
`NA` = c(NA_character_, NA_character_), AAL = c("1384", "1363.5")),
.Names = c(NA, "AVON", "BA.", "NA", "AAL"), row.names = 7785:7786, class = "data.frame")
View(a)
Error in View : undefined columns selected
names(a)
[1] NA "AVON" "BA." "NA" "AAL"
a <- structure(list(`NA` = structure(c(16343, 16344), class = "Date"),
AVON = c("615.5", "621.5"),
BA. = c("471.5", "463.2"),
`NA` = c(NA_character_, NA_character_), AAL = c("1384", "1363.5")),
.Names = c("NA", "AVON", "BA.", "NA", "AAL"), row.names = 7785:7786, class = "data.frame")
View(a)
names(a)
[1] "NA" "AVON" "BA." "NA" "AAL"
You need to have proper names in the data frame to View it.
Code:
library(nnet)
library(caret)
#K-folds resampling method for fitting model
ctrl <- trainControl(method = "repeatedcv", number = 10, repeats = 10,
allowParallel = TRUE) #10 separate 10-fold cross-validations
nnetGrid <- expand.grid(decay = seq(0.0002, .0008, length = 4),
size = seq(6, 10, by = 2),
bag = FALSE)
set.seed(100)
nnetFitcv <- train(R ~ .,
data = trainSet,
method = "avNNet",
tuneGrid = nnetGrid,
trControl = ctrl,
preProc = c("center", "scale"),
linout = TRUE,
## Reduce the amount of printed output
trace = FALSE,
## Expand the number of iterations to find
## parameter estimates..
maxit = 2000,
## and the number of parameters used by the model
MaxNWts = 5 * (34 + 1) + 5 + 1)
Error:
Error in train.default(x, y, weights = w, ...) :
final tuning parameters could not be determined
In addition: Warning messages:
1: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, :
There were missing values in resampled performance measures.
2: In train.default(x, y, weights = w, ...) :
missing values found in aggregated results
data:
dput(head(trainSet))
structure(list(fy = c(317.913756282, 365.006253069, 392.548100067,
305.350697829, 404.999341917, 326.558279739), fu = c(538.962896683,
484.423120589, 607.974981919, 566.461909098, 580.287855801, 454.178316794
), E = c(194617.707566, 181322.455065, 206661.286272, 182492.029532,
189867.929239, 181991.379749), eu = c(0.153782620813, 0.208857408687,
0.29933255604, 0.277013319499, 0.251278125174, 0.20012525805),
imp_local = c(1555.3450957, 1595.41614044, 763.56392418,
1716.78277731, 1045.72429616, 802.742305814), imp_global = c(594.038972858,
1359.48216529, 1018.89209367, 850.887850177, 1381.3557372,
1714.66351462), teta1c = c(0.033375064111, 0.021482368218,
0.020905367537, 0.006956337817, 0.034913536977, 0.03009770223
), k1c = c(4000921.55552, 4499908.41979, 9764999.26902, 9273400.46159,
6163057.88855, 12338543.5703), k2_2L = c(98633499.5682, 53562216.5496,
51597126.6866, 79496746.0098, 54060378.6334, 88854286.5457
), k2_3L = c(53752551.0262, 125020222.794, 124021434.482,
125817803.431, 75021821.6702, 35160224.288), k2_4L = c(56725106.5978,
126865701.893, 145764489.664, 64837586.8755, 49128911.0832,
70088564.0166), bmaxc = c(3481281.32908, 4393584.00639, 2614830.02391,
3128593.72039, 3179348.29527, 4274637.35956), dfactorc = c(2.5474729895,
2.94296926288, 2.79505551368, 2.47882735165, 2.46407943564,
1.41121223341), amaxc = c(73832.9746763, 99150.5068997, 77165.4338508,
128546.996471, 53819.0447533, 54870.9707106), teta1s = c(0.015467320192,
0.013675755546, 0.031668366149, 0.028898297322, 0.019211801086,
0.013349768955), k1s = c(5049506.54552, 11250622.6842, 13852560.5089,
18813117.5726, 18362782.7372, 14720875.0829), k2_ab1s = c(276542468.441,
275768806.723, 211613299.608, 264475187.749, 162043062.526,
252936228.465), k2_ab2s = c(108971516.033, 114017918.32,
248886114.151, 213529935.615, 236891513.077, 142986118.909
), k2_ab3s = c(33306211.9166, 28220338.4744, 40462423.2281,
23450400.4429, 46044346.1128, 23695405.2598), bmaxab1 = c(4763935.86742,
4297372.01966, 3752983.00638, 4861240.46459, 4269771.8481,
4162098.23435), bmaxab2 = c(1864128.647, 1789714.6047, 2838412.50704,
2122535.96812, 2512362.60884, 1176995.61871), ab1 = c(66.4926766666,
42.7771212442, 45.4212664748, 50.3764074404, 35.4792060556,
34.1116517971), ab2 = c(21.0285105309, 23.5869838719, 18.8524808986,
10.1121885612, 10.9695055644, 12.1154127169), dfactors = c(2.47803921947,
0.874644748155, 0.749837099991, 1.96711589185, 2.5407774352,
1.28554379333), teta1f = c(0.037308451805, 0.035718600749,
0.012495093438, 0.000815957999, 0.002155991091, 0.02579104469
), k1f = c(14790480.9871, 17223538.1853, 19930679.8931, 3524230.46974,
15721827.0137, 13599317.0371), k2f = c(55614283.976, 54695745.7762,
86690362.7036, 99857853.7312, 63119072.711, 37510791.5472
), bmaxf = c(2094770.19484, 3633133.51482, 1361188.05421,
2001027.51219, 2534273.6726, 3765850.14143), dfactorf = c(0.745459795314,
2.04869176933, 0.853221909609, 1.76652410119, 0.523675021418,
1.0808768613), k2b = c(1956.92858062, 1400.78738327, 1771.23607857,
1104.05501369, 1756.6767193, 1509.9294956), amaxb = c(38588.0915097,
35158.1672213, 25711.062782, 21103.1603387, 27230.6973685,
43720.3558889999), dfactorb = c(0.822346959126, 2.34421354848,
0.79990635332, 2.99070447299, 1.76373031599, 1.38640223249
), roti = c(16.1560390049, 12.7223971386, 6.43238062144,
15.882552267, 16.0836252663, 18.2734832893), rotmaxbp = c(0.235615453341,
0.343204895932, 0.370304533553, 0.488746319999, 0.176135112774,
0.46921999001), R = c(0.022186087, 0.023768855, 0.023911029,
0.023935705, 0.023655335, 0.022402726)), .Names = c("fy",
"fu", "E", "eu", "imp_local", "imp_global", "teta1c", "k1c",
"k2_2L", "k2_3L", "k2_4L", "bmaxc", "dfactorc", "amaxc", "teta1s",
"k1s", "k2_ab1s", "k2_ab2s", "k2_ab3s", "bmaxab1", "bmaxab2",
"ab1", "ab2", "dfactors", "teta1f", "k1f", "k2f", "bmaxf", "dfactorf",
"k2b", "amaxb", "dfactorb", "roti", "rotmaxbp", "R"), row.names = c(7L,
8L, 20L, 23L, 28L, 29L), class = "data.frame")
data has no equal rows or zero values or NaNs. Any help is appreciated.
I guess the problem is caused by MaxNWts, which is The maximum allowable number of weights. The value you gave is less than the weights for networks with size larger than 5 units. It should be at least:
MaxNWts = max(nnetGrid$size)*(ncol(trainSet) + output_neron)
+ max(nnetGrid$size) + output_neron
So, in your case, it should be at least MaxNWts = 10 * (34 + 1) + 10 + 1