I have many hierarchical GAMs fitted with the brm() function of the brms package. Some of my GAMs have a 'I type' structure :
y ~ 1 + s(year, by = site, bs = "tp", m = 2) + s(level, bs = "tp") + (1|r|transect),
zi ~ 1 + s(year, by = site, bs = "tp", m = 2) + s(level, bs = "tp") +(1|r|transect)
Where year and level are continuous and site and level are categorical. I also have GAMs with the 'GI type' structure:
y ~ 1 + s(year,bs = "tp")+ s(year, by = site, bs="tp") + s(level, bs = "tp") + s(site, bs = "re") +(1|r|intertidal_transect),
zi ~ 1 + s(year,bs = "tp")+ s(year, by = site, bs="tp") + s(level, bs = "tp") + s(site, bs = "re") +(1|r|intertidal_transect)
I want to calculate the derivative of the splines with 95% confidence intervals. To do so, I extracted posterior predictions of 1000 draws with:
msms <- conditional_smooths(model, spaghetti = T)
Extracting each draw as data a frame and making a data frame for each spline (each site and the global trend for the GS model) with something like that plus some data manipulation to have a draw per column and time in the first column:
f <- attributes(msms_count_4$mu_y: s(YEAR,by=SITE,m=1,bs="tp"))$spaghetti
f_SW <- filter(f, cond__ == "SITE_SW")
f_NW <- filter(f, cond__ == "SITE_NW")
I get these draws for the site-specific posterior prediction of the GI modes:
structure(list(year = c(1982, 1982.38383838384, 1982.76767676768,
1983.15151515152, 1983.53535353535, 1983.91919191919, 1984.30303030303,
1984.68686868687, 1985.07070707071, 1985.45454545455, 1985.83838383838,
1986.22222222222, 1986.60606060606, 1986.9898989899, 1987.37373737374,
1987.75757575758, 1988.14141414141, 1988.52525252525, 1988.90909090909,
1989.29292929293, 1989.67676767677, 1990.06060606061, 1990.44444444444,
1990.82828282828, 1991.21212121212, 1991.59595959596, 1991.9797979798,
1992.36363636364, 1992.74747474747, 1993.13131313131, 1993.51515151515,
1993.89898989899, 1994.28282828283, 1994.66666666667, 1995.05050505051,
1995.43434343434, 1995.81818181818, 1996.20202020202, 1996.58585858586,
1996.9696969697, 1997.35353535354, 1997.73737373737, 1998.12121212121,
1998.50505050505, 1998.88888888889, 1999.27272727273, 1999.65656565657,
2000.0404040404, 2000.42424242424, 2000.80808080808, 2001.19191919192,
2001.57575757576, 2001.9595959596, 2002.34343434343, 2002.72727272727,
2003.11111111111, 2003.49494949495, 2003.87878787879, 2004.26262626263,
2004.64646464646, 2005.0303030303, 2005.41414141414, 2005.79797979798,
2006.18181818182, 2006.56565656566, 2006.94949494949, 2007.33333333333,
2007.71717171717, 2008.10101010101, 2008.48484848485, 2008.86868686869,
2009.25252525253, 2009.63636363636, 2010.0202020202, 2010.40404040404,
2010.78787878788, 2011.17171717172, 2011.55555555556, 2011.93939393939,
2012.32323232323, 2012.70707070707, 2013.09090909091, 2013.47474747475,
2013.85858585859, 2014.24242424242, 2014.62626262626, 2015.0101010101,
2015.39393939394, 2015.77777777778, 2016.16161616162, 2016.54545454545,
2016.92929292929, 2017.31313131313, 2017.69696969697, 2018.08080808081,
2018.46464646465, 2018.84848484848, 2019.23232323232, 2019.61616161616,
2020), `1` = c(0.0903901438415091, 0.102171563196727, 0.113952982551945,
0.129014020398404, 0.149103806598073, 0.169193592797743, 0.191117621275084,
0.213530781026471, 0.235097627348092, 0.252916514194966, 0.270735401041841,
0.282408024975746, 0.289610639518398, 0.296813254061055, 0.2902458944272,
0.283306373329117, 0.271128354990156, 0.249970055666827, 0.228811756343484,
0.199417802015595, 0.167467954755259, 0.134773132363749, 0.0981051092726421,
0.0614370861815133, 0.0261287888687987, -0.00807877805188554,
-0.0422863449725695, -0.0680311864152342, -0.0933058764424676,
-0.11409845247677, -0.1262715785247, -0.13844470457263, -0.140297494457864,
-0.138464449999272, -0.135109382021439, -0.121708958816611, -0.108308535611782,
-0.0914434142869732, -0.0714600646541677, -0.051476715021374,
-0.031052940336138, -0.0105914149321213, 0.00825495044447179,
0.0236018024283193, 0.0389486544121578, 0.0480260680379783, 0.0545492660364586,
0.0600291597020738, 0.0566409665383091, 0.0532527733745464, 0.0456414389225931,
0.0338069631824496, 0.0219724874423062, 0.00546466821953165,
-0.0115929561188369, -0.0290745116954201, -0.0475966257658035,
-0.0661187398361978, -0.0835478313887446, -0.100472451009983,
-0.117150683727973, -0.130954402574742, -0.144758121421519, -0.157185149748638,
-0.168082521942813, -0.178979894136988, -0.189877266331164, -0.200774638525346,
-0.211688584495069, -0.222648937036328, -0.233609289577587, -0.24520100413079,
-0.257121026930209, -0.269072787916266, -0.2815958362619, -0.294118884607533,
-0.306130253196955, -0.317509546793416, -0.32888884038987, -0.337055347879519,
-0.344619457974144, -0.350646697044373, -0.351721796147105, -0.352796895249835,
-0.348754056481851, -0.34172575395595, -0.33449119001513, -0.319624953722208,
-0.304758717429294, -0.287607586291897, -0.267314724743337, -0.247021863194777,
-0.22567487363744, -0.20408985517491, -0.183252248402289, -0.165217435466824,
-0.14718262253136, -0.133864392469005, -0.123622194715213, -0.113379996961422
), `2` = c(0.0281589447753082, 0.0360228847553418, 0.0438868247353754,
0.0542065220785714, 0.0682917140452664, 0.0823769060119616, 0.0990794558874814,
0.116479967872021, 0.133800230892946, 0.150765105646378, 0.167729980399809,
0.18214865686438, 0.194715552755233, 0.207282448646093, 0.212271721394666,
0.217056193517774, 0.218234485136086, 0.213230753031891, 0.208227020927692,
0.195733806596412, 0.18091627019466, 0.164887189511126, 0.142396539324755,
0.11990588913837, 0.0953530003646379, 0.06913068035352, 0.0429083603424022,
0.0177391720305717, -0.00737150896464664, -0.0305577749798716,
-0.050043242956679, -0.069528710933486, -0.0826863717986276,
-0.0935841015524644, -0.103275101184265, -0.105001682010679,
-0.106728262837092, -0.104648921431536, -0.0991442500172005,
-0.0936395786028685, -0.0850102188719535, -0.076113028713903,
-0.0675699556156343, -0.0597941361468887, -0.0520183166781475,
-0.0477026543788269, -0.0447966857411216, -0.0425926203390234,
-0.0463547324396034, -0.050116844540181, -0.0568723517603612,
-0.0666212541001435, -0.0763701564399258, -0.0889238481726288,
-0.101807515128021, -0.114445937172545, -0.12648239443585, -0.138518851699162,
-0.147442691700809, -0.154929939120153, -0.161889368679005, -0.162690923198772,
-0.163492477718539, -0.161194439863554, -0.155452410481069, -0.149710381098584,
-0.143968351716099, -0.138226322333611, -0.131983419557772, -0.124338071280545,
-0.116692723003317, -0.109620461991288, -0.102846206357157, -0.0962364745144016,
-0.0925881709163024, -0.0889398673182031, -0.0871389645791802,
-0.0876201452543117, -0.088101325929443, -0.0916514779767321,
-0.0957770621563005, -0.100373400438514, -0.106486613051482,
-0.112599825664444, -0.118637332497359, -0.124630677625246, -0.130575331305669,
-0.134718401429927, -0.138861471554181, -0.141911821692888, -0.143459681851466,
-0.145007542010045, -0.144765806376384, -0.144119968467057, -0.1432689308015,
-0.141648394050079, -0.140027857298658, -0.138648923662264, -0.137427557274803,
-0.136206190887341), `3` = c(-0.0539709293028476, -0.0537496486563462,
-0.0535283680098447, -0.0522786240455355, -0.0494519029939236,
-0.0466251819423118, -0.0391992632840588, -0.0305468919307008,
-0.0204744488785012, -0.00411311687432633, 0.012248215129849,
0.0323498353609559, 0.0551716652116509, 0.0779934950623591, 0.102357980235101,
0.126764158794924, 0.149058589751514, 0.167732881959705, 0.186407174167906,
0.195190131244194, 0.200903363624378, 0.20390162164128, 0.192420016387331,
0.180938411133375, 0.160428244631795, 0.132609242834518, 0.10479024103724,
0.0674567551735125, 0.0295946868616242, -0.00826605904612237,
-0.0461242618690254, -0.0839824646919283, -0.114022138115743,
-0.141269479611326, -0.16611936316185, -0.175146024275105, -0.184172685388359,
-0.182546736516697, -0.171333438658595, -0.1601201408005, -0.134141380135618,
-0.106897008373012, -0.0773884635227222, -0.0429742103157592,
-0.00855995710881638, 0.0236217573947781, 0.0548939190933054,
0.0849778245533993, 0.104961551986795, 0.124945279420179, 0.137628477017008,
0.143011144777282, 0.148393812537556, 0.142732296530229, 0.135771464785536,
0.127093455002492, 0.114200553670764, 0.101307652339029, 0.0895675478807852,
0.0783595035179952, 0.0677226764786005, 0.0637500515454905, 0.0597774266123786,
0.0598004232658113, 0.0642629994598431, 0.0687255756538748, 0.0731881518479065,
0.0776507280419409, 0.0824611632887457, 0.0882456038833142, 0.0940300444778826,
0.0973139227929918, 0.0992975087227821, 0.100846136701156, 0.0945655215540797,
0.0882849064070032, 0.0774710464544904, 0.0610572958599553, 0.0446435452654299,
0.0210351933610963, -0.00392214628882514, -0.0298341358608949,
-0.0588222196265737, -0.0878103033922348, -0.115712683511179,
-0.142981736502873, -0.170077615674274, -0.190766063494826, -0.211454511315367,
-0.228409357364048, -0.240230500976425, -0.252051644588802, -0.257338274195062,
-0.261149368380579, -0.263958087453723, -0.263007899855467, -0.262057712257211,
-0.260596000222232, -0.258800685293739, -0.257005370365246)), row.names = c(NA,
-100L), class = c("tbl_df", "tbl", "data.frame"))
And this is site-specific with the I model:
structure(list(year = c(1982, 1982.38383838384, 1982.76767676768,
1983.15151515152, 1983.53535353535, 1983.91919191919, 1984.30303030303,
1984.68686868687, 1985.07070707071, 1985.45454545455, 1985.83838383838,
1986.22222222222, 1986.60606060606, 1986.9898989899, 1987.37373737374,
1987.75757575758, 1988.14141414141, 1988.52525252525, 1988.90909090909,
1989.29292929293, 1989.67676767677, 1990.06060606061, 1990.44444444444,
1990.82828282828, 1991.21212121212, 1991.59595959596, 1991.9797979798,
1992.36363636364, 1992.74747474747, 1993.13131313131, 1993.51515151515,
1993.89898989899, 1994.28282828283, 1994.66666666667, 1995.05050505051,
1995.43434343434, 1995.81818181818, 1996.20202020202, 1996.58585858586,
1996.9696969697, 1997.35353535354, 1997.73737373737, 1998.12121212121,
1998.50505050505, 1998.88888888889, 1999.27272727273, 1999.65656565657,
2000.0404040404, 2000.42424242424, 2000.80808080808, 2001.19191919192,
2001.57575757576, 2001.9595959596, 2002.34343434343, 2002.72727272727,
2003.11111111111, 2003.49494949495, 2003.87878787879, 2004.26262626263,
2004.64646464646, 2005.0303030303, 2005.41414141414, 2005.79797979798,
2006.18181818182, 2006.56565656566, 2006.94949494949, 2007.33333333333,
2007.71717171717, 2008.10101010101, 2008.48484848485, 2008.86868686869,
2009.25252525253, 2009.63636363636, 2010.0202020202, 2010.40404040404,
2010.78787878788, 2011.17171717172, 2011.55555555556, 2011.93939393939,
2012.32323232323, 2012.70707070707, 2013.09090909091, 2013.47474747475,
2013.85858585859, 2014.24242424242, 2014.62626262626, 2015.0101010101,
2015.39393939394, 2015.77777777778, 2016.16161616162, 2016.54545454545,
2016.92929292929, 2017.31313131313, 2017.69696969697, 2018.08080808081,
2018.46464646465, 2018.84848484848, 2019.23232323232, 2019.61616161616,
2020), `1` = c(-0.000282416573603693, -0.00271591733455456, -0.00516199276939573,
-0.00763317487364931, -0.0101402837408418, -0.01269188461781,
-0.0152950907197726, -0.0179470899003348, -0.0206404685928068,
-0.0233625756144517, -0.0260886006877375, -0.0287915737231001,
-0.031437529963855, -0.0339868139836993, -0.036399445019488,
-0.0386347491982104, -0.0406519674557182, -0.0424151739981505,
-0.0438952673011647, -0.0450652167105189, -0.045910951429386,
-0.046424850317861, -0.0466048619383107, -0.0464628167632121,
-0.0460130873752293, -0.0452772390725702, -0.0442831064189103,
-0.0430590363960309, -0.0416347932713879, -0.0400404976146581,
-0.0383029757461397, -0.0364440659241958, -0.0344842706185839,
-0.032435721440793, -0.0303060775005323, -0.0281003022315046,
-0.0258161655729922, -0.0234501743005459, -0.0209976917685226,
-0.0184530137867315, -0.0158114123798943, -0.0130733195957459,
-0.0102408732690352, -0.00732122639306472, -0.00432967830237086,
-0.00128303518770019, 0.00179432791763776, 0.00487369521893103,
0.00792538944203596, 0.0109170705038135, 0.0138160888004594,
0.0165938183520947, 0.0192256568027686, 0.0216894482731145, 0.0239755222428691,
0.0260785671039841, 0.0280010274528969, 0.0297588681317019, 0.0313703301860457,
0.0328636713810393, 0.0342733156479352, 0.0356346133760946, 0.0369855582747931,
0.0383644502188473, 0.0398039785036263, 0.0413308230683103, 0.0429713691168698,
0.0447520018532761, 0.0466990099638965, 0.0488283941309507, 0.0511368816299008,
0.0536177739273218, 0.0562486839040952, 0.0589968627764145, 0.0618266563341742,
0.0646931096016521, 0.0675493345034615, 0.0703499950485326, 0.0730515359489744,
0.0756122918458651, 0.0780021869538043, 0.0801958101461476, 0.0821751680228707,
0.0839371812497683, 0.0854814166872423, 0.0868187774408077, 0.0879681955145901,
0.088950472444891, 0.0897926924122859, 0.0905233522093606, 0.091169613195777,
0.0917569952883759, 0.0923098773939077, 0.0928453286771147, 0.0933772651590566,
0.0939160138803542, 0.0944641526576321, 0.0950230238032931, 0.0955907167807124,
0.0961628514579845), `2` = c(-0.0893638592715282, -0.0875317466764431,
-0.0856958539552433, -0.0838523737545187, -0.0819964065877801,
-0.080121614455885, -0.0782214608016236, -0.0762886414261416,
-0.0743154988298857, -0.072294645465396, -0.0702193278776524,
-0.068083006882641, -0.0658811632845179, -0.0636109212944955,
-0.0612704321011602, -0.0588620858596529, -0.0563894802637761,
-0.0538590859979185, -0.0512814301960677, -0.0486677649179678,
-0.0460328295545333, -0.0433930961160075, -0.0407654030739626,
-0.0381674785115223, -0.035617065089155, -0.0331296702143186,
-0.0307188530185536, -0.0283967465072349, -0.0261691074381822,
-0.0240397453046267, -0.0220078796109846, -0.0200657802894222,
-0.0182044146607224, -0.01640816753255, -0.0146579078505463,
-0.0129331035313241, -0.0112095140996819, -0.00946231584134767,
-0.00766748096428189, -0.00580172478182936, -0.00384289673934499,
-0.00177445567984667, 0.000418299642416235, 0.002745604596645,
0.00521044063815636, 0.0078144028256126, 0.0105519025043017,
0.0134132303961086, 0.0163869166070966, 0.0194564825092788, 0.0226045179116448,
0.0258131458624685, 0.0290640226493096, 0.0323394097864151, 0.0356250069233987,
0.0389077235804468, 0.042177410412913, 0.0454290455924111, 0.0486585497860806,
0.0518665074960122, 0.0550563743888019, 0.0582324619655299, 0.0614016542174202,
0.0645713089873207, 0.0677479168036409, 0.0709370392352279, 0.0741441922891273,
0.077374891972387, 0.080634624839924, 0.083925738085757, 0.0872446976679729,
0.0905869194747165, 0.0939429757375555, 0.0973002396652585, 0.10064529044772,
0.103962187748657, 0.107234489740723, 0.110446837286768, 0.113585113417898,
0.116636018636522, 0.119591098484954, 0.122443847080729, 0.12519065289008,
0.127833723088064, 0.13037627958946, 0.13282573317239, 0.135192461376869,
0.137487432629253, 0.13972356747589, 0.141914210342991, 0.144071772628747,
0.146207518903139, 0.148332146487061, 0.150452861424054, 0.152575364192648,
0.154703722761441, 0.156838480820587, 0.158979625635534, 0.161125745294678,
0.163274365617306), `3` = c(0.0731504541724651, 0.0656497283632369,
0.0581727946194858, 0.0507434571968103, 0.0433860093175434, 0.0361253882507971,
0.028985514791221, 0.0219822021524561, 0.015127504610437, 0.00842913405350016,
0.00188472084093085, -0.00450979591794638, -0.0107626558146839,
-0.016885498117432, -0.022889051724781, -0.0287796775341719,
-0.0345623829106009, -0.0402340542407231, -0.0457801116431233,
-0.0511830953713278, -0.0564087665136821, -0.0614145396640546,
-0.0661521388573141, -0.0705591368971187, -0.0745707637001095,
-0.078119434617396, -0.08113511178774, -0.0835500802414415, -0.0853078934650334,
-0.0863556510074676, -0.0866544696675715, -0.0861866898203826,
-0.0849395803451227, -0.0829291581710035, -0.080186801986411,
-0.0767543113394985, -0.072701343191673, -0.0681028188942274,
-0.0630456864047119, -0.0576281222096396, -0.051948432096607,
-0.0461033962912715, -0.0401891661019579, -0.0342918951241707,
-0.0284814954648778, -0.0228237871778161, -0.017358673878057,
-0.0121111901579864, -0.00709633925879568, -0.00230028798482445,
0.00229664856665519, 0.00672807767697247, 0.0110415290060614,
0.0152856516752799, 0.0195119207479934, 0.0237726904150084, 0.0281130007957383,
0.0325651349262517, 0.0371580086377128, 0.0418977759216758, 0.0467765761386907,
0.0517774864620065, 0.0568555866974089, 0.0619599024250542, 0.0670254724478584,
0.0719723546693145, 0.0767198722389097, 0.0811873483061336, 0.0852941659477631,
0.0889660960106614, 0.0921408761030976, 0.0947595169893155, 0.0967865269064204,
0.0982019339356302, 0.0989959634901983, 0.0991927480185298, 0.0988243466086145,
0.0979434195877062, 0.0966262630430475, 0.0949532795258985, 0.0930230203521725,
0.0909412626037069, 0.0888144800044607, 0.0867505119293326, 0.0848563383917516,
0.0832253911771243, 0.0819415064348888, 0.0810803616636772, 0.0806882260354935,
0.0808039343542264, 0.0814442566921647, 0.0825988423321457, 0.0842517372796568,
0.0863593151278179, 0.0888660537100147, 0.0917104862462849, 0.0948183657654647,
0.0981143443513779, 0.101531436833672, 0.105009007121677)), row.names = c(NA,
-100L), class = c("tbl_df", "tbl", "data.frame"))
I put one data frame at a time in this function to calculate the derivative with forward difference for draws of each spline and calculate the estimate and confidence interval of the derivative from the 1000 draws:
Deriv <- function(data){
mty1 <- data[,-1]
#calculate delta y
mty2 <- mty1[-1,]
mty1 <- mty1[-nrow(data),]
m_delta_y <- mty2 - mty1
#calculate delta x
x1 <- data$year[-nrow(data)]
x2 <- data$year[-1]
m_delta_x <- x2-x1
#calculate derivative
mt_deriv <- as.data.frame(matrix(ncol = ncol(mty1), nrow = nrow(m_delta_y)))
for (i in c(1:ncol(m_delta_y))) {
mt_deriv[,i] <- m_delta_y[,i]/m_delta_x
}
deriv <- as.data.frame(matrix(nrow = nrow(mt_deriv), ncol = 5))
names(deriv) <- c("year","y_deriv_estim","sd", "upper_CI", "lower_CI")
deriv$year <- (x1 + m_delta_x/2)
for(i in c(1:nrow(deriv))) {
deriv$y_deriv_estim[i] <- mean(as.numeric(mt_deriv[i,]))
deriv$sd[i] <- sd(as.numeric(mt_deriv[i,]))
deriv$upper_CI[i] <- ci(as.numeric(mt_deriv[i,]))$CI_high
deriv$lower_CI[i] <- ci(as.numeric(mt_deriv[i,]))$CI_low
}
return(deriv)
}
It works super well for the site-specific trends of the I-type model and the global trend of the GI-type model, but the site-specific trend of the GI model is not very smooth, which is a big problem for me because I want to calculate the second derivative and each default in smoothness is a value close to zero.
What would be the cause of the difference in smoothness? What is the solution?
Thank you!
P.S. the required packages to run the derivative calculation function are library(bayestestR) and library(tidyverse)
I state that I never used the root mean squared deviation before.
I'm just tryng to reproduce what I found in an article.
Pratically, I have to quantify the noise of a "methodology" (that is the result of different noises, because of the coupling of two instruments), measuring the noise of the methodology at three different points out of the normal operation, where we know that it measures only the noise.
At the end, in the process that I'm following, you have to calculate the standard deviation among this three points and multiply it for the factor of 1.96 for the 95th percentile confidence interval (and in this way get a Limit of Detection of the method).
The time resolution is 30 minutes, then a standard deviation among three points, then the following three points, and so on. I already have a dataset organized in this way and I have calculated the standard deviation.
Just because I'm following the method of an article, they compare the limit of detection calculated using the standard deviation and the one using the root mean squared deviation.
Because at the end I have to use this limit of detection to filter my data by noise, I want to compare which is the best method for my situation, as they did.
How can I calculate the root mean squared deviation, as well as I calculated the standard deviation? Each three points (three differente columns), then the three following points (same three columns but the following row) and so on?
I already tried with the rmse function of the Metrics package, but the problem is the it require only two values: an actual and a predicted values.
Of course, as well as I did for the standard deviation calculation, I should use aggregate to iterate the function each three columns per each row.
EDIT
I put a piece of the article that I'm following for this method, to try to make you understand better what I would like
"For time lags much different from the true time lag, the standard deviation of the flux provides a measure of the random error affecting the flux"......."Multiplying this measure of the random error by α gives an estimate of the measurement precision at a given confidence interval (α =1.96 for the 95th percentile; α =3 for the 99th percentile) which can be used as the flux limit of detection (LoD) (i.e. LoDσ = α ×REσ )......"A modification of the LoDσ approach is to calculate the random error based on the root mean squared deviation (RMSE) of flux from zero ,which reflects the variability in the crosscovariance function in these regions, but also its offset from zero
I share with you part of the dataset and the code the I tried to use.
Dataset
structure(list(`101_LOD` = c(-0.00647656063436054, 0.00645714072316343,
0.00174533523902105, -0.000354643362187957, -0.000599093190801188,
0.00086188829059792), `101_LOD.1` = c(0.00380625456526623, -0.00398115037246045,
0.00158673927930099, -0.00537583996746438, -0.00280048350643599,
0.00348232298529063), `101_LOD.2` = c(-0.00281100080425964, -0.00335537844222041,
0.00611652518452308, -0.000738139825060029, 0.00485039477849737,
0.00412428118507656), `107_LOD` = c(0.00264717678436649, 0.00339296025595841,
0.00392733001719888, 0.0106686039973083, 0.00886643251752075,
0.0426091484273961), `107_LOD.1` = c(0.000242380702002215, -0.00116108069669281,
0.0119784744970561, 0.00380805756323248, 0.00190407945251567,
0.00199684331869391), `107_LOD.2` = c(-0.0102716279438754, -0.00706528150567528,
-0.0108745954674186, -0.0122962259781756, -0.00590383880635847,
-0.00166664119985051), `111_LOD` = c(-0.00174374098054644, 0.00383270191075735,
-0.00118363208946644, 0.00107908760333878, -9.30127551375776e-05,
-0.00141500588842743), `111_LOD.1` = c(0.000769378300959002,
0.00253820252869653, 0.00110643824418424, -0.000338050323261079,
-0.00313666295753596, 0.0043919374295125), `111_LOD.2` = c(0.000177265973907964,
0.00199829884609846, -0.000490950219515303, -0.00100263695578483,
0.00122606902671889, 0.00934018452187161), `113_LOD` = c(0.000997977666838309,
0.0062400770296875, -0.00153620247996209, 0.00136849054508488,
-0.00145700847633675, -0.000591288575933268), `113_LOD.1` = c(-0.00114161441697546,
0.00152607521404826, 0.000811193628975422, -0.000799514037634276,
-0.000319008435039752, -0.0010086036089075), `113_LOD.2` = c(-0.000722312098377764,
0.00364767954707251, 0.000547744649351312, 0.000352509651080838,
-0.000852173274761947, 0.00360487150682726), `135_LOD` = c(-0.00634051802134062,
0.00426062889500736, 0.00484049067127332, 0.00216220020394825,
0.00165634168942681, -0.00537970105199375), `135_LOD.1` = c(-0.00209301968088832,
0.00535855274344209, -0.00119679744329422, 0.0041216882161451,
0.00512978202611836, 0.0014048506490567), `135_LOD.2` = c(0.00022377545723911,
0.00400550696583795, 0.00198972253447825, 0.00301341644871015,
0.00256802839330668, 0.00946109288597202), `137_LOD` = c(-0.0108508893475138,
-0.0231919072487789, -0.00346546003410657, -0.00154066625155414,
0.0247266017774909, -0.0254464953061609), `137_LOD.1` = c(-0.00363025194918789,
-0.00291104074373261, 0.0024998477144967, 0.000877707284759669,
0.0095477003599792, 0.0501795740749602), `137_LOD.2` = c(0.00930498343499501,
-0.011839104725282, 0.000274929503053888, 0.000715665078729413,
0.0145503185102915, 0.0890428314632625), `149_LOD` = c(-0.000194406250680231,
0.000355157226357547, -0.000353931679163222, 0.000101471293242973,
-0.000429409422518444, 0.000344585379249552), `149_LOD.1` = c(-0.000494386150759807,
0.000384907974061922, 0.000582537329068263, -0.000173285705433721,
-6.92758935962043e-05, 0.00237942557324254), `149_LOD.2` = c(0.000368606958615297,
0.000432568466833549, 3.33092313366271e-05, 0.000715304544370804,
-0.000656902381786168, 0.000855422043674721), `155_LOD` = c(-0.000696168382693618,
-0.000917607266525328, 4.77049670728094e-06, 0.000140297660927979,
-5.99898679530658e-06, 6.71169142984434e-06), `155_LOD.1` = c(-0.000213644203677328,
-3.44396001911029e-07, -0.000524232671878577, -0.000830180665933627,
1.47799998238307e-06, -5.97640014667251e-05), `155_LOD.2` = c(-0.000749882784933487,
0.000345737159390042, -0.00076916001239521, -0.000135205762575321,
-2.55352420251723e-06, -3.07199008030628e-05), `31_LOD` = c(-0.00212014938530172,
0.0247411322547065, -0.00107990654365844, -0.000409195814154659,
-0.00768439381433953, 0.001860128524035), `31_LOD.1` = c(-0.00248488588195854,
-0.011146734518705, -0.000167943850441196, -0.0021998906531997,
0.0166775965182051, -0.0156939303287719), `31_LOD.2` = c(0.00210626277375321,
-0.00327815351414411, -0.00271043947479133, 0.00118991079627845,
-0.00838520090692615, 0.0255825346347586), `33_LOD` = c(0.0335175783154054,
0.0130192144768818, 0.0890608024914352, -0.0142431454793663,
0.00961009674973182, -0.0429774973256228), `33_LOD.1` = c(0.018600175159935,
0.04588362587764, 0.0517479021554752, 0.0453766081395813, -0.0483559729403664,
0.123771869764484), `33_LOD.2` = c(0.01906507758481, -0.00984821669825455,
0.134177176083007, -0.00544320457445977, 0.0516083894733814,
-0.0941500564321804), `39_LOD` = c(-0.148517395684098, -0.21311281527214,
0.112875846920874, -0.134256453140454, 0.0429030528286934, -0.0115143877745049
), `39_LOD.1` = c(-0.0431568202849291, -0.159003698955288, 0.0429009071238143,
-0.126060096927082, -0.078848020069061, -0.0788748111534866),
`39_LOD.2` = c(-0.16276833960171, 0.0236589399437796, 0.0828435027244962,
-0.50219849047847, -0.105196237549017, -0.161206838628339
), `42_LOD` = c(-0.00643926654994104, -0.0069253267922805,
7.63419856289838e-05, -0.0185223126108671, 0.00120855708103566,
-0.00275288147011515), `42_LOD.1` = c(-0.000866169150506504,
-0.00147791175852563, -0.000670310173141084, -0.00757733007180311,
0.0151353172950393, -0.00114193461500327), `42_LOD.2` = c(0.00719928454572906,
0.00311615354837406, 0.00270759483782046, -0.0108062423259522,
0.00158765505419478, -0.0034831499672973), `45_LOD` = c(0.00557787518897268,
0.022337270533665, 0.00657118689440082, -0.00247269227623608,
0.0191646343214611, 0.0233090596023039), `45_LOD.1` = c(-0.0305395220788143,
0.077105031761457, -0.00101713990356452, 0.0147500116150713,
-5.43009569586179e-05, -0.0235006181977403), `45_LOD.2` = c(-0.0216498682456909,
-0.0413426968184435, -0.0210779895848601, -0.0147549519865421,
0.00305229143870313, -0.0483293292336662), `47_LOD` = c(-0.00467568767221499,
-0.0199796182799552, 0.00985966068611855, -0.031010117051163,
0.0319279109813341, 0.0350743318265918), `47_LOD.1` = c(0.00820166533285921,
-0.00748186905620154, -0.010483251821707, -0.00921919551377505,
0.0129546148757833, 0.000223462281435923), `47_LOD.2` = c(0.00172469728530889,
0.0181683409295075, 0.00264937907258855, -0.0569837400476351,
0.00514558635349483, 0.0963339573489031), `59_LOD` = c(-0.00664210061621158,
-0.062069664217766, 0.0104345353700492, 0.0115323589989968,
-0.000701276829098035, -0.0397759501000331), `59_LOD.1` = c(-0.00844888486350536,
0.0207426674766074, -0.0227755432761471, -0.00370561240222376,
0.0152046240483297, -0.0127327412801225), `59_LOD.2` = c(-0.000546590647534814,
0.0178115310450356, 0.00776130696191998, 0.00162470375408126,
-0.036140754156005, 0.0197791914089296), `61_LOD` = c(0.00797528044191513,
-0.00358928087671818, 0.000662870138322471, -0.0412142836466128,
-0.00571822580078707, -0.0333870884803465), `61_LOD.1` = c(0.000105849888219735,
-0.00694734283847093, -0.00656216592134899, 0.00161225110022219,
0.0125744958934939, -0.0178560868664668), `61_LOD.2` = c(0.0049288443167774,
0.0059411543659837, -0.00165857112209555, -0.0093669075333705,
0.00655185371925189, 0.00516436591134869), `69_LOD` = c(0.0140014747729604,
0.0119645827116724, 0.0059880663080946, -0.00339119330845176,
0.00406436116298777, 0.00374425148741196), `69_LOD.1` = c(0.00465076983995792,
0.00664902297016735, -0.00183936649215524, 0.00496509351837152,
-0.0224812403463345, -0.0193087796456654), `69_LOD.2` = c(-0.00934638876711703,
-0.00802183076602164, 0.00406752039394799, -0.000421337136630527,
-0.00406768983408334, -0.0046016148041856), `71_LOD` = c(-0.00206064862123214,
0.0058604630066848, -0.00353440181333921, -0.000305197461077327,
0.00266085011303462, -0.00105635261106644), `71_LOD.1` = c(3.66652318354654e-06,
0.00542612739642576, 0.000860385212430484, 0.00157520645492044,
-0.00280256517377998, -0.00474358065422048), `71_LOD.2` = c(-0.00167098030843413,
0.0059622082597603, -0.00121597491543965, -0.000791592953383716,
-0.0022790991468459, 0.00508978650148816), `75_LOD` = c(NA,
-0.00562613898652477, -0.000103076958936504, -3.76628574664693e-05,
-0.000325767611573817, 0.000117404893823389), `75_LOD.1` = c(NA,
NA, -0.000496324358203359, -0.000517476831074487, -0.00213096062838051,
-0.00111202867609916), `75_LOD.2` = c(NA, NA, -0.000169651845347418,
-4.72864955070539e-05, -0.00144880109085214, 0.00421635976535877
), `79_LOD` = c(-0.0011901810540199, 0.00731686066269579,
0.00538551997145174, -0.00578723012473479, -0.0030246805255648,
0.00146141135533218), `79_LOD.1` = c(-0.00424278455960268,
-0.010593752642875, 0.0065136497427927, -0.00427355522802769,
0.000539975609490915, -0.0206849687839064), `79_LOD.2` = c(-0.00366739576561779,
-0.00374066839898667, -0.00132764684703939, -0.00534145222725701,
0.00920940542227595, -0.0101871763957068), `85_LOD` = c(-0.0120254177480422,
0.00369546541331518, -0.00420718877886963, 0.00414911885475517,
-0.00130381692844529, -0.00812757789798261), `85_LOD.1` = c(-0.00302024868281014,
0.00537704163310547, 0.00184264538884543, -0.00159032685888543,
-0.0062127769817834, 0.00349476605688194), `85_LOD.2` = c(0.0122689407380797,
-0.00509605601025503, -0.00641413996554198, 0.000592176121486696,
0.00131237912317341, -0.00535018996837309), `87_LOD` = c(0.00613621268007298,
0.000410268892659307, -0.00239014321624482, -0.00171179729894864,
-0.00107159765522861, -0.00708388174601732), `87_LOD.1` = c(0.00144787264098156,
-0.0025946273860992, -0.00194897899110034, 0.00157863310440493,
-0.0048913305554607, -0.000585669821053749), `87_LOD.2` = c(-0.00224691693198253,
-0.00277315666829267, 0.00166487067514155, -0.00173757960229744,
-0.00362252480121682, -0.0101992979591839), `93_LOD` = c(-0.0234225447373586,
0.0390095666365413, 0.00606244490932179, 0.0264258422783391,
0.0161211132913951, -0.0617678157059), `93_LOD.1` = c(-0.0124876313221369,
-0.0309636779639578, 0.00610883313140442, -0.0192442672220773,
0.0129557286224975, -0.00869066964782635), `93_LOD.2` = c(-0.0219837540560547,
-0.00521242297372905, 0.0179965615561871, 0.0081370991723329,
1.45427765512579e-06, -0.0111199632179688), `99_LOD` = c(0.00412086456443205,
-0.00259940538393106, 0.00742537463584133, -0.00302091572866969,
-0.00320466045653491, -0.00168702410433936), `99_LOD.1` = c(0.00280546156134205,
-0.00472591065687533, 0.00518402193979284, -0.00130887074314965,
0.00148769905391341, 0.00366250488078969), `99_LOD.2` = c(-0.00240469207099292,
-9.57307699040024e-05, -0.000145493235845501, 0.000667454164326723,
-0.0057445759245933, 0.00433464631989088), H_LOD = c(-6248.9128518109,
-10081.9540490064, -6696.91582671427, -5414.20614601348,
-3933.64339240365, -13153.7509294302), H_LOD.1 = c(-6.2489128518109,
-10.0819540490064, -6.69691582671427, -5.41420614601348,
-3.93364339240365, -13.1537509294302), H_LOD.2 = c(-6248.9128518109,
-10081.9540490064, -6696.91582671427, -5414.20614601348,
-3933.64339240365, -13153.7509294302)), row.names = c(NA,
6L), class = "data.frame")
Code
LOD_rdu=sapply(split.default(LOD_ut, rep(seq((ncol(LOD_ut) / 3)), each = 3)), function(i)
apply(i, 1, rmse))
And I get this error Error in mse(actual, predicted) :
argument "predicted" is missing, with no default
It is difficult to understand precisely what you need, I will try to answer you,
From wikipedia, RMSD serves to compare a dataset generated by a model (the model in your article i guess) to an observed distribution.
From CRAN, RMSE function in modelr package has two arguments, model and data:
modelr::rmse(model = ,data = )
This function will give you the fit of your model to the data. The first argument is a model, meaning you will probably use a function like lm() to generate it. Because you don't detail the model I can't help you more.
The second argument is the dataset, the one you provide is quite disturbing to me. R will expect a tidy set with two columns x the time of the observation and y the value.
you can first what groups your columns:
GRP = sub("[.][0-9]*","",colnames(LOD_ut))
head(GRP)
[1] "101_LOD" "101_LOD" "101_LOD" "107_LOD" "107_LOD" "107_LOD"
you can also use 1:(ncol(LOD_ut)/3), just that the above gives you the groups back.
We can use the above like this:
LOD_ut[,GRP=="101_LOD"]
101_LOD 101_LOD.1 101_LOD.2
1 -0.0064765606 0.003806255 -0.0028110008
2 0.0064571407 -0.003981150 -0.0033553784
3 0.0017453352 0.001586739 0.0061165252
4 -0.0003546434 -0.005375840 -0.0007381398
5 -0.0005990932 -0.002800484 0.0048503948
6 0.0008618883 0.003482323 0.0041242812
This calls out your first three grouped columns. Now if you do apply(..,1,sd) you get the standard deviation, now we just do it over all groups
SD = sapply(unique(GRP),function(i)apply(LOD_ut[,GRP==i],1,sd,na.rm=T))
If you must do RMSE, use predicted to be the mean:
rmse_func = function(x){
Metrics:::rmse(mean(x,na.rm=T),x[!is.na(x)])
}
RMSE = sapply(unique(GRP),function(i){
apply(LOD_ut[,GRP==i],1,rmse_func)
})