I am trying to merge / join two objects together, one an xts object and the other a list. I want to keep the data as an xts object (so merge the second data with the first data I post below). Basically, the Data 2 consists of predicted probabilities from the Data 1.
I have tried creating Data 2 as a data.frame and unlisting it and then merging it by = "index(data)" but it fails.
The dimensions of the data are the following:
dim(dat)
212 4
prob_predictions <- as.data.frame(unlist(probs))
dim(prob_predictions)
191 1
I am also trying subset(index(df1) %in% index(df2))
Data 1:
dat <- structure(c(0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1,
1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1,
0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0,
1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1,
0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0,
0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1,
1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0,
1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1,
1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1,
0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1,
0, 1, 1, 1, 1, 1, 1157.4779907, 1161.2739868, 1165.064978, 1162.5039794,
1152.5029784, 1143.5659789, 1131.9999755, 1115.114978, 1101.3089843,
1088.9449828, 1077.7859863, 1067.7619873, 1059.9439942, 1058.2339967,
1062.8999879, 1065.9739869, 1071.7789918, 1084.3059937, 1094.9029908,
1101.5380006, 1106.801001, 1106.7830079, 1105.7230103, 1105.3360108,
1104.5960206, 1104.4260255, 1106.363025, 1109.688025, 1111.763025,
1113.7510255, 1118.2270265, 1126.2330201, 1131.9140137, 1132.8030029,
1133.0679931, 1131.1919921, 1123.4999877, 1109.6529845, 1098.5239806,
1085.2169738, 1070.7239746, 1058.9449829, 1046.018982, 1037.3779847,
1030.1209901, 1023.8139955, 1019.6099977, 1018.9979982, 1016.8410036,
1018.3280031, 1021.1230043, 1020.8710024, 1024.0220033, 1030.0970094,
1034.7910035, 1040.7799927, 1047.371991, 1052.5719849, 1051.4059814,
1051.5269836, 1052.2799865, 1052.3579894, 1050.2929931, 1046.6079956,
1041.8380005, 1035.4400025, 1032.9650025, 1031.6990113, 1035.0920167,
1041.2500184, 1047.0030091, 1053.8240052, 1062.1109986, 1066.3029907,
1072.0419922, 1077.5289917, 1079.3439941, 1081.8229858, 1083.4049804,
1083.0979735, 1081.2649779, 1079.0049803, 1075.0169798, 1073.8739867,
1074.1959837, 1078.2869871, 1085.5799925, 1091.5880003, 1098.3030028,
1102.7200072, 1106.8830077, 1112.3160033, 1120.2160033, 1126.9150023,
1133.6280028, 1136.9040038, 1140.320996, 1143.1609985, 1146.4569946,
1149.8369995, 1153.297998, 1152.7800049, 1150.6940064, 1147.6130005,
1143.8229981, 1140.1619995, 1135.5619995, 1129.0449951, 1124.4880005,
1122.7390015, 1122.5960084, 1125.3989991, 1128.9430054, 1136.8930054,
1144.3530029, 1151.173999, 1158.3080078, 1167.6070068, 1173.8760009,
1178.3499999, 1183.494995, 1193.018994, 1203.9989867, 1212.4839843,
1217.4519897, 1221.0399902, 1222.8859863, 1225.2989868, 1229.2179931,
1233.0979858, 1235.0249878, 1234.4389893, 1232.6299927, 1230.7069947,
1230.6179932, 1232.1449952, 1234.6289918, 1234.0659913, 1232.0999879,
1229.8249879, 1228.1249879, 1224.0649903, 1220.2369874, 1215.8649903,
1214.1689942, 1214.8499878, 1213.7549926, 1217.246997, 1220.5099975,
1222.2329955, 1221.1559935, 1219.641992, 1216.0529905, 1211.9979856,
1206.3969847, 1199.9509886, 1193.1179808, 1185.7209715, 1179.0619749,
1172.8479857, 1169.2699828, 1167.7309814, 1169.2739868, 1169.3999878,
1170.2729858, 1171.0019897, 1172.7689941, 1174.7, 1176.7939942,
1180.7199952, 1184.6089966, 1187.7949951, 1185.9269897, 1185.0529907,
1182.6129883, 1178.0299805, 1168.1029786, 1156.5709717, 1148.2319702,
1137.9259643, 1130.0429687, 1121.3169677, 1113.2949707, 1107.2059692,
1102.4249755, 1098.911975, 1095.860974, 1097.485974, 1093.6249755,
1086.4079772, 1077.9009704, 1074.0089783, 1072.2119812, 1068.344989,
1062.2379822, 1057.449994, 1061.7179994, 1060.4010072, 1059.8690125,
1061.7240113, 1061.7080201, 1058.3970215, 1057.8680176, 1058.2380127,
1056.2290161, 1053.2240112, 1047.6460082, 1041.7940063, 1040.0410034,
1040.6190063, 1045.6369994, 1050.1010009, 1128.81199335, 1132.72894074524,
1136.05951315045, 1133.75860942184, 1126.33398461976, 1121.97836475121,
1114.98804010824, 1104.18156200269, 1097.85760647863, 1093.48449548066,
1089.54311267298, 1087.65328775174, 1087.83107177539, 1088.49478389202,
1089.82480075944, 1091.87386411569, 1093.27921086657, 1096.47071830785,
1100.97350704044, 1102.6227005604, 1102.82339384036, 1099.6516439508,
1097.67720586025, 1097.0346199688, 1096.8465665432, 1098.06499020575,
1100.72546732901, 1106.37447415482, 1111.91023852103, 1114.41117237617,
1117.75201214987, 1120.7832448975, 1122.20674347869, 1120.07466752834,
1117.94469547802, 1115.36710590868, 1109.05404401262, 1100.7222309638,
1096.19725287201, 1087.52132174134, 1079.62024328978, 1075.06498573838,
1068.53212719186, 1063.28239822121, 1059.64979029538, 1056.61743493392,
1051.89577236878, 1048.42474757175, 1046.82620161254, 1044.26846536373,
1043.14861247194, 1041.82684176033, 1041.46047397363, 1044.57471778567,
1047.19426428227, 1051.05194873158, 1053.13842609047, 1054.50142846281,
1051.21367146635, 1048.35332113622, 1047.56157998039, 1045.89381512512,
1043.17345339892, 1042.61503488473, 1040.8783653719, 1039.24423257458,
1040.09811147224, 1041.49734266536, 1042.67950374485, 1046.49669481677,
1051.36081397707, 1055.8274040745, 1060.05336092454, 1061.8797055984,
1063.77402125569, 1065.18506361229, 1065.29696088731, 1066.65724613614,
1066.94988745651, 1068.16322588922, 1069.21815580453, 1069.83166801363,
1068.92578972661, 1068.81857632408, 1070.35871095988, 1075.03883372561,
1081.15799613269, 1086.72961878672, 1091.50584604513, 1094.58719261226,
1097.09031664919, 1100.22361887307, 1103.94707859945, 1106.8845033995,
1111.19264545669, 1115.10382303224, 1120.66155045774, 1125.17569412844,
1129.42943430668, 1132.1180628489, 1134.34300733948, 1133.43510749763,
1132.00890306928, 1129.33948182459, 1127.89952841272, 1126.73290894484,
1126.80215199772, 1124.52480561698, 1124.50054032013, 1125.99287400392,
1128.66498590831, 1130.96736496466, 1133.15142772993, 1137.94462318423,
1142.78989202382, 1146.70132945013, 1151.6631122644, 1155.87424490588,
1158.8347892958, 1161.3181459343, 1165.5259415596, 1173.38822864916,
1181.98934506353, 1190.21226039081, 1194.81109273454, 1197.18527342649,
1199.09715310016, 1201.08885375729, 1203.47563187564, 1205.40271083986,
1207.24721647416, 1210.57795500043, 1213.91433880992, 1217.26535187564,
1219.20293598272, 1220.70837160341, 1222.74566726023, 1221.94893752116,
1220.47665680486, 1218.61792387106, 1217.58479016906, 1216.06433348629,
1215.23248801141, 1214.29415629603, 1214.89947702975, 1217.46333121739,
1218.76682576811, 1221.6747517902, 1223.33620352446, 1222.84608328404,
1220.3845515427, 1217.15554472911, 1212.80167770729, 1208.2329423066,
1204.08123494406, 1201.53635399701, 1197.84907704491, 1195.70439885016,
1193.49731600729, 1189.93090962564, 1187.19653451844, 1185.66257561192,
1185.77756793459, 1183.90255822654, 1182.89945696687, 1183.06617763669,
1182.8208264332, 1183.94646343956, 1184.8534641596, 1185.84933033488,
1187.20748792203, 1188.70677011993, 1186.75278639422, 1183.95251873763,
1180.62084752452, 1176.63980928409, 1167.55220563799, 1159.14913329151,
1154.47587831137, 1148.54960418648, 1145.95250178776, 1143.07035314131,
1137.82269769928, 1133.88338944221, 1130.76687940009, 1128.18812336199,
1120.80925075608, 1118.40550744598, 1113.93545635589, 1104.9968430839,
1098.44571145686, 1096.38135988954, 1093.86884942387, 1090.43277224064,
1085.63821926534, 1082.79744209722, 1083.80625856415, 1083.6723314628,
1082.00354027587, 1077.87272739245, 1073.8896151646, 1071.01060743464,
1070.41054586943, 1069.56096911996, 1064.84087682282, 1061.11888950636,
1058.87994622004, 1055.5466184848, 1054.88694005768, 1053.88913948076,
1056.96921953021, 1059.95310805114, 77.1228859956622, 81.0362538530292,
78.8404654349793, 46.4728298378735, 33.7103494024937, 38.1634534707235,
33.5520386736078, 26.2429467891094, 30.5979953728327, 30.5979953728327,
31.2223518673486, 33.7665461425831, 36.6962580582319, 37.7398082531122,
40.5860776927095, 41.0627097257687, 40.7556533339627, 52.526559398101,
67.2093345204357, 57.3558861837519, 61.809628052695, 65.0522479908148,
60.3356537763659, 59.9025026642582, 60.6951031882524, 60.0950548232381,
59.3846485649388, 64.6199416069941, 64.1051430716001, 55.6515339908006,
58.7835089189351, 55.0890845598537, 48.1838706704649, 46.0064642542491,
48.4030879681908, 55.5793562399467, 43.3339041496164, 35.5089178322478,
42.157901440901, 32.5975281088021, 28.6602735068277, 26.9110067493817,
23.5372731683978, 27.6575715257538, 27.7636741048428, 28.4241344813052,
27.7437779358905, 33.8748748481366, 38.0173561927228, 37.3614293051309,
46.7027642395441, 51.6960358269122, 46.2684476430283, 67.9712504992444,
67.4307596718059, 65.3539239654913, 69.3859268680975, 65.8884694613497,
48.7463489665683, 48.3776103610145, 58.1513743683333, 53.5784372311078,
46.4319595892114, 54.1515204375632, 48.0571628692748, 48.6571396623733,
52.2995925118996, 44.9774509790143, 45.2591195805464, 48.7943143049565,
56.0044804919092, 57.6982718090011, 75.947686211121, 66.6475291255686,
63.2031704734223, 66.0494138822722, 66.2641524590373, 64.6800962380417,
66.0941051628946, 68.6330617447997, 62.298871330898, 58.4734193157287,
52.329016147723, 43.5650542408412, 44.6973713488007, 56.9666746925596,
61.477502601121, 70.1850582389349, 68.3785649248245, 64.1672444920065,
68.1060250901431, 67.2130080618559, 73.8468747118516, 69.6113702464934,
73.1570958144156, 74.8830412236628, 85.4049570826199, 81.7882678868151,
79.8159292966814, 65.9053697697576, 57.9091367119927, 44.4025529377091,
43.2388424796772, 42.7803356293289, 47.7057738515549, 44.7755737074884,
45.7557906780512, 40.016244653124, 41.4992896665767, 46.6336286507843,
44.3657650232027, 45.4718259236287, 45.2372613787558, 56.9881807801438,
58.8717301068573, 68.2039283244873, 73.5215112680329, 78.8594307629251,
73.0335410836162, 71.845824268758, 73.323376014074, 89.1748677280385,
88.8275948061702, 88.079358554904, 72.9197089804835, 66.5774741060939,
65.5905607795046, 60.3560855296636, 60.5351059532554, 61.4085229097936,
58.076745639994, 63.2173375817626, 67.2733875032827, 68.7459719049055,
59.9037653356146, 44.6491666372171, 40.4929666577831, 30.2655738215587,
36.0522832244009, 40.7505784647263, 45.517250253278, 41.5835266382263,
41.3526668380199, 41.539756712543, 48.3189167794286, 49.8415866657383,
44.5858982397584, 50.0675010891207, 50.5139938354098, 44.9097955003298,
37.4247186375495, 41.3952548987526, 39.6467050713014, 39.3953595896288,
36.8289128008105, 42.8772642627352, 37.5760511024063, 42.0791664435174,
36.4236440580649, 25.1434697637668, 29.0666072154372, 25.3668839063101,
34.1040319281821, 34.1351918720353, 42.138526061446, 49.3942545777117,
53.2282422165058, 60.0907410718325, 59.6946479180297, 56.5126081396889,
64.5584522103826, 61.6638469740838, 48.5567687748239, 50.4491176695018,
45.8595330253583, 39.1134283844586, 22.2017732449298, 24.6509068125481,
33.7409449463083, 27.0354908046699, 36.9033514343542, 31.849732552439,
28.384694400023, 30.2843907497844, 30.2566110685775, 30.1702095862,
28.1229085893699, 39.7891005017724, 37.8236546439287, 33.4844836408483,
42.9231744072258, 49.6425369989148, 43.9761986844232, 44.7318583977582,
37.1424843378588, 40.8120228103859, 50.807226927847, 47.9214803669887,
44.995279725301, 41.3197867616665, 47.7401787161256, 40.9599257198947,
48.8101085201251, 58.7773921954413, 46.8976151314924, 38.7370234461344,
43.0052200556536, 42.7247275761847, 51.7764243779359, 47.5063348907638,
48.4623219235214, 51.3175593621287), class = c("xts", "zoo"), .indexCLASS = "Date", .indexTZ = "UTC", tclass = "Date", tzone = "UTC", src = "yahoo", updated = structure(1545418313.79923, class = c("POSIXct",
"POSIXt")), index = structure(c(1517356800, 1517443200, 1517529600,
1517788800, 1517875200, 1517961600, 1518048000, 1518134400, 1518393600,
1518480000, 1518566400, 1518652800, 1518739200, 1519084800, 1519171200,
1519257600, 1519344000, 1519603200, 1519689600, 1519776000, 1519862400,
1519948800, 1520208000, 1520294400, 1520380800, 1520467200, 1520553600,
1520812800, 1520899200, 1520985600, 1521072000, 1521158400, 1521417600,
1521504000, 1521590400, 1521676800, 1521763200, 1522022400, 1522108800,
1522195200, 1522281600, 1522627200, 1522713600, 1522800000, 1522886400,
1522972800, 1523232000, 1523318400, 1523404800, 1523491200, 1523577600,
1523836800, 1523923200, 1524009600, 1524096000, 1524182400, 1524441600,
1524528000, 1524614400, 1524700800, 1524787200, 1525046400, 1525132800,
1525219200, 1525305600, 1525392000, 1525651200, 1525737600, 1525824000,
1525910400, 1525996800, 1526256000, 1526342400, 1526428800, 1526515200,
1526601600, 1526860800, 1526947200, 1527033600, 1527120000, 1527206400,
1527552000, 1527638400, 1527724800, 1527811200, 1528070400, 1528156800,
1528243200, 1528329600, 1528416000, 1528675200, 1528761600, 1528848000,
1528934400, 1529020800, 1529280000, 1529366400, 1529452800, 1529539200,
1529625600, 1529884800, 1529971200, 1530057600, 1530144000, 1530230400,
1530489600, 1530576000, 1530748800, 1530835200, 1531094400, 1531180800,
1531267200, 1531353600, 1531440000, 1531699200, 1531785600, 1531872000,
1531958400, 1532044800, 1532304000, 1532390400, 1532476800, 1532563200,
1532649600, 1532908800, 1532995200, 1533081600, 1533168000, 1533254400,
1533513600, 1533600000, 1533686400, 1533772800, 1533859200, 1534118400,
1534204800, 1534291200, 1534377600, 1534464000, 1534723200, 1534809600,
1534896000, 1534982400, 1535068800, 1535328000, 1535414400, 1535500800,
1535587200, 1535673600, 1536019200, 1536105600, 1536192000, 1536278400,
1536537600, 1536624000, 1536710400, 1536796800, 1536883200, 1537142400,
1537228800, 1537315200, 1537401600, 1537488000, 1537747200, 1537833600,
1537920000, 1538006400, 1538092800, 1538352000, 1538438400, 1538524800,
1538611200, 1538697600, 1538956800, 1539043200, 1539129600, 1539216000,
1539302400, 1539561600, 1539648000, 1539734400, 1539820800, 1539907200,
1540166400, 1540252800, 1540339200, 1540425600, 1540512000, 1540771200,
1540857600, 1540944000, 1541030400, 1541116800, 1541376000, 1541462400,
1541548800, 1541635200, 1541721600, 1541980800, 1542067200, 1542153600,
1542240000, 1542326400, 1542585600, 1542672000, 1542758400, 1542931200,
1543190400, 1543276800, 1543363200, 1543449600, 1543536000), tzone = "UTC", tclass = "Date"), .Dim = c(212L,
4L), .Dimnames = list(NULL, c("y", "x1", "x2", "x3")))
Data 2:
probs <- list(c(`2018-03-02` = 0.420560689116128), c(`2018-03-05` = 0.711907085203791),
c(`2018-03-06` = 0.751215601254221), c(`2018-03-07` = 0.547750838805509),
c(`2018-03-08` = 0.593534104939348), c(`2018-03-09` = 0.527119339702333),
c(`2018-03-12` = 0.0723935802517959), c(`2018-03-13` = 0.147386150528317),
c(`2018-03-14` = 0.0110208387470279), c(`2018-03-15` = 0.448885780391426),
c(`2018-03-16` = 0.390256452877606), c(`2018-03-19` = 0.455819375756108),
c(`2018-03-20` = 0.373370265212439), c(`2018-03-21` = 0.285897356504093),
c(`2018-03-22` = 0.28174860433147), c(`2018-03-23` = 0.411629206184693),
c(`2018-03-26` = 0.602595990820247), c(`2018-03-27` = 0.914769125894976),
c(`2018-03-28` = 0.567802770307713), c(`2018-03-29` = 1.06192517431111),
c(`2018-04-02` = 1.26133164688967), c(`2018-04-03` = 0.765505424480004),
c(`2018-04-04` = 0.599705106418312), c(`2018-04-05` = 0.783480703900658),
c(`2018-04-06` = 0.58925226449577), c(`2018-04-09` = 0.581888802979559),
c(`2018-04-10` = 0.69731544035042), c(`2018-04-11` = 0.811834990407441),
c(`2018-04-12` = 0.40358830120227), c(`2018-04-13` = 0.732777147245629),
c(`2018-04-16` = 0.351131289433891), c(`2018-04-17` = 0.858564500915946),
c(`2018-04-18` = 0.398843971991605), c(`2018-04-19` = 0.232313329163781),
c(`2018-04-20` = 0.511838387056659), c(`2018-04-23` = 0.470527532699864),
c(`2018-04-24` = 0.75300602307451), c(`2018-04-25` = 1.7150087789751),
c(`2018-04-26` = 0.71773864194899), c(`2018-04-27` = 0.718150438655544),
c(`2018-04-30` = 0.60025651434085), c(`2018-05-01` = 0.516209030172394),
c(`2018-05-02` = 0.597683826516939), c(`2018-05-03` = 0.602186621767186),
c(`2018-05-04` = 0.837921470849356), c(`2018-05-07` = 0.801165076649731),
c(`2018-05-08` = 0.795649126872325), c(`2018-05-09` = 0.669657050275692),
c(`2018-05-10` = 0.432442105016062), c(`2018-05-11` = 0.241267377515004),
c(`2018-05-14` = 0.188621015982501), c(`2018-05-15` = -0.18303573453157),
c(`2018-05-16` = 0.152752056639494), c(`2018-05-17` = 0.514420711650191),
c(`2018-05-18` = 0.156339542628494), c(`2018-05-21` = 0.49960357405928),
c(`2018-05-22` = 0.711680804214282), c(`2018-05-23` = 0.460762316943894),
c(`2018-05-24` = 0.550255124643012), c(`2018-05-25` = 0.988972200664625),
c(`2018-05-29` = 0.846476234446601), c(`2018-05-30` = 0.50919230317936),
c(`2018-05-31` = 0.723166021404194), c(`2018-06-01` = 0.835744277153273),
c(`2018-06-04` = 0.525528654060598), c(`2018-06-05` = 0.620015291063136),
c(`2018-06-06` = 0.176251554900159), c(`2018-06-07` = 0.136995698782173),
c(`2018-06-08` = 0.0522471325520983), c(`2018-06-11` = 0.191934977257687),
c(`2018-06-12` = 0.429006006600039), c(`2018-06-13` = 0.559913925676721),
c(`2018-06-14` = 0.672077559523321), c(`2018-06-15` = 0.76032956322575),
c(`2018-06-18` = 0.92544891831305), c(`2018-06-19` = 0.464788295902493),
c(`2018-06-20` = 0.726125381878034), c(`2018-06-21` = 0.68716814788161),
c(`2018-06-22` = 1.05616788651542), c(`2018-06-25` = 0.868015147566888),
c(`2018-06-26` = 0.644906830154394), c(`2018-06-27` = 0.208974693095105),
c(`2018-06-28` = 0.138107192015791), c(`2018-06-29` = 0.166994363352344),
c(`2018-07-02` = -0.0251122167094578), c(`2018-07-03` = -0.177699586946273),
c(`2018-07-05` = -0.326862202836793), c(`2018-07-06` = 0.0201775156053099),
c(`2018-07-09` = 0.426881152297873), c(`2018-07-10` = 0.744979059597089),
c(`2018-07-11` = 0.41243241951448), c(`2018-07-12` = 0.502955033278997),
c(`2018-07-13` = 0.354841947278703), c(`2018-07-16` = 0.139218821204331),
c(`2018-07-17` = -0.202178270120416), c(`2018-07-18` = 0.128150204047404),
c(`2018-07-19` = -0.0686370513411129), c(`2018-07-20` = -0.0431244526756323),
c(`2018-07-23` = -0.0752612046047743), c(`2018-07-24` = 0.341756550136681),
c(`2018-07-25` = -0.0195230327845111), c(`2018-07-26` = 0.330832699288839),
c(`2018-07-27` = 0.828860453502682), c(`2018-07-30` = 0.709201233874251),
c(`2018-07-31` = 0.140975580247175), c(`2018-08-01` = 0.0294266713325949),
c(`2018-08-02` = -0.0309860327622304), c(`2018-08-03` = 0.328432490391903),
c(`2018-08-06` = 0.246261721432661), c(`2018-08-07` = 0.155023310480069),
c(`2018-08-08` = 0.264199818076836), c(`2018-08-09` = 0.619737926626806),
c(`2018-08-10` = 0.192071322493496), c(`2018-08-13` = 0.0642317026042935),
c(`2018-08-14` = -0.0489313867619856), c(`2018-08-15` = 0.44527540734492),
c(`2018-08-16` = 0.250841276758185), c(`2018-08-17` = 0.166886285921369),
c(`2018-08-20` = 0.14501608322908), c(`2018-08-21` = 0.299868775235037),
c(`2018-08-22` = 0.220950088232105), c(`2018-08-23` = 0.556141681401253),
c(`2018-08-24` = 0.332399640608902), c(`2018-08-27` = 0.31601373354441),
c(`2018-08-28` = 0.536228568950966), c(`2018-08-29` = 0.461926306171486),
c(`2018-08-30` = 0.454570840347704), c(`2018-08-31` = 0.226220781244086),
c(`2018-09-04` = 0.285296899508341), c(`2018-09-05` = 0.444687467396326),
c(`2018-09-06` = 0.576303992437837), c(`2018-09-07` = 0.904209619267055),
c(`2018-09-10` = 1.22770238231023), c(`2018-09-11` = 0.936841682214141),
c(`2018-09-12` = 1.05849630327662), c(`2018-09-13` = 0.637323180199244),
c(`2018-09-14` = 0.932850919631093), c(`2018-09-17` = 0.541733974001656),
c(`2018-09-18` = 0.36306729369803), c(`2018-09-19` = 0.642546689499425),
c(`2018-09-20` = 0.916481643939073), c(`2018-09-21` = 0.614426300203428),
c(`2018-09-24` = 0.508766547811766), c(`2018-09-25` = 0.338267252107242),
c(`2018-09-26` = 0.455102005496381), c(`2018-09-27` = 0.0598758629606586),
c(`2018-09-28` = 0.291185862573928), c(`2018-10-01` = 0.579664627999074),
c(`2018-10-02` = 0.543265160634182), c(`2018-10-03` = 0.518432853545448),
c(`2018-10-04` = 0.541374575635435), c(`2018-10-05` = 1.12461397204987),
c(`2018-10-08` = 0.701101617497031), c(`2018-10-09` = 0.610361538453297),
c(`2018-10-10` = 0.393709447767468), c(`2018-10-11` = 0.326895135027112),
c(`2018-10-12` = 0.9286505914526), c(`2018-10-15` = 1.56529837818139),
c(`2018-10-16` = 0.91095701538889), c(`2018-10-17` = 0.953883434361834),
c(`2018-10-18` = 0.653974845171448), c(`2018-10-19` = 0.417176906768297),
c(`2018-10-22` = 0.586281560385281), c(`2018-10-23` = 0.381443970694156),
c(`2018-10-24` = 0.510316238211796), c(`2018-10-25` = 0.100366157783568),
c(`2018-10-26` = -0.421064599732561), c(`2018-10-29` = 0.656960441730497),
c(`2018-10-30` = 0.494031922279178), c(`2018-10-31` = 0.597524731270041),
c(`2018-11-01` = 0.674525019989957), c(`2018-11-02` = 0.589723916825039),
c(`2018-11-05` = 0.438332501211451), c(`2018-11-06` = 0.699001705555669),
c(`2018-11-07` = 0.63181540579542), c(`2018-11-08` = 0.149239387764848),
c(`2018-11-09` = 0.272625539714475), c(`2018-11-12` = 0.524676229501674),
c(`2018-11-13` = 0.956295144895798), c(`2018-11-14` = 0.386944846002602),
c(`2018-11-15` = 0.582054913107624), c(`2018-11-16` = 0.263083233959702),
c(`2018-11-19` = -0.055491287398159), c(`2018-11-20` = 0.700720198889711),
c(`2018-11-21` = 1.24722134579426), c(`2018-11-23` = 0.88799587539072),
c(`2018-11-26` = 0.61561373514364), c(`2018-11-27` = 0.228747410170771),
c(`2018-11-28` = 0.710493869896566), c(`2018-11-29` = 0.639454784579899),
c(`2018-11-30` = 0.553917730992259))
The following works. You either unlist or create a named vector of probs. Then you create an xts object of this, but you need to use as.Date on the names of the probs (or rownames from the data.frame) otherwise you try to merge a character value with the date values of the dat xts.
library(xts)
# creates a named vector
vals_probs <- sapply(probs, `[`)
# also works
# vals <- data.frame(unlist(probs))
# need to set the names as dates otherwise they are characters
probs_xts <- xts(vals, as.Date(names(vals)))
tail(merge(dat, probs_xts))
y x1 x2 x3 probs_xts
2018-11-23 0 1047.646 1058.880 43.00522 0.8879959
2018-11-26 1 1041.794 1055.547 42.72473 0.6156137
2018-11-27 1 1040.041 1054.887 51.77642 0.2287474
2018-11-28 1 1040.619 1053.889 47.50633 0.7104939
2018-11-29 1 1045.637 1056.969 48.46232 0.6394548
2018-11-30 1 1050.101 1059.953 51.31756 0.5539177
Related
I have the following data frame that I'd like to aggregate all the values in the data column (a table in this nested data frame), and then mutate 6 corresponding variables in the df.test to save these values.
structure(list(Unipro.ID = c("Q13177", "Q8TD19"), Gene.name = c("PAK2",
"NEK9"), Mod.site = c("Q13177_278", "Q8TD19_81"), Mod.site2 = c("PAK2_278",
"NEK9_81"), `mod.or.not(Y/N)` = c("Y", "Y"), `kinase.or.not(Y/N)` = c("Y",
"Y"), data = list(structure(list(`First Scan` = c(18638, 18640,
18699, 18889, 18890, 19117, 19277, 19387, 19395, 19495, 19502,
19576, 19650, 19726, 19802, 19879, 19956, 20034, 20111, 20189,
20266, 20342, 20420, 20497, 20574, 20670, 20727, 20803, 20918,
20956, 21033, 21147, 21185, 21263, 21377, 21416, 21492, 21607,
21646, 21762, 21840, 21879, 21994, 22072, 22113, 22240, 22298,
22356, 22473, 22530, 22703, 22760, 22820, 22936, 23012, 23165,
23243, 23281, 23396, 23472, 23572, 23590, 23665, 23741, 23760,
23894, 23970, 23990, 24122, 24200, 24372, 24428, 24568, 24605,
24661, 24702, 24860, 25051, 25116, 25457, 25459, 25470, 25522,
25806, 26042, 26288, 26516, 27123, 27344, 27573, 27867, 28024,
28224, 29830, 30443, 32854), TMT126 = c(0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 683.603289421257, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 1058.63721546934, 5105.06075558397, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 786.877999584963, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 1158.61443528477, 0, 0, 0, 0, 0, 790.552967308115,
1769.24251582031, 0, 0, 4230.5333458008, 0, 754.027149866911,
0, 0, 937.667214973757, 0, 0, 16065.4787548108, 0, 1033.79325384522,
0, 1545.17012178734, 22490.6473715619, 0, 12391.863457809, 1694.80635977683,
1669.92554702357, 2915.19207246094, 1305.76068893333, 1218.70367092158,
1724.48555844726, 1220.71902820389, 1142.29342043404, 15089.1510828024,
786.642299340822, 0, 0, 803.014098225646), TMT127N = c(0, 0,
0, 0, 0, 0, 0, 865.592199462889, 0, 0, 959.490090490725, 0, 0,
0, 3632.481907304, 0, 0, 0, 0, 0, 2281.36683418381, 0, 0, 1148.88409380685,
2282.96815473633, 0, 737.964767776176, 0, 0, 714.336386722035,
1336.78857436523, 0, 992.266982382272, 0, 8167.51299697265, 1481.83583737793,
0, 0, 0, 0, 0, 994.583403515622, 0, 3822.09582135578, 730.363546764861,
0, 0, 1601.84151123047, 0, 0, 0, 0, 1248.0926793457, 0, 2349.57136347657,
0, 925.784469760195, 0, 0, 0, 0, 0, 0, 0, 2869.82125708008, 779.87640303955,
0, 4092.33059296876, 0, 0, 0, 1133.40195924235, 0, 0, 0, 15684.5063332031,
0, 0, 0, 2583.52765913085, 24540.0170683595, 1238.62926386719,
11428.895775, 1483.56418256835, 1332.67443391114, 1825.25018671875,
1123.61925747071, 1030.31205838623, 1533.51140185547, 2153.17955595703,
1008.89100509033, 12831.9995203125, 1684.62781442872, 0, 1136.51165412598,
0), TMT128C = c(1611.91088437501, 1786.68805515069, 0, 1823.53765170799,
874.032758349609, 0, 0, 1177.50190689896, 646.466474431654, 1499.41582602539,
1255.4240426748, 1783.93500465848, 5996.15327152243, 8208.12153691405,
21348.7184882813, 33456.1981237878, 46046.4520991817, 5945.01241588586,
6798.69927773436, 18048.0166453125, 3490.51877753905, 7744.45986702618,
11529.8069519532, 3396.17701757812, 4651.27894687501, 14325.2306858641,
1561.40410561524, 0, 9592.47815625001, 2089.33504892578, 3367.76334082031,
10669.6799841273, 1439.18170268554, 0, 9310.75753007808, 1700.90959455201,
0, 6206.48464218753, 3397.72012818408, 0, 7079.64974935495, 1845.17109756826,
1177.34360229492, 6097.26335625001, 1754.74566738282, 1392.17984296875,
7026.71143068691, 2193.1615467585, 2836.98221749547, 6301.54294276034,
0, 3623.93436796874, 1312.70517940113, 0, 4724.16524276671, 0,
3946.52595689374, 2970.83282951206, 865.842519287108, 3283.01703994418,
0, 0, 1262.2417984723, 3482.3909258789, 4262.98461721732, 725.868942626953,
3139.75638657914, 6559.00482506506, 719.443235961914, 1980.26911301631,
735.412520327744, 2538.58699423828, 0, 0, 3345.73545624628, 17476.7195602775,
0, 3061.52193035407, 0, 3370.09478829905, 23476.7103351433, 1087.0128899699,
11912.6099077086, 1957.73038989257, 3158.21345361328, 3399.21823401529,
2593.01804501953, 934.40066220703, 2667.04373535038, 2390.35165488282,
1286.71902949219, 13590.0106023424, 1817.04336728435, 720.287557226125,
916.984646430313, 0), TMT129N = c(5236.22537109374, 5253.53062265625,
2574.52071173515, 6331.35667763673, 2909.4254578125, 2260.77584003907,
2299.19597588428, 2146.42980395508, 1524.82949099121, 4594.55742216798,
2945.06894956055, 3353.94465205077, 17265.8083640626, 58543.518953906,
42651.7719375001, 85288.5246984373, 147283.5420375, 16268.584838086,
31273.8547945313, 64984.9260843748, 15376.6111124999, 18807.9172359374,
41234.0440359375, 7038.37527421877, 14994.4310613281, 35007.7380726561,
5527.67209306639, 8827.66559818178, 32485.2518203124, 6771.8957270508,
10744.4285308594, 21706.4754328126, 7351.69939775389, 7286.74206159669,
31413.8810906249, 5402.31355253905, 7530.87224590352, 22346.2738640626,
7845.50157832029, 8598.84479102795, 13646.5870857421, 3948.41240126953,
4709.55193242188, 14502.506803125, 5527.37162548826, 4609.58657929687,
17346.4145701172, 3806.28572124023, 6695.23604707033, 10312.3288382812,
5030.34046109047, 12238.5130183594, 4008.11956391603, 3366.81581267473,
10163.4030287109, 5657.196356792, 9102.44150449222, 3997.46163222657,
3410.17096083986, 7459.02988359372, 2817.01574699993, 3228.9395395409,
1668.4944914795, 8344.1621935547, 6186.76663623048, 4037.21344028321,
4953.59359042969, 6163.53660527345, 2403.32564355468, 7143.26629951174,
1674.84291965332, 6149.94359912111, 2371.20030146484, 2708.46439392749,
8739.23083300782, 17708.4475347656, 2940.52380873895, 6755.28018515623,
2318.903026372, 5194.7135689453, 37003.6496414061, 1967.3172459961,
11835.060703125, 6357.79309687501, 6929.07336386721, 4922.18842499999,
5119.39548720702, 5098.90275791014, 5251.26713525389, 3925.45210825196,
3679.82248959962, 14610.4555587891, 3124.13869614258, 1892.91264873047,
2424.18607792969, 1217.02803508301), TMT130C = c(8534.51942065459,
7915.5220132809, 5427.32003829632, 6570.76122664215, 4059.66840776324,
6061.80182135341, 4680.93892861494, 3924.40329255471, 1651.02954082353,
9963.97493859979, 2095.70639501953, 6791.52962729408, 29089.1005494669,
84188.5986999335, 79102.1477485965, 111076.559352845, 228355.265937366,
31033.9549731673, 34443.9266034838, 100401.13011289, 20754.0266423186,
29334.7579023722, 70572.5366141732, 12419.2333360566, 17842.2049581652,
61518.6150214895, 10110.7438562966, 17502.4708992952, 61357.4708251869,
9995.86560411634, 17530.0854369149, 53203.79872126, 11560.0754183956,
13608.3534926174, 46828.7269257867, 11019.0036476036, 12751.4728461078,
26876.5493340097, 15817.3356058367, 13322.0303678209, 28346.2564980389,
6207.28749858749, 9509.23448457102, 36589.1961503305, 10263.6194964664,
7564.4130620443, 29223.5021208951, 7015.27604618795, 8985.64276558643,
18886.2813530505, 7104.5605795249, 17804.4468985063, 6812.45760606556,
6606.50436956103, 18488.5028791766, 5863.36827860186, 13255.0296116594,
6453.52502342659, 5515.76995189115, 13991.8058338637, 3809.92839574631,
5061.53991882257, 5991.94758913396, 11334.5748098943, 6206.71330898436,
5219.26272399663, 10963.7722168793, 7489.68230742191, 3361.11015884112,
9301.91002246388, 2802.03791217282, 10320.7319848397, 2509.93049810913,
3062.67670544402, 12115.818468418, 22596.7672476562, 5210.67744771832,
12678.5528095273, 3674.33047767757, 10158.3722420368, 38620.8279703125,
4299.66130273203, 16002.3112734375, 9597.19786319193, 10816.7672483378,
10449.5441895035, 8882.17509875803, 5726.84640941102, 8043.21098886215,
7002.83669129089, 4997.59224291582, 17214.8268832032, 4203.51644882184,
2044.40880964859, 2570.59274774035, 1621.14734118819), TMT131 = c(9418.17036445313,
8878.68274218749, 8378.77222617187, 5645.64330000002, 4753.99889121092,
4420.32600410157, 5180.99550117188, 4787.09710488281, 4537.51669335937,
11573.2264957031, 3011.21359365234, 6855.67281093751, 33135.919678125,
78639.2005781249, 89222.1922968751, 117395.8545375, 221719.7701125,
31420.3548656251, 46548.5491968752, 99822.2995687501, 21221.19759375,
31903.8316523437, 65884.2084140624, 12792.6073300781, 32494.3474710937,
55493.9358187499, 10832.2051148437, 16887.821878125, 56011.2740859373,
11016.5575394531, 15928.4518980469, 45726.2466890625, 10025.5855992188,
21737.7840796876, 59706.8427656249, 9716.2620714844, 18071.6204882813,
35495.48536875, 19939.4042976562, 13032.988340625, 32501.3787140625,
7199.69156542971, 10150.1181632812, 40466.1293437499, 10645.1514984375,
11109.5601890625, 28522.8374835938, 5834.91936972658, 6796.13434628906,
23190.480225, 9266.24059453124, 29075.5950890626, 7621.3969927734,
9322.90506210938, 22800.4644656249, 7069.82396484372, 15978.9590683593,
6530.85736874999, 6011.4814611328, 17323.3141382813, 4554.70168183593,
5415.33720234375, 6123.79340332032, 14385.484575, 10146.5210988281,
3506.75934082032, 13551.8667152343, 9222.76466718747, 5501.55972480469,
13922.8457027344, 4250.15196503907, 9017.17755468752, 3575.14872451172,
5562.49420546876, 17743.048228125, 24449.2857210939, 5704.12143984373,
15124.9334800781, 5079.61518925782, 10921.1189097656, 35210.8651640626,
4445.87090449217, 15901.6300136718, 11364.1122761719, 9222.57776601566,
8388.0708205078, 9823.76574609376, 6197.90128417968, 8764.95285703127,
9621.21499453124, 6218.11219218749, 15561.3999234375, 4677.02849882812,
3778.64337919922, 2264.6449546875, 2498.85718242187)), class = c("tbl_df",
"tbl", "data.frame"), row.names = c(NA, -96L)), structure(list(
`First Scan` = c(16668, 16732, 16751, 16894, 16948, 16970,
17120, 17177, 17210, 17367, 17403, 17406, 17633, 17638, 17833,
17917, 18052, 18090, 18149, 18285, 18338, 18376, 18397, 18504,
18561, 18605, 18642, 18662, 18702, 18753, 18755, 18792, 18932,
18983, 19022, 19210, 19238, 19249, 19438, 19586, 19669, 19707,
19742, 19898, 19937, 20040, 20131, 20172, 20302, 20361, 20405,
20613, 20633, 20790, 20962, 21004, 21148, 21187, 21382, 21454,
21455, 21585, 21726, 21729, 21966, 22000, 22197, 22267, 22585,
22789, 23382, 23725), TMT126 = c(1882.9047259105, 3449.99254746093,
1024.2977771073, 5158.15467082831, 15759.4817531249, 5163.23808916981,
23084.5250109376, 11158.1465572494, 2542.16341682018, 6236.75949960936,
2540.56668486328, 4643.57579941405, 3676.30715654298, 2933.73809824218,
0, 3710.6456292118, 4085.0294783203, 675.688557714844, 1950.23111088867,
2205.57922470703, 8296.83276855471, 2672.59435136718, 3034.29849873047,
45624.8063390625, 22890.978928125, 3676.17324550782, 804.357999226392,
2339.07756328126, 2307.76013144531, 21624.511528125, 1518.1371897772,
10635.9306925781, 2680.27191738282, 11041.5375597656, 7065.95542382811,
8761.31611523439, 1801.56362783203, 7668.99884179686, 7505.24644894218,
1129.2046734375, 5552.81345566407, 6607.18650630685, 2071.91356259765,
5463.23402109373, 5149.03205670459, 1219.86494693985, 4910.47049139314,
3155.11395608923, 1398.27528804432, 6326.38970151936, 3541.72186757812,
4198.17516679688, 1440.07868518477, 0, 3465.05218769531,
961.89374368008, 2280.46133583984, 2356.01639560548, 0, 2526.26822314453,
1853.50988980811, 1310.57137792969, 1419.80935503652, 1688.71382270508,
1531.11097573243, 1821.7144977539, 1352.28257709961, 2137.31568546879,
1951.55034804646, 1311.16770682422, 0, 1720.31448781044),
TMT127N = c(2771.68523671875, 5795.16860742187, 1532.51465844727,
4997.19921035158, 25241.5731433593, 7411.63585078126, 35598.7388882813,
11632.0612353516, 3981.85727929688, 8431.46010673831, 4482.49877431641,
6676.69583173827, 4001.42575561522, 3282.20197690431, 1573.56998100586,
3935.30607905274, 7116.32664960937, 2307.64589311523, 4102.16486528321,
4772.48413124998, 14428.697888086, 3332.28787353515, 5485.6754455078,
91312.7273437499, 37781.7556101563, 4225.51941313476, 1839.82785454101,
4019.30935473634, 3973.05389443358, 32996.0530066406, 2037.37071027832,
20023.8569953125, 3530.19987890626, 16245.3700406251, 11744.1997277344,
14524.1289123047, 2272.03076718749, 7951.62546621093, 11169.627975,
2088.36168669433, 8426.40678837892, 5546.88783515623, 825.817551295166,
11964.5538246094, 5850.03650859376, 1951.7206411377, 6372.5401716797,
2720.9217121582, 2928.83477036133, 7842.02046181641, 4470.19431181639,
5518.54197597655, 4886.56620761718, 2598.45832382813, 5150.38777236328,
1764.69598916016, 4431.15874072266, 5851.29773701172, 3535.65924873047,
4506.44162783202, 3948.30497885742, 3474.47286108398, 4899.0403397461,
3668.68391484374, 1715.68195283204, 1517.17190141602, 2988.61810048828,
3049.25398110351, 3346.69140673828, 1629.18274760742, 1652.68488032226,
3111.11825932616), TMT128C = c(3546.93682792969, 5443.04733562549,
2406.94690743869, 7409.37070198117, 24566.1728096405, 7605.80923058386,
37319.8058793624, 13554.5726592211, 4322.44607961136, 10647.2819212435,
5886.92810075651, 8753.17848761496, 6553.53110235196, 4646.29586668501,
1968.9492260545, 4154.15849160926, 7670.78133562564, 2768.92031395084,
3603.54312466775, 4079.51357784559, 16511.9767178725, 3885.32503013637,
4789.39730709742, 75477.2031668271, 39696.3689381062, 6235.92097131154,
2324.05125227432, 4163.9149784173, 5315.6094992347, 31155.5901563382,
1816.81292346028, 16308.3860112984, 4287.50902664535, 15935.5273428347,
12014.8172808907, 13939.1329255981, 3921.1247165634, 11449.7683673977,
13003.5107817882, 2150.04455439802, 10889.4027696227, 6249.74721525204,
3815.90910915446, 9774.84044695728, 5968.13675351661, 3450.75965045113,
6496.88961226217, 4185.18567482653, 3493.2378779832, 6249.25306422371,
4966.6136218745, 7318.69851904571, 4807.8110810617, 1540.76095735796,
6832.23484476677, 2965.60747444571, 4139.87531020355, 4987.87787026309,
3610.04293593895, 5899.23647575952, 5796.31857744029, 3008.6643611543,
5433.73871616973, 2796.2558469155, 2533.88330506493, 3404.51135442734,
1989.89578135028, 3963.71640004554, 1855.68417729492, 1858.78970920819,
1801.24628626465, 1521.81316450196), TMT129N = c(5603.3715375,
7466.48746787111, 5737.87805507811, 10125.7479275391, 27223.0979882812,
11989.9738019531, 44015.6930296875, 17091.8407880859, 7938.03088417966,
12459.8042378906, 7051.86322177736, 10014.4530556641, 7992.80055556639,
5937.58603710939, 3499.28743828124, 8939.26554726561, 11082.9714445312,
3852.90205781249, 4749.96272050781, 6339.30068320313, 18824.0227183594,
5960.32649501952, 4809.15115634765, 84223.8693140624, 36909.3952734375,
7050.83312226562, 2210.35349326172, 5805.73487548829, 6812.68262490236,
29434.3397519532, 2379.38804296874, 20404.2909726562, 5217.36262587891,
19563.0384855469, 13420.0282283203, 12340.821178125, 3700.34673647461,
10695.0951164062, 15392.6850773437, 2774.59824887696, 12177.5412128907,
7713.37043554689, 3815.38668457032, 11456.5398427734, 11993.45544375,
3913.54135283203, 9596.82833613281, 5349.46382695312, 3289.20360688477,
8314.06658349611, 5784.25196894531, 7882.8373013672, 5610.97252675783,
3071.51274536133, 7498.057575293, 3661.76029086913, 6717.39973330079,
6506.75830341797, 4466.70268945312, 5969.45083388673, 7538.86863662107,
3945.00036079102, 7081.96355947265, 3515.28444770509, 4725.6043602539,
4937.22756269533, 3392.35801362304, 5545.30827919922, 4755.80240244142,
3465.95602353517, 2446.60899624023, 3763.70442260743), TMT130C = c(10983.9436123188,
9211.23508829853, 10557.3906937124, 10405.446046875, 28290.6707273437,
16687.4414721715, 47013.3588375, 24663.9287051498, 10819.8059091653,
12112.6441605469, 12221.0968142757, 14143.6632554762, 8870.24921835937,
10222.1638647599, 4387.39878905819, 7903.13580132335, 12659.3879214159,
4977.95176757538, 6713.09197134976, 8619.08616316083, 25601.7183693629,
9371.93738344518, 8431.95007587771, 96391.5124218752, 44929.6342171874,
7378.82902968752, 3945.11645284669, 8071.18750848737, 7751.84701707912,
38104.5129609375, 3638.03642401144, 20403.32641875, 7808.49545553976,
23539.4361984374, 18752.01165285, 20811.394168736, 5824.4538993184,
18893.4697542139, 12674.1161636719, 2807.67847851562, 12264.333778125,
11014.4240599255, 6251.73090996096, 13488.8996818318, 15964.1844357857,
5828.08199702709, 11239.0816570313, 8253.48396721728, 4132.48828007812,
13706.3856985815, 7642.39674902346, 8508.96873105467, 8428.07027860162,
5015.03055391062, 9927.36947998573, 4157.77188427735, 6591.46973203123,
10252.0377530773, 5571.05266915471, 8466.28209717723, 9830.67984953953,
6387.25448143305, 8227.90628531475, 6429.95347392001, 5927.99930589914,
6179.43049557376, 4779.11406556715, 5853.8402419922, 5984.37583438957,
2653.83845332031, 4653.86852737383, 4421.67880346553), TMT131 = c(13383.1659585937,
10027.0432195313, 16206.8438039063, 16355.4699128906, 29514.0965624999,
32393.8092585938, 53039.4243328127, 23782.9527632812, 21228.6799054688,
15934.2437460937, 11102.0789214844, 15037.0606933593, 13194.4500703125,
9207.67683515628, 6735.74908359374, 12899.2203527343, 13469.5999195312,
6745.45593691405, 7115.74944609379, 11411.3147414063, 27412.2978679687,
11739.3367394532, 8886.12224414064, 96689.4600375001, 45453.11203125,
7980.59650781252, 6961.24482539061, 10250.2241015625, 13306.6513335937,
39924.4417171876, 6483.56406328127, 22704.3346148437, 9771.777984375,
24691.8792656249, 18723.0097054687, 16477.3002410156, 7680.46611621096,
17120.4574535157, 18478.2819375, 4914.95838925784, 14805.4535964844,
11972.9694691407, 6757.70631679689, 12590.9775105469, 19122.4749375,
7233.17976562498, 14106.4787144532, 11431.40191875, 7873.65249257815,
16542.6268992187, 8767.54336992188, 11536.5155554687, 10534.8182027344,
6824.66757714841, 13787.927071875, 7553.96639121096, 7917.68129238283,
12813.6645140625, 7735.42080351564, 7866.51996796874, 12833.8237371094,
7160.45798144533, 11867.8380820313, 7840.4843214844, 6481.4439357422,
8916.9703347656, 7413.33213457034, 12091.0962304687, 10392.2846542969,
5704.91811914064, 5982.57808242186, 6364.45633183592)), class = c("tbl_df",
"tbl", "data.frame"), row.names = c(NA, -72L))), `# of PSMs` = c(96,
72)), row.names = c(NA, -2L), groups = structure(list(Unipro.ID = c("Q13177",
"Q8TD19"), Gene.name = c("PAK2", "NEK9"), Mod.site = c("Q13177_278",
"Q8TD19_81"), Mod.site2 = c("PAK2_278", "NEK9_81"), `mod.or.not(Y/N)` = c("Y",
"Y"), `kinase.or.not(Y/N)` = c("Y", "Y"), .rows = structure(list(
1L, 2L), ptype = integer(0), class = c("vctrs_list_of", "vctrs_vctr",
"list"))), row.names = c(NA, -2L), class = c("tbl_df", "tbl",
"data.frame"), .drop = TRUE), class = c("grouped_df", "tbl_df",
"tbl", "data.frame"))
Here is the final data frame that I want (of course, the NA would be replaced by the sum values).
Besides this, how to call pivot_longer for 6 TMT columns in all nested tables?
Does anyone know how to do that? Thanks for any help!
df %>%
mutate(data = map(data, ~summarise(.x, across(where(is.numeric), sum)))) %>%
unnest_wider(data) %>%
select(-`First Scan`) %>%
ungroup()
# A tibble: 2 x 13
Unipro.ID Gene.name Mod.site Mod.site2 `mod.or.not(Y/N)` `kinase.or.not(Y/N)` TMT126
<chr> <chr> <chr> <chr> <chr> <chr> <dbl>
1 Q13177 PAK2 Q13177_278 PAK2_278 Y Y 1.00e5
2 Q8TD19 NEK9 Q8TD19_81 NEK9_81 Y Y 3.56e5
# ... with 6 more variables: TMT127N <dbl>, TMT128C <dbl>, TMT129N <dbl>,
# TMT130C <dbl>, TMT131 <dbl>, # of PSMs <dbl>
You can do summarise(across(starts_with("TMT"), sum)) after calling unnest(data):
df <- structure(list(Unipro.ID = c("Q13177", "Q8TD19"), Gene.name = c(
"PAK2",
"NEK9"
), Mod.site = c("Q13177_278", "Q8TD19_81"), Mod.site2 = c(
"PAK2_278",
"NEK9_81"
), `mod.or.not(Y/N)` = c("Y", "Y"), `kinase.or.not(Y/N)` = c(
"Y",
"Y"
), data = list(structure(list(`First Scan` = c(
18638, 18640,
18699, 18889, 18890, 19117, 19277, 19387, 19395, 19495, 19502,
19576, 19650, 19726, 19802, 19879, 19956, 20034, 20111, 20189,
20266, 20342, 20420, 20497, 20574, 20670, 20727, 20803, 20918,
20956, 21033, 21147, 21185, 21263, 21377, 21416, 21492, 21607,
21646, 21762, 21840, 21879, 21994, 22072, 22113, 22240, 22298,
22356, 22473, 22530, 22703, 22760, 22820, 22936, 23012, 23165,
23243, 23281, 23396, 23472, 23572, 23590, 23665, 23741, 23760,
23894, 23970, 23990, 24122, 24200, 24372, 24428, 24568, 24605,
24661, 24702, 24860, 25051, 25116, 25457, 25459, 25470, 25522,
25806, 26042, 26288, 26516, 27123, 27344, 27573, 27867, 28024,
28224, 29830, 30443, 32854
), TMT126 = c(
0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 683.603289421257, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 1058.63721546934, 5105.06075558397, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 786.877999584963, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 1158.61443528477, 0, 0, 0, 0, 0, 790.552967308115,
1769.24251582031, 0, 0, 4230.5333458008, 0, 754.027149866911,
0, 0, 937.667214973757, 0, 0, 16065.4787548108, 0, 1033.79325384522,
0, 1545.17012178734, 22490.6473715619, 0, 12391.863457809, 1694.80635977683,
1669.92554702357, 2915.19207246094, 1305.76068893333, 1218.70367092158,
1724.48555844726, 1220.71902820389, 1142.29342043404, 15089.1510828024,
786.642299340822, 0, 0, 803.014098225646
), TMT127N = c(
0, 0,
0, 0, 0, 0, 0, 865.592199462889, 0, 0, 959.490090490725, 0, 0,
0, 3632.481907304, 0, 0, 0, 0, 0, 2281.36683418381, 0, 0, 1148.88409380685,
2282.96815473633, 0, 737.964767776176, 0, 0, 714.336386722035,
1336.78857436523, 0, 992.266982382272, 0, 8167.51299697265, 1481.83583737793,
0, 0, 0, 0, 0, 994.583403515622, 0, 3822.09582135578, 730.363546764861,
0, 0, 1601.84151123047, 0, 0, 0, 0, 1248.0926793457, 0, 2349.57136347657,
0, 925.784469760195, 0, 0, 0, 0, 0, 0, 0, 2869.82125708008, 779.87640303955,
0, 4092.33059296876, 0, 0, 0, 1133.40195924235, 0, 0, 0, 15684.5063332031,
0, 0, 0, 2583.52765913085, 24540.0170683595, 1238.62926386719,
11428.895775, 1483.56418256835, 1332.67443391114, 1825.25018671875,
1123.61925747071, 1030.31205838623, 1533.51140185547, 2153.17955595703,
1008.89100509033, 12831.9995203125, 1684.62781442872, 0, 1136.51165412598,
0
), TMT128C = c(
1611.91088437501, 1786.68805515069, 0, 1823.53765170799,
874.032758349609, 0, 0, 1177.50190689896, 646.466474431654, 1499.41582602539,
1255.4240426748, 1783.93500465848, 5996.15327152243, 8208.12153691405,
21348.7184882813, 33456.1981237878, 46046.4520991817, 5945.01241588586,
6798.69927773436, 18048.0166453125, 3490.51877753905, 7744.45986702618,
11529.8069519532, 3396.17701757812, 4651.27894687501, 14325.2306858641,
1561.40410561524, 0, 9592.47815625001, 2089.33504892578, 3367.76334082031,
10669.6799841273, 1439.18170268554, 0, 9310.75753007808, 1700.90959455201,
0, 6206.48464218753, 3397.72012818408, 0, 7079.64974935495, 1845.17109756826,
1177.34360229492, 6097.26335625001, 1754.74566738282, 1392.17984296875,
7026.71143068691, 2193.1615467585, 2836.98221749547, 6301.54294276034,
0, 3623.93436796874, 1312.70517940113, 0, 4724.16524276671, 0,
3946.52595689374, 2970.83282951206, 865.842519287108, 3283.01703994418,
0, 0, 1262.2417984723, 3482.3909258789, 4262.98461721732, 725.868942626953,
3139.75638657914, 6559.00482506506, 719.443235961914, 1980.26911301631,
735.412520327744, 2538.58699423828, 0, 0, 3345.73545624628, 17476.7195602775,
0, 3061.52193035407, 0, 3370.09478829905, 23476.7103351433, 1087.0128899699,
11912.6099077086, 1957.73038989257, 3158.21345361328, 3399.21823401529,
2593.01804501953, 934.40066220703, 2667.04373535038, 2390.35165488282,
1286.71902949219, 13590.0106023424, 1817.04336728435, 720.287557226125,
916.984646430313, 0
), TMT129N = c(
5236.22537109374, 5253.53062265625,
2574.52071173515, 6331.35667763673, 2909.4254578125, 2260.77584003907,
2299.19597588428, 2146.42980395508, 1524.82949099121, 4594.55742216798,
2945.06894956055, 3353.94465205077, 17265.8083640626, 58543.518953906,
42651.7719375001, 85288.5246984373, 147283.5420375, 16268.584838086,
31273.8547945313, 64984.9260843748, 15376.6111124999, 18807.9172359374,
41234.0440359375, 7038.37527421877, 14994.4310613281, 35007.7380726561,
5527.67209306639, 8827.66559818178, 32485.2518203124, 6771.8957270508,
10744.4285308594, 21706.4754328126, 7351.69939775389, 7286.74206159669,
31413.8810906249, 5402.31355253905, 7530.87224590352, 22346.2738640626,
7845.50157832029, 8598.84479102795, 13646.5870857421, 3948.41240126953,
4709.55193242188, 14502.506803125, 5527.37162548826, 4609.58657929687,
17346.4145701172, 3806.28572124023, 6695.23604707033, 10312.3288382812,
5030.34046109047, 12238.5130183594, 4008.11956391603, 3366.81581267473,
10163.4030287109, 5657.196356792, 9102.44150449222, 3997.46163222657,
3410.17096083986, 7459.02988359372, 2817.01574699993, 3228.9395395409,
1668.4944914795, 8344.1621935547, 6186.76663623048, 4037.21344028321,
4953.59359042969, 6163.53660527345, 2403.32564355468, 7143.26629951174,
1674.84291965332, 6149.94359912111, 2371.20030146484, 2708.46439392749,
8739.23083300782, 17708.4475347656, 2940.52380873895, 6755.28018515623,
2318.903026372, 5194.7135689453, 37003.6496414061, 1967.3172459961,
11835.060703125, 6357.79309687501, 6929.07336386721, 4922.18842499999,
5119.39548720702, 5098.90275791014, 5251.26713525389, 3925.45210825196,
3679.82248959962, 14610.4555587891, 3124.13869614258, 1892.91264873047,
2424.18607792969, 1217.02803508301
), TMT130C = c(
8534.51942065459,
7915.5220132809, 5427.32003829632, 6570.76122664215, 4059.66840776324,
6061.80182135341, 4680.93892861494, 3924.40329255471, 1651.02954082353,
9963.97493859979, 2095.70639501953, 6791.52962729408, 29089.1005494669,
84188.5986999335, 79102.1477485965, 111076.559352845, 228355.265937366,
31033.9549731673, 34443.9266034838, 100401.13011289, 20754.0266423186,
29334.7579023722, 70572.5366141732, 12419.2333360566, 17842.2049581652,
61518.6150214895, 10110.7438562966, 17502.4708992952, 61357.4708251869,
9995.86560411634, 17530.0854369149, 53203.79872126, 11560.0754183956,
13608.3534926174, 46828.7269257867, 11019.0036476036, 12751.4728461078,
26876.5493340097, 15817.3356058367, 13322.0303678209, 28346.2564980389,
6207.28749858749, 9509.23448457102, 36589.1961503305, 10263.6194964664,
7564.4130620443, 29223.5021208951, 7015.27604618795, 8985.64276558643,
18886.2813530505, 7104.5605795249, 17804.4468985063, 6812.45760606556,
6606.50436956103, 18488.5028791766, 5863.36827860186, 13255.0296116594,
6453.52502342659, 5515.76995189115, 13991.8058338637, 3809.92839574631,
5061.53991882257, 5991.94758913396, 11334.5748098943, 6206.71330898436,
5219.26272399663, 10963.7722168793, 7489.68230742191, 3361.11015884112,
9301.91002246388, 2802.03791217282, 10320.7319848397, 2509.93049810913,
3062.67670544402, 12115.818468418, 22596.7672476562, 5210.67744771832,
12678.5528095273, 3674.33047767757, 10158.3722420368, 38620.8279703125,
4299.66130273203, 16002.3112734375, 9597.19786319193, 10816.7672483378,
10449.5441895035, 8882.17509875803, 5726.84640941102, 8043.21098886215,
7002.83669129089, 4997.59224291582, 17214.8268832032, 4203.51644882184,
2044.40880964859, 2570.59274774035, 1621.14734118819
), TMT131 = c(
9418.17036445313,
8878.68274218749, 8378.77222617187, 5645.64330000002, 4753.99889121092,
4420.32600410157, 5180.99550117188, 4787.09710488281, 4537.51669335937,
11573.2264957031, 3011.21359365234, 6855.67281093751, 33135.919678125,
78639.2005781249, 89222.1922968751, 117395.8545375, 221719.7701125,
31420.3548656251, 46548.5491968752, 99822.2995687501, 21221.19759375,
31903.8316523437, 65884.2084140624, 12792.6073300781, 32494.3474710937,
55493.9358187499, 10832.2051148437, 16887.821878125, 56011.2740859373,
11016.5575394531, 15928.4518980469, 45726.2466890625, 10025.5855992188,
21737.7840796876, 59706.8427656249, 9716.2620714844, 18071.6204882813,
35495.48536875, 19939.4042976562, 13032.988340625, 32501.3787140625,
7199.69156542971, 10150.1181632812, 40466.1293437499, 10645.1514984375,
11109.5601890625, 28522.8374835938, 5834.91936972658, 6796.13434628906,
23190.480225, 9266.24059453124, 29075.5950890626, 7621.3969927734,
9322.90506210938, 22800.4644656249, 7069.82396484372, 15978.9590683593,
6530.85736874999, 6011.4814611328, 17323.3141382813, 4554.70168183593,
5415.33720234375, 6123.79340332032, 14385.484575, 10146.5210988281,
3506.75934082032, 13551.8667152343, 9222.76466718747, 5501.55972480469,
13922.8457027344, 4250.15196503907, 9017.17755468752, 3575.14872451172,
5562.49420546876, 17743.048228125, 24449.2857210939, 5704.12143984373,
15124.9334800781, 5079.61518925782, 10921.1189097656, 35210.8651640626,
4445.87090449217, 15901.6300136718, 11364.1122761719, 9222.57776601566,
8388.0708205078, 9823.76574609376, 6197.90128417968, 8764.95285703127,
9621.21499453124, 6218.11219218749, 15561.3999234375, 4677.02849882812,
3778.64337919922, 2264.6449546875, 2498.85718242187
)), class = c(
"tbl_df",
"tbl", "data.frame"
), row.names = c(NA, -96L)), structure(list(
`First Scan` = c(
16668, 16732, 16751, 16894, 16948, 16970,
17120, 17177, 17210, 17367, 17403, 17406, 17633, 17638, 17833,
17917, 18052, 18090, 18149, 18285, 18338, 18376, 18397, 18504,
18561, 18605, 18642, 18662, 18702, 18753, 18755, 18792, 18932,
18983, 19022, 19210, 19238, 19249, 19438, 19586, 19669, 19707,
19742, 19898, 19937, 20040, 20131, 20172, 20302, 20361, 20405,
20613, 20633, 20790, 20962, 21004, 21148, 21187, 21382, 21454,
21455, 21585, 21726, 21729, 21966, 22000, 22197, 22267, 22585,
22789, 23382, 23725
), TMT126 = c(
1882.9047259105, 3449.99254746093,
1024.2977771073, 5158.15467082831, 15759.4817531249, 5163.23808916981,
23084.5250109376, 11158.1465572494, 2542.16341682018, 6236.75949960936,
2540.56668486328, 4643.57579941405, 3676.30715654298, 2933.73809824218,
0, 3710.6456292118, 4085.0294783203, 675.688557714844, 1950.23111088867,
2205.57922470703, 8296.83276855471, 2672.59435136718, 3034.29849873047,
45624.8063390625, 22890.978928125, 3676.17324550782, 804.357999226392,
2339.07756328126, 2307.76013144531, 21624.511528125, 1518.1371897772,
10635.9306925781, 2680.27191738282, 11041.5375597656, 7065.95542382811,
8761.31611523439, 1801.56362783203, 7668.99884179686, 7505.24644894218,
1129.2046734375, 5552.81345566407, 6607.18650630685, 2071.91356259765,
5463.23402109373, 5149.03205670459, 1219.86494693985, 4910.47049139314,
3155.11395608923, 1398.27528804432, 6326.38970151936, 3541.72186757812,
4198.17516679688, 1440.07868518477, 0, 3465.05218769531,
961.89374368008, 2280.46133583984, 2356.01639560548, 0, 2526.26822314453,
1853.50988980811, 1310.57137792969, 1419.80935503652, 1688.71382270508,
1531.11097573243, 1821.7144977539, 1352.28257709961, 2137.31568546879,
1951.55034804646, 1311.16770682422, 0, 1720.31448781044
),
TMT127N = c(
2771.68523671875, 5795.16860742187, 1532.51465844727,
4997.19921035158, 25241.5731433593, 7411.63585078126, 35598.7388882813,
11632.0612353516, 3981.85727929688, 8431.46010673831, 4482.49877431641,
6676.69583173827, 4001.42575561522, 3282.20197690431, 1573.56998100586,
3935.30607905274, 7116.32664960937, 2307.64589311523, 4102.16486528321,
4772.48413124998, 14428.697888086, 3332.28787353515, 5485.6754455078,
91312.7273437499, 37781.7556101563, 4225.51941313476, 1839.82785454101,
4019.30935473634, 3973.05389443358, 32996.0530066406, 2037.37071027832,
20023.8569953125, 3530.19987890626, 16245.3700406251, 11744.1997277344,
14524.1289123047, 2272.03076718749, 7951.62546621093, 11169.627975,
2088.36168669433, 8426.40678837892, 5546.88783515623, 825.817551295166,
11964.5538246094, 5850.03650859376, 1951.7206411377, 6372.5401716797,
2720.9217121582, 2928.83477036133, 7842.02046181641, 4470.19431181639,
5518.54197597655, 4886.56620761718, 2598.45832382813, 5150.38777236328,
1764.69598916016, 4431.15874072266, 5851.29773701172, 3535.65924873047,
4506.44162783202, 3948.30497885742, 3474.47286108398, 4899.0403397461,
3668.68391484374, 1715.68195283204, 1517.17190141602, 2988.61810048828,
3049.25398110351, 3346.69140673828, 1629.18274760742, 1652.68488032226,
3111.11825932616
), TMT128C = c(
3546.93682792969, 5443.04733562549,
2406.94690743869, 7409.37070198117, 24566.1728096405, 7605.80923058386,
37319.8058793624, 13554.5726592211, 4322.44607961136, 10647.2819212435,
5886.92810075651, 8753.17848761496, 6553.53110235196, 4646.29586668501,
1968.9492260545, 4154.15849160926, 7670.78133562564, 2768.92031395084,
3603.54312466775, 4079.51357784559, 16511.9767178725, 3885.32503013637,
4789.39730709742, 75477.2031668271, 39696.3689381062, 6235.92097131154,
2324.05125227432, 4163.9149784173, 5315.6094992347, 31155.5901563382,
1816.81292346028, 16308.3860112984, 4287.50902664535, 15935.5273428347,
12014.8172808907, 13939.1329255981, 3921.1247165634, 11449.7683673977,
13003.5107817882, 2150.04455439802, 10889.4027696227, 6249.74721525204,
3815.90910915446, 9774.84044695728, 5968.13675351661, 3450.75965045113,
6496.88961226217, 4185.18567482653, 3493.2378779832, 6249.25306422371,
4966.6136218745, 7318.69851904571, 4807.8110810617, 1540.76095735796,
6832.23484476677, 2965.60747444571, 4139.87531020355, 4987.87787026309,
3610.04293593895, 5899.23647575952, 5796.31857744029, 3008.6643611543,
5433.73871616973, 2796.2558469155, 2533.88330506493, 3404.51135442734,
1989.89578135028, 3963.71640004554, 1855.68417729492, 1858.78970920819,
1801.24628626465, 1521.81316450196
), TMT129N = c(
5603.3715375,
7466.48746787111, 5737.87805507811, 10125.7479275391, 27223.0979882812,
11989.9738019531, 44015.6930296875, 17091.8407880859, 7938.03088417966,
12459.8042378906, 7051.86322177736, 10014.4530556641, 7992.80055556639,
5937.58603710939, 3499.28743828124, 8939.26554726561, 11082.9714445312,
3852.90205781249, 4749.96272050781, 6339.30068320313, 18824.0227183594,
5960.32649501952, 4809.15115634765, 84223.8693140624, 36909.3952734375,
7050.83312226562, 2210.35349326172, 5805.73487548829, 6812.68262490236,
29434.3397519532, 2379.38804296874, 20404.2909726562, 5217.36262587891,
19563.0384855469, 13420.0282283203, 12340.821178125, 3700.34673647461,
10695.0951164062, 15392.6850773437, 2774.59824887696, 12177.5412128907,
7713.37043554689, 3815.38668457032, 11456.5398427734, 11993.45544375,
3913.54135283203, 9596.82833613281, 5349.46382695312, 3289.20360688477,
8314.06658349611, 5784.25196894531, 7882.8373013672, 5610.97252675783,
3071.51274536133, 7498.057575293, 3661.76029086913, 6717.39973330079,
6506.75830341797, 4466.70268945312, 5969.45083388673, 7538.86863662107,
3945.00036079102, 7081.96355947265, 3515.28444770509, 4725.6043602539,
4937.22756269533, 3392.35801362304, 5545.30827919922, 4755.80240244142,
3465.95602353517, 2446.60899624023, 3763.70442260743
), TMT130C = c(
10983.9436123188,
9211.23508829853, 10557.3906937124, 10405.446046875, 28290.6707273437,
16687.4414721715, 47013.3588375, 24663.9287051498, 10819.8059091653,
12112.6441605469, 12221.0968142757, 14143.6632554762, 8870.24921835937,
10222.1638647599, 4387.39878905819, 7903.13580132335, 12659.3879214159,
4977.95176757538, 6713.09197134976, 8619.08616316083, 25601.7183693629,
9371.93738344518, 8431.95007587771, 96391.5124218752, 44929.6342171874,
7378.82902968752, 3945.11645284669, 8071.18750848737, 7751.84701707912,
38104.5129609375, 3638.03642401144, 20403.32641875, 7808.49545553976,
23539.4361984374, 18752.01165285, 20811.394168736, 5824.4538993184,
18893.4697542139, 12674.1161636719, 2807.67847851562, 12264.333778125,
11014.4240599255, 6251.73090996096, 13488.8996818318, 15964.1844357857,
5828.08199702709, 11239.0816570313, 8253.48396721728, 4132.48828007812,
13706.3856985815, 7642.39674902346, 8508.96873105467, 8428.07027860162,
5015.03055391062, 9927.36947998573, 4157.77188427735, 6591.46973203123,
10252.0377530773, 5571.05266915471, 8466.28209717723, 9830.67984953953,
6387.25448143305, 8227.90628531475, 6429.95347392001, 5927.99930589914,
6179.43049557376, 4779.11406556715, 5853.8402419922, 5984.37583438957,
2653.83845332031, 4653.86852737383, 4421.67880346553
), TMT131 = c(
13383.1659585937,
10027.0432195313, 16206.8438039063, 16355.4699128906, 29514.0965624999,
32393.8092585938, 53039.4243328127, 23782.9527632812, 21228.6799054688,
15934.2437460937, 11102.0789214844, 15037.0606933593, 13194.4500703125,
9207.67683515628, 6735.74908359374, 12899.2203527343, 13469.5999195312,
6745.45593691405, 7115.74944609379, 11411.3147414063, 27412.2978679687,
11739.3367394532, 8886.12224414064, 96689.4600375001, 45453.11203125,
7980.59650781252, 6961.24482539061, 10250.2241015625, 13306.6513335937,
39924.4417171876, 6483.56406328127, 22704.3346148437, 9771.777984375,
24691.8792656249, 18723.0097054687, 16477.3002410156, 7680.46611621096,
17120.4574535157, 18478.2819375, 4914.95838925784, 14805.4535964844,
11972.9694691407, 6757.70631679689, 12590.9775105469, 19122.4749375,
7233.17976562498, 14106.4787144532, 11431.40191875, 7873.65249257815,
16542.6268992187, 8767.54336992188, 11536.5155554687, 10534.8182027344,
6824.66757714841, 13787.927071875, 7553.96639121096, 7917.68129238283,
12813.6645140625, 7735.42080351564, 7866.51996796874, 12833.8237371094,
7160.45798144533, 11867.8380820313, 7840.4843214844, 6481.4439357422,
8916.9703347656, 7413.33213457034, 12091.0962304687, 10392.2846542969,
5704.91811914064, 5982.57808242186, 6364.45633183592
)
), class = c(
"tbl_df",
"tbl", "data.frame"
), row.names = c(NA, -72L))), `# of PSMs` = c(
96,
72
)), row.names = c(NA, -2L), groups = structure(list(Unipro.ID = c(
"Q13177",
"Q8TD19"
), Gene.name = c("PAK2", "NEK9"), Mod.site = c(
"Q13177_278",
"Q8TD19_81"
), Mod.site2 = c("PAK2_278", "NEK9_81"), `mod.or.not(Y/N)` = c(
"Y",
"Y"
), `kinase.or.not(Y/N)` = c("Y", "Y"), .rows = structure(list(
1L, 2L
), ptype = integer(0), class = c(
"vctrs_list_of", "vctrs_vctr",
"list"
))), row.names = c(NA, -2L), class = c(
"tbl_df", "tbl",
"data.frame"
), .drop = TRUE), class = c(
"grouped_df", "tbl_df",
"tbl", "data.frame"
))
library(tidyverse)
df %>%
unnest(data) %>%
summarise(across(starts_with("TMT"), sum)) %>%
pivot_longer(starts_with("TMT")) %>%
# select only a few columns for optimized printing
select(Unipro.ID, Gene.name, Mod.site, Mod.site2, name, value)
#> `summarise()` has grouped output by 'Unipro.ID', 'Gene.name', 'Mod.site', 'Mod.site2', 'mod.or.not(Y/N)'. You can override using the `.groups` argument.
#> Adding missing grouping variables: `mod.or.not(Y/N)`
#> # A tibble: 12 × 7
#> # Groups: Unipro.ID, Gene.name, Mod.site, Mod.site2, mod.or.not(Y/N) [2]
#> `mod.or.not(Y/N)` Unipro.ID Gene.name Mod.site Mod.site2 name value
#> <chr> <chr> <chr> <chr> <chr> <chr> <dbl>
#> 1 Y Q13177 PAK2 Q13177_278 PAK2_278 TMT126 100372.
#> 2 Y Q13177 PAK2 Q13177_278 PAK2_278 TMT127N 127769.
#> 3 Y Q13177 PAK2 Q13177_278 PAK2_278 TMT128C 435779.
#> 4 Y Q13177 PAK2 Q13177_278 PAK2_278 TMT129N 1189693.
#> 5 Y Q13177 PAK2 Q13177_278 PAK2_278 TMT130C 1869744.
#> 6 Y Q13177 PAK2 Q13177_278 PAK2_278 TMT131 2008380.
#> 7 Y Q8TD19 NEK9 Q8TD19_81 NEK9_81 TMT126 355683.
#> 8 Y Q8TD19 NEK9 Q8TD19_81 NEK9_81 TMT127N 572770.
#> 9 Y Q8TD19 NEK9 Q8TD19_81 NEK9_81 TMT128C 594897.
#> 10 Y Q8TD19 NEK9 Q8TD19_81 NEK9_81 TMT129N 716943.
#> 11 Y Q8TD19 NEK9 Q8TD19_81 NEK9_81 TMT130C 904627.
#> 12 Y Q8TD19 NEK9 Q8TD19_81 NEK9_81 TMT131 1075259.
Created on 2021-09-19 by the reprex package (v2.0.1)
I have some time series data called dat and what I am trying to do is to split it into training and test on a rolling basis.
Say we have 100 days in total, I want to train the model on the first 20 days and test on the next 10 days (so using 30 days for train & test). Then move from day 2 until day 22 (training on 20 days) and then test on the next 10 days (22 - 32). Then do the same but begin on day 3 and train until day 23, and test on the next 10 observations until 33. Keep going until the final model begins on day 70 and trains until 90, tests on the last 10 observations.
I am trying to make it so that the number of days an change, i.e. the total days can be 1000, 1250, 87 etc.
I have a function which trains a logistic model on some data but the data expands as the days increase but its not exactly what I am after.
If I can créate the different training and tests splits then using the rollapply function might give the results I am after.
EDIT: I am not sure if it would be better/ or interesting to train on the first 20 days and then test just on the next 1 day instead of 10 days.
Code:
myfun <- function(model_len, dat, ...){
dat <- data.frame(dat)
names(dat) <- c("y", "x1", "x2", "x3")
fit <- glm(formula, data=dat[(1:model_len),])
predict(fit, dat[(model_len + 1),])
}
sapply(1:50, myfun, dat=dat)
Data:
dat <- structure(c(0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1,
1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1,
0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0,
1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1,
0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0,
0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1,
1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0,
1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1,
1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1,
0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1,
0, 1, 1, 1, 1, 1, 1157.4779907, 1161.2739868, 1165.064978, 1162.5039794,
1152.5029784, 1143.5659789, 1131.9999755, 1115.114978, 1101.3089843,
1088.9449828, 1077.7859863, 1067.7619873, 1059.9439942, 1058.2339967,
1062.8999879, 1065.9739869, 1071.7789918, 1084.3059937, 1094.9029908,
1101.5380006, 1106.801001, 1106.7830079, 1105.7230103, 1105.3360108,
1104.5960206, 1104.4260255, 1106.363025, 1109.688025, 1111.763025,
1113.7510255, 1118.2270265, 1126.2330201, 1131.9140137, 1132.8030029,
1133.0679931, 1131.1919921, 1123.4999877, 1109.6529845, 1098.5239806,
1085.2169738, 1070.7239746, 1058.9449829, 1046.018982, 1037.3779847,
1030.1209901, 1023.8139955, 1019.6099977, 1018.9979982, 1016.8410036,
1018.3280031, 1021.1230043, 1020.8710024, 1024.0220033, 1030.0970094,
1034.7910035, 1040.7799927, 1047.371991, 1052.5719849, 1051.4059814,
1051.5269836, 1052.2799865, 1052.3579894, 1050.2929931, 1046.6079956,
1041.8380005, 1035.4400025, 1032.9650025, 1031.6990113, 1035.0920167,
1041.2500184, 1047.0030091, 1053.8240052, 1062.1109986, 1066.3029907,
1072.0419922, 1077.5289917, 1079.3439941, 1081.8229858, 1083.4049804,
1083.0979735, 1081.2649779, 1079.0049803, 1075.0169798, 1073.8739867,
1074.1959837, 1078.2869871, 1085.5799925, 1091.5880003, 1098.3030028,
1102.7200072, 1106.8830077, 1112.3160033, 1120.2160033, 1126.9150023,
1133.6280028, 1136.9040038, 1140.320996, 1143.1609985, 1146.4569946,
1149.8369995, 1153.297998, 1152.7800049, 1150.6940064, 1147.6130005,
1143.8229981, 1140.1619995, 1135.5619995, 1129.0449951, 1124.4880005,
1122.7390015, 1122.5960084, 1125.3989991, 1128.9430054, 1136.8930054,
1144.3530029, 1151.173999, 1158.3080078, 1167.6070068, 1173.8760009,
1178.3499999, 1183.494995, 1193.018994, 1203.9989867, 1212.4839843,
1217.4519897, 1221.0399902, 1222.8859863, 1225.2989868, 1229.2179931,
1233.0979858, 1235.0249878, 1234.4389893, 1232.6299927, 1230.7069947,
1230.6179932, 1232.1449952, 1234.6289918, 1234.0659913, 1232.0999879,
1229.8249879, 1228.1249879, 1224.0649903, 1220.2369874, 1215.8649903,
1214.1689942, 1214.8499878, 1213.7549926, 1217.246997, 1220.5099975,
1222.2329955, 1221.1559935, 1219.641992, 1216.0529905, 1211.9979856,
1206.3969847, 1199.9509886, 1193.1179808, 1185.7209715, 1179.0619749,
1172.8479857, 1169.2699828, 1167.7309814, 1169.2739868, 1169.3999878,
1170.2729858, 1171.0019897, 1172.7689941, 1174.7, 1176.7939942,
1180.7199952, 1184.6089966, 1187.7949951, 1185.9269897, 1185.0529907,
1182.6129883, 1178.0299805, 1168.1029786, 1156.5709717, 1148.2319702,
1137.9259643, 1130.0429687, 1121.3169677, 1113.2949707, 1107.2059692,
1102.4249755, 1098.911975, 1095.860974, 1097.485974, 1093.6249755,
1086.4079772, 1077.9009704, 1074.0089783, 1072.2119812, 1068.344989,
1062.2379822, 1057.449994, 1061.7179994, 1060.4010072, 1059.8690125,
1061.7240113, 1061.7080201, 1058.3970215, 1057.8680176, 1058.2380127,
1056.2290161, 1053.2240112, 1047.6460082, 1041.7940063, 1040.0410034,
1040.6190063, 1045.6369994, 1050.1010009, 1128.81199335, 1132.72894074524,
1136.05951315045, 1133.75860942184, 1126.33398461976, 1121.97836475121,
1114.98804010824, 1104.18156200269, 1097.85760647863, 1093.48449548066,
1089.54311267298, 1087.65328775174, 1087.83107177539, 1088.49478389202,
1089.82480075944, 1091.87386411569, 1093.27921086657, 1096.47071830785,
1100.97350704044, 1102.6227005604, 1102.82339384036, 1099.6516439508,
1097.67720586025, 1097.0346199688, 1096.8465665432, 1098.06499020575,
1100.72546732901, 1106.37447415482, 1111.91023852103, 1114.41117237617,
1117.75201214987, 1120.7832448975, 1122.20674347869, 1120.07466752834,
1117.94469547802, 1115.36710590868, 1109.05404401262, 1100.7222309638,
1096.19725287201, 1087.52132174134, 1079.62024328978, 1075.06498573838,
1068.53212719186, 1063.28239822121, 1059.64979029538, 1056.61743493392,
1051.89577236878, 1048.42474757175, 1046.82620161254, 1044.26846536373,
1043.14861247194, 1041.82684176033, 1041.46047397363, 1044.57471778567,
1047.19426428227, 1051.05194873158, 1053.13842609047, 1054.50142846281,
1051.21367146635, 1048.35332113622, 1047.56157998039, 1045.89381512512,
1043.17345339892, 1042.61503488473, 1040.8783653719, 1039.24423257458,
1040.09811147224, 1041.49734266536, 1042.67950374485, 1046.49669481677,
1051.36081397707, 1055.8274040745, 1060.05336092454, 1061.8797055984,
1063.77402125569, 1065.18506361229, 1065.29696088731, 1066.65724613614,
1066.94988745651, 1068.16322588922, 1069.21815580453, 1069.83166801363,
1068.92578972661, 1068.81857632408, 1070.35871095988, 1075.03883372561,
1081.15799613269, 1086.72961878672, 1091.50584604513, 1094.58719261226,
1097.09031664919, 1100.22361887307, 1103.94707859945, 1106.8845033995,
1111.19264545669, 1115.10382303224, 1120.66155045774, 1125.17569412844,
1129.42943430668, 1132.1180628489, 1134.34300733948, 1133.43510749763,
1132.00890306928, 1129.33948182459, 1127.89952841272, 1126.73290894484,
1126.80215199772, 1124.52480561698, 1124.50054032013, 1125.99287400392,
1128.66498590831, 1130.96736496466, 1133.15142772993, 1137.94462318423,
1142.78989202382, 1146.70132945013, 1151.6631122644, 1155.87424490588,
1158.8347892958, 1161.3181459343, 1165.5259415596, 1173.38822864916,
1181.98934506353, 1190.21226039081, 1194.81109273454, 1197.18527342649,
1199.09715310016, 1201.08885375729, 1203.47563187564, 1205.40271083986,
1207.24721647416, 1210.57795500043, 1213.91433880992, 1217.26535187564,
1219.20293598272, 1220.70837160341, 1222.74566726023, 1221.94893752116,
1220.47665680486, 1218.61792387106, 1217.58479016906, 1216.06433348629,
1215.23248801141, 1214.29415629603, 1214.89947702975, 1217.46333121739,
1218.76682576811, 1221.6747517902, 1223.33620352446, 1222.84608328404,
1220.3845515427, 1217.15554472911, 1212.80167770729, 1208.2329423066,
1204.08123494406, 1201.53635399701, 1197.84907704491, 1195.70439885016,
1193.49731600729, 1189.93090962564, 1187.19653451844, 1185.66257561192,
1185.77756793459, 1183.90255822654, 1182.89945696687, 1183.06617763669,
1182.8208264332, 1183.94646343956, 1184.8534641596, 1185.84933033488,
1187.20748792203, 1188.70677011993, 1186.75278639422, 1183.95251873763,
1180.62084752452, 1176.63980928409, 1167.55220563799, 1159.14913329151,
1154.47587831137, 1148.54960418648, 1145.95250178776, 1143.07035314131,
1137.82269769928, 1133.88338944221, 1130.76687940009, 1128.18812336199,
1120.80925075608, 1118.40550744598, 1113.93545635589, 1104.9968430839,
1098.44571145686, 1096.38135988954, 1093.86884942387, 1090.43277224064,
1085.63821926534, 1082.79744209722, 1083.80625856415, 1083.6723314628,
1082.00354027587, 1077.87272739245, 1073.8896151646, 1071.01060743464,
1070.41054586943, 1069.56096911996, 1064.84087682282, 1061.11888950636,
1058.87994622004, 1055.5466184848, 1054.88694005768, 1053.88913948076,
1056.96921953021, 1059.95310805114, 77.1228859956622, 81.0362538530292,
78.8404654349793, 46.4728298378735, 33.7103494024937, 38.1634534707235,
33.5520386736078, 26.2429467891094, 30.5979953728327, 30.5979953728327,
31.2223518673486, 33.7665461425831, 36.6962580582319, 37.7398082531122,
40.5860776927095, 41.0627097257687, 40.7556533339627, 52.526559398101,
67.2093345204357, 57.3558861837519, 61.809628052695, 65.0522479908148,
60.3356537763659, 59.9025026642582, 60.6951031882524, 60.0950548232381,
59.3846485649388, 64.6199416069941, 64.1051430716001, 55.6515339908006,
58.7835089189351, 55.0890845598537, 48.1838706704649, 46.0064642542491,
48.4030879681908, 55.5793562399467, 43.3339041496164, 35.5089178322478,
42.157901440901, 32.5975281088021, 28.6602735068277, 26.9110067493817,
23.5372731683978, 27.6575715257538, 27.7636741048428, 28.4241344813052,
27.7437779358905, 33.8748748481366, 38.0173561927228, 37.3614293051309,
46.7027642395441, 51.6960358269122, 46.2684476430283, 67.9712504992444,
67.4307596718059, 65.3539239654913, 69.3859268680975, 65.8884694613497,
48.7463489665683, 48.3776103610145, 58.1513743683333, 53.5784372311078,
46.4319595892114, 54.1515204375632, 48.0571628692748, 48.6571396623733,
52.2995925118996, 44.9774509790143, 45.2591195805464, 48.7943143049565,
56.0044804919092, 57.6982718090011, 75.947686211121, 66.6475291255686,
63.2031704734223, 66.0494138822722, 66.2641524590373, 64.6800962380417,
66.0941051628946, 68.6330617447997, 62.298871330898, 58.4734193157287,
52.329016147723, 43.5650542408412, 44.6973713488007, 56.9666746925596,
61.477502601121, 70.1850582389349, 68.3785649248245, 64.1672444920065,
68.1060250901431, 67.2130080618559, 73.8468747118516, 69.6113702464934,
73.1570958144156, 74.8830412236628, 85.4049570826199, 81.7882678868151,
79.8159292966814, 65.9053697697576, 57.9091367119927, 44.4025529377091,
43.2388424796772, 42.7803356293289, 47.7057738515549, 44.7755737074884,
45.7557906780512, 40.016244653124, 41.4992896665767, 46.6336286507843,
44.3657650232027, 45.4718259236287, 45.2372613787558, 56.9881807801438,
58.8717301068573, 68.2039283244873, 73.5215112680329, 78.8594307629251,
73.0335410836162, 71.845824268758, 73.323376014074, 89.1748677280385,
88.8275948061702, 88.079358554904, 72.9197089804835, 66.5774741060939,
65.5905607795046, 60.3560855296636, 60.5351059532554, 61.4085229097936,
58.076745639994, 63.2173375817626, 67.2733875032827, 68.7459719049055,
59.9037653356146, 44.6491666372171, 40.4929666577831, 30.2655738215587,
36.0522832244009, 40.7505784647263, 45.517250253278, 41.5835266382263,
41.3526668380199, 41.539756712543, 48.3189167794286, 49.8415866657383,
44.5858982397584, 50.0675010891207, 50.5139938354098, 44.9097955003298,
37.4247186375495, 41.3952548987526, 39.6467050713014, 39.3953595896288,
36.8289128008105, 42.8772642627352, 37.5760511024063, 42.0791664435174,
36.4236440580649, 25.1434697637668, 29.0666072154372, 25.3668839063101,
34.1040319281821, 34.1351918720353, 42.138526061446, 49.3942545777117,
53.2282422165058, 60.0907410718325, 59.6946479180297, 56.5126081396889,
64.5584522103826, 61.6638469740838, 48.5567687748239, 50.4491176695018,
45.8595330253583, 39.1134283844586, 22.2017732449298, 24.6509068125481,
33.7409449463083, 27.0354908046699, 36.9033514343542, 31.849732552439,
28.384694400023, 30.2843907497844, 30.2566110685775, 30.1702095862,
28.1229085893699, 39.7891005017724, 37.8236546439287, 33.4844836408483,
42.9231744072258, 49.6425369989148, 43.9761986844232, 44.7318583977582,
37.1424843378588, 40.8120228103859, 50.807226927847, 47.9214803669887,
44.995279725301, 41.3197867616665, 47.7401787161256, 40.9599257198947,
48.8101085201251, 58.7773921954413, 46.8976151314924, 38.7370234461344,
43.0052200556536, 42.7247275761847, 51.7764243779359, 47.5063348907638,
48.4623219235214, 51.3175593621287), class = c("xts", "zoo"), .indexCLASS = "Date", .indexTZ = "UTC", tclass = "Date", tzone = "UTC", src = "yahoo", updated = structure(1544977543.47594, class = c("POSIXct",
"POSIXt")), index = structure(c(1517356800, 1517443200, 1517529600,
1517788800, 1517875200, 1517961600, 1518048000, 1518134400, 1518393600,
1518480000, 1518566400, 1518652800, 1518739200, 1519084800, 1519171200,
1519257600, 1519344000, 1519603200, 1519689600, 1519776000, 1519862400,
1519948800, 1520208000, 1520294400, 1520380800, 1520467200, 1520553600,
1520812800, 1520899200, 1520985600, 1521072000, 1521158400, 1521417600,
1521504000, 1521590400, 1521676800, 1521763200, 1522022400, 1522108800,
1522195200, 1522281600, 1522627200, 1522713600, 1522800000, 1522886400,
1522972800, 1523232000, 1523318400, 1523404800, 1523491200, 1523577600,
1523836800, 1523923200, 1524009600, 1524096000, 1524182400, 1524441600,
1524528000, 1524614400, 1524700800, 1524787200, 1525046400, 1525132800,
1525219200, 1525305600, 1525392000, 1525651200, 1525737600, 1525824000,
1525910400, 1525996800, 1526256000, 1526342400, 1526428800, 1526515200,
1526601600, 1526860800, 1526947200, 1527033600, 1527120000, 1527206400,
1527552000, 1527638400, 1527724800, 1527811200, 1528070400, 1528156800,
1528243200, 1528329600, 1528416000, 1528675200, 1528761600, 1528848000,
1528934400, 1529020800, 1529280000, 1529366400, 1529452800, 1529539200,
1529625600, 1529884800, 1529971200, 1530057600, 1530144000, 1530230400,
1530489600, 1530576000, 1530748800, 1530835200, 1531094400, 1531180800,
1531267200, 1531353600, 1531440000, 1531699200, 1531785600, 1531872000,
1531958400, 1532044800, 1532304000, 1532390400, 1532476800, 1532563200,
1532649600, 1532908800, 1532995200, 1533081600, 1533168000, 1533254400,
1533513600, 1533600000, 1533686400, 1533772800, 1533859200, 1534118400,
1534204800, 1534291200, 1534377600, 1534464000, 1534723200, 1534809600,
1534896000, 1534982400, 1535068800, 1535328000, 1535414400, 1535500800,
1535587200, 1535673600, 1536019200, 1536105600, 1536192000, 1536278400,
1536537600, 1536624000, 1536710400, 1536796800, 1536883200, 1537142400,
1537228800, 1537315200, 1537401600, 1537488000, 1537747200, 1537833600,
1537920000, 1538006400, 1538092800, 1538352000, 1538438400, 1538524800,
1538611200, 1538697600, 1538956800, 1539043200, 1539129600, 1539216000,
1539302400, 1539561600, 1539648000, 1539734400, 1539820800, 1539907200,
1540166400, 1540252800, 1540339200, 1540425600, 1540512000, 1540771200,
1540857600, 1540944000, 1541030400, 1541116800, 1541376000, 1541462400,
1541548800, 1541635200, 1541721600, 1541980800, 1542067200, 1542153600,
1542240000, 1542326400, 1542585600, 1542672000, 1542758400, 1542931200,
1543190400, 1543276800, 1543363200, 1543449600, 1543536000), tzone = "UTC", tclass = "Date"), .Dim = c(212L,
4L), .Dimnames = list(NULL, c("y", "x1", "x2", "x3")))
EDIT:
Just so I undestand the function output a little.
I set:
, n_train = 5
, n_test = 1
and get the following final 3 outputs:
[[203]]
2018-11-16 2018-11-19 2018-11-20 2018-11-21 2018-11-23 2018-11-26
1.00045650 0.08862828 0.61874897 1.00620776 0.67800147 0.60795702
[[204]]
2018-11-19 2018-11-20 2018-11-21 2018-11-23 2018-11-26 2018-11-27
0.05759443 0.69372082 0.93025186 0.72564291 0.60694731 0.98584268
[[205]]
2018-11-20 2018-11-21 2018-11-23 2018-11-26 2018-11-27 2018-11-28
0.8507988 0.8028078 0.7412901 0.6416496 0.9538837 1.0095700
Are these the predicted probabilities of the event happening? How can we have 1.0095700 as one of the probabilities?
Secondly since n train = 5 and n test = 1, the last output tells me that the first 5 results are the predicted probabilities on the training data and the 6th results is the predicted on the test data, i.e. data 2018-11-28 = 1.0095700
?, the same being for result 204, 2018-11-27 = 0.98584268.
I am not sure how you intend to use such a function, but you can wrap some of the code in an extra function, where you compute the training and testing indexes. For example, like so
myfun <- function(fm, dat, train_index, test_index){
fit <- glm(fm, data=dat[train_index, ])
predict(fit, newdata = dat[test_index, ], type = 'response')
}
wrapper_myfun <- function(
dat
, n_train = 20
, n_test = 10
){
stopifnot('y' %in% names(dat))
f_ <- formula(paste0('y~', paste(setdiff(names(dat), 'y'), collapse = ' + ')))
stride <- n_train + n_test
start_position <- seq(1, dim(dat)[1] - stride)
train_index_list <- lapply(start_position
, function(i) seq(i, i + n_train))
test_index_list <- lapply(start_position
, function(i) seq((i + n_train + 1)
, (i + n_train + n_test)))
mapply(
myfun
, train_index = train_index_list
, test_index = test_index_list
, MoreArgs = list(fm = f_, dat = dat)
, SIMPLIFY = F
)
}
You can further optimize this code.
Choosing between 1 and 10 time periods for test purposes depends on application quite a bit.
HTH
I have trained a simple model and I am now trying to backtest it using quantstrat. I am still getting to know how quantstrat works, so I know I am making some mistakes in the code. The model is very simple (54% test accuracy) but I am just applying it to quantstrat for education on the quantstrat package.
The data looks like:
open high low close volume adjusted direction returns
2018-11-23 1030.00 1037.590 1022.399 1023.88 691500 1023.88 1 -0.013232313
2018-11-26 1038.35 1049.310 1033.910 1048.62 1850400 1048.62 0 0.024162978
2018-11-27 1041.00 1057.580 1038.490 1044.41 1803200 1044.41 0 -0.004014763
2018-11-28 1048.76 1086.840 1035.760 1086.23 2475400 1086.23 0 0.040041693
2018-11-29 1076.08 1094.245 1076.000 1088.30 1468900 1088.30 0 0.001905737
2018-11-30 1089.07 1095.570 1077.880 1094.43 2580200 1094.43 0 0.005632642
sma rsi momentum roc pred
2018-11-23 1041.794 42.72473 -31.93005 -0.03910309 1
2018-11-26 1040.041 51.77642 -44.77002 -0.01219856 0
2018-11-27 1040.619 47.50633 -37.98999 0.02364954 0
2018-11-28 1045.637 48.46232 20.07996 0.05727916 0
2018-11-29 1050.101 51.31756 49.67004 0.04769691 0
2018-11-30 1053.073 56.61429 58.38000 0.06663439 0
Much of the information in the table is not of any use anymore. The strategy I am trying to apply is when the column pred = 0, long the position. When the column pred = 1, short the position and close each position at the end of each period. (In this case I have daily data, but the period could be hours, weeks, months etc.)
Questions:
Where am I going wrong in the add.signal and add.rule.
How can I add a rule to close out a position at the end of each period?
Code:
require(quantstrat)
require(PerformanceAnalytics)
library(e1071)
mean(test$pred == test$direction)
initDate="2007-01-01"
from <- "2017-07-09"
to <- "2018-12-01"
init_equity <- 1
adjustment <- TRUE
currency('USD')
Sys.setenv(TZ="UTC")
symbols <- 'GOOG'
GOOG <- test
currency('USD')
Sys.setenv(TZ="UTC")
stock("GOOG", currency="USD", multiplier=1)
symbols <- c("GOOG")
strategy.st <- portfolio.st <- account.st <- "SVMstrat"
rm.strat(strategy.st)
rm.strat(portfolio.st)
rm.strat(account.st)
initPortf(name = portfolio.st,
symbols = symbols,
initDate = initDate,
currency = 'USD')
initAcct(name = account.st,
portfolios = portfolio.st,
initDate = initDate,
currency = 'USD',
initEq = init_equity)
initOrders(portfolio.st,
symbols = symbols,
initDate = initDate)
strategy(strategy.st, store = TRUE)
########### Adding to the strategy
add.signal(strategy.st,
name = "sigThreshold",
arguments = list(column = "pred",
threshold = 0.5,
relationship = "gte",
cross = TRUE),
label = "longEntry")
add.signal(strategy.st,
name = "sigThreshold",
arguments = list(column = "pred",
threshold = 0.5,
relationship = "lte",
cross = TRUE),
label = "longExit")
applySignals(strategy = strategy.st, mktdata=GOOG)
###################################################
add.rule(strategy.st,
name="ruleSignal",
arguments=list(sigcol="longEntry",
sigval=TRUE,
orderqty = 100,
ordertype="market",
orderside="long",
replace=FALSE,
TxnFees = -10,
prefer="Open"),
type="enter",
path.dep=TRUE)
add.rule(strategy.st,
name="ruleSignal",
arguments=list(sigcol="longExit",
sigval=TRUE,
orderqty = 100,
ordertype="market",
orderside="long",
replace=FALSE,
TxnFees = -10,
prefer="Open"),
type="exit",
path.dep=TRUE)
applyStrategy(strategy=strategy.st,portfolios=portfolio.st)
updatePortf(portfolio.st)
dateRange <- time(getPortfolio(portfolio.st)$summary)[-1]
updateAcct(portfolio.st,dateRange)
updateEndEq(account.st)
chart.Posn(portfolio.st, 'GOOG')
Data:
test <- structure(c(1156.97998, 1144.589966, 1159.890015, 1185, 1189.390015,
1172.219971, 1196.560059, 1191, 1186.959961, 1181.01001, 1262.589966,
1239.130005, 1251, 1271, 1228.01001, 1220.01001, 1228, 1205.900024,
1229.619995, 1225, 1237, 1240.469971, 1249.900024, 1243, 1236.97998,
1235.189941, 1229.26001, 1224.72998, 1202.030029, 1205.02002,
1208, 1200, 1207.140015, 1208.819946, 1227.599976, 1241.290039,
1237.449951, 1244.22998, 1234.97998, 1204.27002, 1193.800049,
1186.300049, 1158.670044, 1172.189941, 1161.630005, 1172.719971,
1170.73999, 1179.099976, 1170.140015, 1157.089966, 1164.97998,
1179.98999, 1192, 1157.170044, 1176.150024, 1185.150024, 1186.72998,
1191.869995, 1199.890015, 1190.959961, 1205, 1195.329956, 1167.5,
1150.109985, 1146.150024, 1131.079956, 1072.939941, 1108, 1108.910034,
1104.589966, 1126.459961, 1121.839966, 1093.369995, 1103.060059,
1080.890015, 1104.25, 1071.790039, 1037.030029, 1082.469971,
1008.460022, 1059.810059, 1075.800049, 1073.72998, 1055, 1039.47998,
1069, 1091.380005, 1073.98999, 1061.390015, 1043.290039, 1050,
1044.709961, 1059.410034, 1057.199951, 1000, 1036.76001, 1030,
1038.349976, 1041, 1048.76001, 1076.079956, 1089.069946, 1159.589966,
1164.290039, 1184.410034, 1195.416992, 1191, 1203.040039, 1204.5,
1200, 1196.859985, 1206.48999, 1266, 1265.859985, 1269.770996,
1273.890015, 1234.916016, 1227.588013, 1233.469971, 1229.880005,
1230, 1226.088013, 1251.170044, 1256.5, 1255.541992, 1245.694946,
1249.272949, 1245.869995, 1235.23999, 1226, 1209.02002, 1211,
1217.26001, 1211.839966, 1221.280029, 1221.650024, 1243.089966,
1242.545044, 1250.660034, 1253.63501, 1238.660034, 1212.98999,
1199.01001, 1186.300049, 1175.26001, 1174.540039, 1178.680054,
1178.609985, 1178.609985, 1180.425049, 1177.23999, 1176.079956,
1173.209961, 1189.890015, 1192.209961, 1178, 1186.880005, 1194.22998,
1202.099976, 1195.410034, 1209.900024, 1209.959961, 1206.410034,
1197.51001, 1173.5, 1168, 1154.349976, 1132.170044, 1106.400024,
1115, 1113.446045, 1124.219971, 1128.98999, 1121.839966, 1110.359985,
1112.22998, 1107.890015, 1106.119995, 1110.97998, 1106.530029,
1097.040039, 1037.48999, 1091.939941, 1083.974976, 1082.974976,
1058.469971, 1064.344971, 1095.459961, 1093.27002, 1075.560059,
1062.119995, 1056.60498, 1054.563965, 1071.849976, 1067, 1060.790039,
1031.73999, 1048.560059, 1037.589966, 1049.310059, 1057.579956,
1086.839966, 1094.244995, 1095.569946, 1149.589966, 1141, 1155.935059,
1180, 1179.280029, 1170.599976, 1190.339966, 1183.319946, 1184.219971,
1181, 1235.560059, 1239.130005, 1249.02002, 1231, 1211.469971,
1205.599976, 1210.209961, 1204.790039, 1215.060059, 1215.796997,
1236.170044, 1238.008057, 1246.01001, 1232, 1233.640991, 1225.109985,
1209.51001, 1202.550049, 1188.23999, 1194.625977, 1200.354004,
1199, 1204.23999, 1206.359009, 1225.715942, 1228.689941, 1236.359009,
1232.589966, 1211.285034, 1192.5, 1162, 1152, 1157.214966, 1160.109985,
1156.23999, 1158.359985, 1162.849976, 1168.329956, 1154.030029,
1157.089966, 1154.579956, 1173.359985, 1166.040039, 1146.910034,
1168, 1174.765015, 1183.630005, 1184.5, 1190.300049, 1186.630005,
1193.829956, 1155.57605, 1145.119995, 1127.364014, 1137.572021,
1081.130005, 1068.27002, 1086.401978, 1089, 1102.5, 1102.189941,
1077.089966, 1087.75, 1091, 1070, 1048.73999, 1069.550049, 1034.089966,
995.830017, 1000.75, 1057, 1062.459961, 1054.609985, 1021.23999,
1038.069946, 1065.900024, 1072.204956, 1053.109985, 1031, 1031.150024,
1031, 1031.780029, 1048.97998, 1016.26001, 996.02002, 1033.469971,
1022.398987, 1033.910034, 1038.48999, 1035.76001, 1076, 1077.880005,
1152.839966, 1153.900024, 1183.47998, 1188.819946, 1183.859985,
1198.800049, 1195.880005, 1186.959961, 1184.910034, 1205.5, 1248.079956,
1263.699951, 1268.329956, 1238.5, 1219.73999, 1217.26001, 1220.01001,
1226.150024, 1223.709961, 1224.77002, 1242.219971, 1245.609985,
1249.099976, 1237.609985, 1235.01001, 1242.099976, 1214.380005,
1206.48999, 1200.959961, 1207.77002, 1201.619995, 1207.329956,
1205.380005, 1220.650024, 1241.819946, 1231.150024, 1249.300049,
1239.119995, 1218.189941, 1197, 1186.47998, 1171.439941, 1164.829956,
1164.640015, 1177.359985, 1162.819946, 1175.329956, 1172.530029,
1156.050049, 1161.219971, 1171.089966, 1186.869995, 1166.089966,
1173.369995, 1184.650024, 1180.48999, 1194.640015, 1193.469971,
1195.310059, 1200.109985, 1202.949951, 1168.189941, 1157.349976,
1148.969971, 1138.819946, 1081.219971, 1079.319946, 1110.079956,
1092.25, 1121.280029, 1115.689941, 1087.969971, 1096.459961,
1101.160034, 1103.689941, 1050.709961, 1095.569946, 1071.469971,
1020.080017, 1036.209961, 1076.77002, 1070, 1057.790039, 1040.089966,
1055.810059, 1093.390015, 1082.400024, 1066.150024, 1038.630005,
1036.050049, 1043.660034, 1064.709961, 1061.48999, 1020, 1025.76001,
1037.609985, 1023.880005, 1048.619995, 1044.410034, 1086.22998,
1088.300049, 1094.430054, 798400, 1120000, 1251900, 1221900,
1055700, 1610400, 1393600, 1276700, 1247400, 2619200, 3318200,
2127800, 2405600, 2130600, 1849900, 1644700, 1567200, 1531300,
1089600, 1081700, 1494000, 1370300, 841800, 1108700, 958100,
1348100, 1828800, 1343200, 1389600, 870800, 1205600, 887400,
992600, 946600, 1156300, 1304000, 1298900, 1331400, 1816400,
1831000, 2061300, 1888500, 1401300, 1115400, 1209300, 1295500,
1431200, 944000, 1306500, 1203600, 1191400, 1210000, 4405600,
1271000, 977700, 1462300, 1260800, 1380600, 1357600, 1687900,
1256200, 2209500, 1184300, 1932400, 1308700, 2675700, 2949000,
2101300, 1372400, 1928500, 1467200, 2094500, 1267600, 1514200,
1848700, 1982400, 2545800, 4187600, 3880700, 3212700, 2529800,
1482000, 1839000, 2441400, 1233300, 2058400, 1488200, 1343200,
1471800, 1513700, 1565900, 1836100, 1658100, 1858600, 2449100,
1534300, 691500, 1850400, 1803200, 2475400, 1468900, 2580200,
1152.839966, 1153.900024, 1183.47998, 1188.819946, 1183.859985,
1198.800049, 1195.880005, 1186.959961, 1184.910034, 1205.5, 1248.079956,
1263.699951, 1268.329956, 1238.5, 1219.73999, 1217.26001, 1220.01001,
1226.150024, 1223.709961, 1224.77002, 1242.219971, 1245.609985,
1249.099976, 1237.609985, 1235.01001, 1242.099976, 1214.380005,
1206.48999, 1200.959961, 1207.77002, 1201.619995, 1207.329956,
1205.380005, 1220.650024, 1241.819946, 1231.150024, 1249.300049,
1239.119995, 1218.189941, 1197, 1186.47998, 1171.439941, 1164.829956,
1164.640015, 1177.359985, 1162.819946, 1175.329956, 1172.530029,
1156.050049, 1161.219971, 1171.089966, 1186.869995, 1166.089966,
1173.369995, 1184.650024, 1180.48999, 1194.640015, 1193.469971,
1195.310059, 1200.109985, 1202.949951, 1168.189941, 1157.349976,
1148.969971, 1138.819946, 1081.219971, 1079.319946, 1110.079956,
1092.25, 1121.280029, 1115.689941, 1087.969971, 1096.459961,
1101.160034, 1103.689941, 1050.709961, 1095.569946, 1071.469971,
1020.080017, 1036.209961, 1076.77002, 1070, 1057.790039, 1040.089966,
1055.810059, 1093.390015, 1082.400024, 1066.150024, 1038.630005,
1036.050049, 1043.660034, 1064.709961, 1061.48999, 1020, 1025.76001,
1037.609985, 1023.880005, 1048.619995, 1044.410034, 1086.22998,
1088.300049, 1094.430054, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0,
0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1,
0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0,
0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0,
1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0,
1, 0, 0, 0, 0, 0, -0.00104855331105314, 0.000919518780805317,
0.0256347650444282, 0.00451208815547521, -0.00417217175459472,
0.012619789662035, -0.00243580570624413, -0.00745897913060267,
-0.00172703972109811, 0.0173768179939304, 0.03532140688511, 0.0125152198181764,
0.00366384836553646, -0.0235190818121779, -0.0151473637464675,
-0.0020332038141998, 0.00225917222073213, 0.00503275706729656,
-0.00199001994229053, 0.000866266545001881, 0.0142475327735407,
0.00272899653776371, 0.0028018328706636, -0.00919861598011906,
-0.00210080318639327, 0.00574081662706516, -0.0223170207999425,
-0.00649715490004299, -0.00458356807419513, 0.00567051294060583,
-0.0050920497264868, 0.00475188580729302, -0.00161509369523183,
0.0126682199278725, 0.0173431545355052, -0.00859216509959326,
0.0147423341154074, -0.00814860609999057, -0.0168910630806179,
-0.017394611699556, -0.0087886549707602, -0.0126761843887159,
-0.00564261535624044, -0.000163063285779752, 0.0109218040219923,
-0.0123496969365746, 0.0107583379894998, -0.00238224762817163,
-0.0140550600772716, 0.00447205724741084, 0.00849967727604639,
0.0134746513574004, -0.0175082604561083, 0.00624311091962526,
0.00961336070298957, -0.00351161432973557, 0.0119865692380838,
-0.000979411358491888, 0.00154179664734944, 0.00401563256651238,
0.00236642144094823, -0.0288956410622938, -0.00927928294838842,
-0.00724068360804975, -0.00883402112865139, -0.050578649594534,
-0.00175729735942876, 0.0284994362552065, -0.0160618664481138,
0.026578190890364, -0.00498545221124236, -0.0248455856607925,
0.00780351501080179, 0.0042865888105148, 0.00229749257318224,
-0.0480025938734183, 0.0426949269209413, -0.0219976598372297,
-0.0479621038301595, 0.0158124301340961, 0.0391427032421665,
-0.00628734072666692, -0.0114111785046729, -0.016733068328695,
0.0151141665758556, 0.0355934816870314, -0.0100512999471647,
-0.0150129338873702, -0.0258125201711763, -0.00248399910225972,
0.00734519052177562, 0.0201693332256125, -0.00302427056940058,
-0.0390865579429535, 0.00564706862745101, 0.0115523854356538,
-0.0132323129099419, 0.024162977965372, -0.00401476323174632,
0.0400416930502221, 0.00190573730988342, 0.00563264240007388,
1125.3989991, 1128.9430054, 1136.8930054, 1144.3530029, 1151.173999,
1158.3080078, 1167.6070068, 1173.8760009, 1178.3499999, 1183.494995,
1193.018994, 1203.9989867, 1212.4839843, 1217.4519897, 1221.0399902,
1222.8859863, 1225.2989868, 1229.2179931, 1233.0979858, 1235.0249878,
1234.4389893, 1232.6299927, 1230.7069947, 1230.6179932, 1232.1449952,
1234.6289918, 1234.0659913, 1232.0999879, 1229.8249879, 1228.1249879,
1224.0649903, 1220.2369874, 1215.8649903, 1214.1689942, 1214.8499878,
1213.7549926, 1217.246997, 1220.5099975, 1222.2329955, 1221.1559935,
1219.641992, 1216.0529905, 1211.9979856, 1206.3969847, 1199.9509886,
1193.1179808, 1185.7209715, 1179.0619749, 1172.8479857, 1169.2699828,
1167.7309814, 1169.2739868, 1169.3999878, 1170.2729858, 1171.0019897,
1172.7689941, 1174.7, 1176.7939942, 1180.7199952, 1184.6089966,
1187.7949951, 1185.9269897, 1185.0529907, 1182.6129883, 1178.0299805,
1168.1029786, 1156.5709717, 1148.2319702, 1137.9259643, 1130.0429687,
1121.3169677, 1113.2949707, 1107.2059692, 1102.4249755, 1098.911975,
1095.860974, 1097.485974, 1093.6249755, 1086.4079772, 1077.9009704,
1074.0089783, 1072.2119812, 1068.344989, 1062.2379822, 1057.449994,
1061.7179994, 1060.4010072, 1059.8690125, 1061.7240113, 1061.7080201,
1058.3970215, 1057.8680176, 1058.2380127, 1056.2290161, 1053.2240112,
1047.6460082, 1041.7940063, 1040.0410034, 1040.6190063, 1045.6369994,
1050.1010009, 1053.0730102, 45.4718259236287, 45.2372613787558,
56.9881807801438, 58.8717301068573, 68.2039283244873, 73.5215112680329,
78.8594307629251, 73.0335410836162, 71.845824268758, 73.323376014074,
89.1748677280385, 88.8275948061702, 88.079358554904, 72.9197089804835,
66.5774741060939, 65.5905607795046, 60.3560855296636, 60.5351059532554,
61.4085229097936, 58.076745639994, 63.2173375817626, 67.2733875032827,
68.7459719049055, 59.9037653356146, 44.6491666372171, 40.4929666577831,
30.2655738215587, 36.0522832244009, 40.7505784647263, 45.517250253278,
41.5835266382263, 41.3526668380199, 41.539756712543, 48.3189167794286,
49.8415866657383, 44.5858982397584, 50.0675010891207, 50.5139938354098,
44.9097955003298, 37.4247186375495, 41.3952548987526, 39.6467050713014,
39.3953595896288, 36.8289128008105, 42.8772642627352, 37.5760511024063,
42.0791664435174, 36.4236440580649, 25.1434697637668, 29.0666072154372,
25.3668839063101, 34.1040319281821, 34.1351918720353, 42.138526061446,
49.3942545777117, 53.2282422165058, 60.0907410718325, 59.6946479180297,
56.5126081396889, 64.5584522103826, 61.6638469740838, 48.5567687748239,
50.4491176695018, 45.8595330253583, 39.1134283844586, 22.2017732449298,
24.6509068125481, 33.7409449463083, 27.0354908046699, 36.9033514343542,
31.849732552439, 28.384694400023, 30.2843907497844, 30.2566110685775,
30.1702095862, 28.1229085893699, 39.7891005017724, 37.8236546439287,
33.4844836408483, 42.9231744072258, 49.6425369989148, 43.9761986844232,
44.7318583977582, 37.1424843378588, 40.8120228103859, 50.807226927847,
47.9214803669887, 44.995279725301, 41.3197867616665, 47.7401787161256,
40.9599257198947, 48.8101085201251, 58.7773921954413, 46.8976151314924,
38.7370234461344, 43.0052200556536, 42.7247275761847, 51.7764243779359,
47.5063348907638, 48.4623219235214, 51.3175593621287, 56.6142900584089,
-4.82006799999999, -1.57995600000004, 58.6699210000002, 70.3599850000001,
79.880005, 84.580078, 80.229981, 59.5, 82.020019, 81.2299800000001,
107.909912, 109.649902, 115.48999, 84.599976, 36.26001, 28.4400639999999,
36.1500249999999, 27.3499750000001, 27.829956, 37.8100589999999,
57.309937, 40.1099850000001, 1.02001999999993, -26.089966, -33.3199460000001,
3.59997599999997, -5.35998500000005, -10.7700199999999, -19.0500489999999,
-18.3800040000001, -22.089966, -17.4400639999999, -36.839966,
-24.959961, -7.2800299999999, -6.45996100000002, 14.290039, -2.97998099999995,
3.80993600000011, -9.48999000000003, -14.479981, -36.3300789999998,
-36.790039, -42.6899410000001, -28.0200199999999, -57.830078,
-66.48999, -58.619995, -93.25, -77.900024, -47.0999750000001,
-10.130005, -20.3900140000001, 1.93005399999993, 19.820068, 15.8499750000001,
17.2800299999999, 30.6500249999999, 19.9801029999999, 27.579956,
46.8999020000001, 6.9699700000001, -13.73999, -37.900024, -27.2700199999999,
-92.150024, -105.330078, -70.410034, -102.390015, -72.189942,
-79.6201179999998, -112.140014, -106.48999, -67.0299070000001,
-53.6600349999999, -98.26001, -43.25, -9.75, -59.2399290000001,
-73.869995, -15.4799800000001, -51.280029, -57.8999020000001,
-47.880005, -40.6499020000001, -7.77001900000005, -21.2899170000001,
15.440063, -56.9399410000001, -35.419922, 23.580017, 28.5, -15.2800299999999,
-50, -32.030029, -2.47998099999995, -31.9300539999999, -44.7700199999999,
-37.98999, 20.079956, 49.670044, 58.380005, 0.0222611532791754,
0.0452135091331929, 0.0513252799641881, 0.0417837604179434, 0.025502736394083,
0.039092663599452, 0.0357347902099496, 0.00293614990186253, -0.00329432207453095,
0.0181141450822189, 0.0402852378681278, 0.0551615667800078, 0.0663056560265511,
0.0442341189174087, 0.0117432944413327, -0.025003895701337, -0.0351848233779339,
-0.0338218411236912, -0.0120137739039121, 0.00411537451562882,
0.0202976386336982, 0.0207662940930682, 0.0185440740696787, 0.0112948922357141,
0.00832598771420034, -9.66018891714882e-05, -0.0253916951198745,
-0.0347079634534282, -0.0300608837885816, -0.0223033747170538,
-0.0331328340436947, -0.00582238916306288, -0.00092043524789176,
0.0162623189210285, 0.0278023013942841, 0.0242780692234357, 0.0341721602379419,
0.0276065721476719, -0.00201742129018712, -0.0367595754116454,
-0.0369577878306506, -0.0643497227781094, -0.0618263303728748,
-0.0449540628540124, -0.0165437954424839, -0.0201428806167474,
0.00331521089287889, 0.0065887162483893, -0.00740297497164288,
-0.0138034795252402, 0.00708686703803085, 0.0097706638268491,
-0.00550758934150863, 0.0148708818967105, 0.0199762409613848,
0.0079946880333539, 0.00652531170373116, 0.0232087632715174,
0.0185256691345472, 0.012965814191138, 0.0188472345861976, -0.0223894054190623,
-0.0307321178168358, -0.0395397517017582, -0.0524206155642792,
-0.106686826243081, -0.0791243287288781, -0.041700843048468,
-0.0506260747757752, -0.0155216754472765, 0.0313829883191348,
0.00798238469401547, -0.0123452722292345, 0.00812441164353306,
-0.0158118578835529, -0.0600069056977848, 0.00696117861714018,
-0.0230552625927682, -0.0764831282344325, -0.0630892695041085,
0.0244997487533043, -0.0236160780123607, -0.0128496474489905,
0.0194261429225948, 0.0187385123731847, 0.0153171376045522, 0.0115221715735752,
0.00787218840953852, -0.00140467331860261, -0.0188928482186341,
-0.0465491768740778, -0.0164783950656826, -0.0043804792361124,
-0.0180999145663217, -0.00998164151028913, -0.00581382135522279,
-0.0391030878479981, -0.0121985628088801, 0.0236495379306092,
0.0572791560421972, 0.0476969136015821, 0.0666343919981838, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0,
0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1,
1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,
1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0,
0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0), .Dim = c(102L,
13L), .Dimnames = list(NULL, c("open", "high", "low", "close",
"volume", "adjusted", "direction", "returns", "sma", "rsi", "momentum",
"roc", "pred")), index = structure(c(1531180800, 1531267200,
1531353600, 1531440000, 1531699200, 1531785600, 1531872000, 1531958400,
1532044800, 1532304000, 1532390400, 1532476800, 1532563200, 1532649600,
1532908800, 1532995200, 1533081600, 1533168000, 1533254400, 1533513600,
1533600000, 1533686400, 1533772800, 1533859200, 1534118400, 1534204800,
1534291200, 1534377600, 1534464000, 1534723200, 1534809600, 1534896000,
1534982400, 1535068800, 1535328000, 1535414400, 1535500800, 1535587200,
1535673600, 1536019200, 1536105600, 1536192000, 1536278400, 1536537600,
1536624000, 1536710400, 1536796800, 1536883200, 1537142400, 1537228800,
1537315200, 1537401600, 1537488000, 1537747200, 1537833600, 1537920000,
1538006400, 1538092800, 1538352000, 1538438400, 1538524800, 1538611200,
1538697600, 1538956800, 1539043200, 1539129600, 1539216000, 1539302400,
1539561600, 1539648000, 1539734400, 1539820800, 1539907200, 1540166400,
1540252800, 1540339200, 1540425600, 1540512000, 1540771200, 1540857600,
1540944000, 1541030400, 1541116800, 1541376000, 1541462400, 1541548800,
1541635200, 1541721600, 1541980800, 1542067200, 1542153600, 1542240000,
1542326400, 1542585600, 1542672000, 1542758400, 1542931200, 1543190400,
1543276800, 1543363200, 1543449600, 1543536000), tzone = "UTC", tclass = "Date"), class = c("xts",
"zoo"), .indexCLASS = "Date", .indexTZ = "UTC", tclass = "Date", tzone = "UTC", src = "yahoo", updated = structure(1544367929.32363, class = c("POSIXct",
"POSIXt")))
Problem:
I have quarterly level data. I need to perform a month over month analysis. Is there a function, or a ggplot feature that will interpolate the quarterly data and fill in monthly data?
Reference Data:
dput(HPF[1:25, ])
structure(list(region = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1), path = c(1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
), date = structure(c(16116, 16205, 16297, 16389, 16481, 16570,
16662, 16754, 16846, 16936, 17028, 17120, 17212, 17301, 17393,
17485, 17577, 17666, 17758, 17850, 17942, 18031, 18123, 18215,
18307), class = "Date"), index_value = c(1, 1.033852765, 1.041697122,
1.038876363, 1.041043093, 1.060900982, 1.073728928, 1.075879441,
1.080898915, 1.10368893, 1.119240863, 1.122827602, 1.128639801,
1.15275796, 1.169021733, 1.172707492, 1.178666441, 1.203634882,
1.220348482, 1.223890323, 1.229770019, 1.255791539, 1.273560554,
1.278236959, 1.285508086), index = c(0, 1, 2, 3, 4, 5, 6, 7,
8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,
24), counter = 1:25, BaseQoQ = c(0, 0.033852765, 0.00758749917354051,
-0.00270784947028013, 0.00208564760655761, 0.0190749923163842,
0.0120915582298895, 0.00200284535874973, 0.00466546139717505,
0.0210843166587877, 0.0140908661646175, 0.00320461762840418,
0.00517639483536669, 0.0213692260175751, 0.0141085757499344,
0.00315285755256367, 0.00508136004984272, 0.0211836361259394,
0.0138859385432799, 0.00290231933930496, 0.00480410367620832,
0.021159663675294, 0.0141496533844698, 0.00367191413499146, 0.00568840303732765
), fdate = structure(c(16116, 16205, 16297, 16389, 16481, 16570,
16662, 16754, 16846, 16936, 17028, 17120, 17212, 17301, 17393,
17485, 17577, 17666, 17758, 17850, 17942, 18031, 18123, 18215,
18307), class = "Date"), StressC = c(0.99749, 1.031342765, 1.039187122,
1.036366363, 1.038533093, 1.058390982, 1.071218928, 1.073369441,
1.078388915, 1.10117893, 1.116730863, 1.120317602, 1.126129801,
1.15024796, 1.166511733, 1.170197492, 1.176156441, 1.201124882,
1.217838482, 1.221380323, 1.229770019, 1.255791539, 1.273560554,
1.278236959, 1.285508086), StressQoQ = c(0, 0.0339379492526242,
0.00760596502560418, -0.0027143898728953, 0.00209069888540969,
0.0191210941026796, 0.0121202336548254, 0.00200753827606026,
0.00467637125510434, 0.0211333913794913, 0.0141229845362187,
0.00321182042946733, 0.00518799221722843, 0.021416855302633,
0.0141393626118667, 0.00315964160130755, 0.00509225924746737,
0.021228843485116, 0.0139149560969629, 0.00290830110260876, 0.0068690282969297,
0.021159663675294, 0.0141496533844698, 0.00367191413499146, 0.00568840303732765
)), .Names = c("region", "path", "date", "index_value", "index",
"counter", "BaseQoQ", "fdate", "StressC", "StressQoQ"), row.names = c(NA,
-25L), class = c("grouped_df", "tbl_df", "tbl", "data.frame"), vars = "region", drop = TRUE, indices = list(
0:24), group_sizes = 25L, biggest_group_size = 25L, labels = structure(list(
region = 1), row.names = c(NA, -1L), class = "data.frame", vars = "region", drop = TRUE, .Names = "region"))
Any insight would be much appreciated! I hope the data I have provided is enough to offer suggestions.
Goal:
I only need to plot/graph the monthly information, I do not need it reference in a data.frame.
So this is what you can do:
1) create a vector of dates to interpolate
months <- lapply(X = data$date, FUN = seq.Date, by = "month", length.out = 3)
months <- data.frame(date = do.call(what = c, months))
2) left join you date.frame to the months data.frame to create NAs for extrapolation
library(dplyr)
monthly_data <- left_join(x = months, y = data, by = "date")
3) use one of na.locf() na.appox() na.spline() to interpolate f.ex StressC
library(zoo)
monthly_data$StressC <- na.spline(object = monthly_data$StressC)
Note: the extrapolations above are
na.locf() - most recent point
na.appox() - linear
na.spline() - spline (often used in graphics)
Run the following (before interpolation) to see the difference:
plot(x = monthly_data$StressC, ylab = "StressC", xlab="", xaxt = "n")
lines(x = na.locf(monthly_data$StressC), col = "red")
lines(x = na.approx(monthly_data$StressC), col = "green")
lines(x = na.spline(monthly_data$StressC), col = "blue")
Can also do something like this to get gglot:
ggplot(monthly_data, aes(x=date)) +
geom_point(aes(y = StressC), colour="black") +
geom_line(aes(y = na.locf(StressC)), col="red") +
geom_line(aes(y = na.spline(StressC)), col="red")
I have an xts timeseries called hourplot in R with a period of 24 (hourly data) over two weeks, indexed by timestamp objects of POSIXlt class, like the following:
> dput(hourplot)
structure(c(1, 1, 1, 1, 1, 1, 1.11221374045802, 1.3368, 1.18,
1.0032, 1, 1, 1, 1, 1, 1, 1.0736, 1.2536, 1, 1.0032, 1.1856,
1.0048, 1, 1, 1, 1, 1, 1, 1, 1, 1.04045801526718, 1.20229007633588,
1.00229007633588, 1, 1, 1, 1, 1, 1, 1, 1.1152, 1.008, 1, 1, 1.2648,
1.1832, 1, 1, 1, 1, 1, 1, 1, 1.0424, 1.2952, 1.6496, 1.1208,
1.0216, 1, 1, 1, 1, 1, 1, 1.1256, 1, 1, 1, 1.0192, 1.3056, 1,
1.008, 1, 1, 1, 1, 1, 1, 1.0192, 1.0976, 1.0016, 1, 1, 1, 1,
1, 1, 1.0016, 1.6512, 1.8112, 1, 1, 1.2712, 1.0288, 1.0248, 1.1984,
1.0736, 1, 1, 1, 1, 1, 1.112, 1.336, 1.6224, 1.38, 2.2848, 1.628,
1, 1, 1, 1.0464, 1.4328, 1.6088, 1, 1, 1.0208, 1.2088, 1.02666666666667,
1.0784, 1.16, 1, 1.0064, 1.0616, 1.008, 1.0768, 1.4304, 1.3112,
1.0416, 1.00133333333333, 1.028, 1, 1, 1.1736, 1, 1.1264, 2.6744,
1.4696, 1, 1, 1.262, 1.2576, 1.0288, 1.1112, 1.008, 1, 1.01866666666667,
1.01733333333333, 1, 1, 1.294, 1.5744, 1.264, 1.182, 1.008, 1,
1, 1, 1, 1, 1, 1, 1.072, 1.8, 1.6424, 1.488, 1.1176, 1, 1, 1,
1, 1, 1.012, 1.2904, 1, 1, 1, 1, 1, 1.3072, 1.2056, 1.06, 1.0016,
1, 1, 1, 1.048, 1.0688, 1, 1, 1, 1, 1, 1, 1, 1, 1.51333333333333,
1.362, 1, 1, 1.0416, 1.03733333333333, 1.0288, 1.0712, 1, 1,
1, 1, 1, 1, 1.1664, 1.3464, 1.004, 1.0016, 1, 1, 1, 1, 1, 1,
1.15801526717557, 1.18396946564885, 1, 1, 1.1128, 1.2352, 1.0712,
1, 1, 1, 1, 1, 1, 1, 1.0744, 1.0048, 1, 1, 1, 1, 1, 1, 1, 1.0512,
1.0616, 1.4352, 1, 1.2064, 1.3368, 1.3296, 1.1608, 1.6696, 1.0328,
1.1976, 1.0912, 1.2024, 1, 1.0128, 1.1528, 1.26, 1, 1, 1.0192,
1, 1, 1, 1, 1, 1.704, 1.6152, 1, 1, 1.088, 1.4096, 1.0832, 1.1224,
1.0144, 1, 1.336, 1.552, 1.2248, 1.168, 1.4288, 1.4224, 1.2536,
1.0048, 1, 1, 1, 1.012, 1, 1, 1.21, 1.008, 1, 1, 1, 1.016, 1,
1.0048, 1, 1, 1.0176, 1.068, 1, 1, 1.0056, 1.2408, 1.0016, 1,
1, 1, 1, 1, 1.1632, 1.124, 1, 1, 1, 1, 1.0032, 1.0544, 1.0112,
1.008, 1.016, 1.0208, 1.084, 1.1688, 1.2384, 1.1736, 1.6168,
1.5984, 1.2784, 1.0608, 1, 1, 1, 1, 1.02824427480916, 1, 1.3064,
1.5216, 1, 1, 1.0672, 1.051), .Dim = c(346L, 1L), index = structure(c(1484722282,
1484725287, 1484729469, 1484731265, 1484737199, 1484740697, 1484744294,
1484747896, 1484751493, 1484755097, 1484758693, 1484762294, 1484765898,
1484769495, 1484773093, 1484776694, 1484780298, 1484783899, 1484787499,
1484791095, 1484794698, 1484798299, 1484800723, 1484805577, 1484809098,
1484812688, 1484816293, 1484819889, 1484823492, 1484827094, 1484830692,
1484834292, 1484837891, 1484841494, 1484845158, 1484848699, 1484852298,
1484855895, 1484859499, 1484863096, 1484866699, 1484870302, 1484873901,
1484877501, 1484881100, 1484884696, 1484888301, 1484891900, 1484895499,
1484899097, 1484902699, 1484906297, 1484909902, 1484913499, 1484917102,
1484920702, 1484924298, 1484927902, 1484931499, 1484935101, 1484938698,
1484942300, 1484945897, 1484949495, 1484953100, 1484956702, 1484960299,
1484963902, 1484967501, 1484971104, 1484974700, 1484978300, 1484981900,
1484985500, 1484989099, 1484992701, 1484996299, 1484999900, 1485003503,
1485007104, 1485010704, 1485014299, 1485017903, 1485021500, 1485025102,
1485028701, 1485032300, 1485035899, 1485039502, 1485043100, 1485046701,
1485050304, 1485053906, 1485057500, 1485061102, 1485064701, 1485068302,
1485071901, 1485075504, 1485079101, 1485082703, 1485086300, 1485089903,
1485093500, 1485097100, 1485100702, 1485104305, 1485107903, 1485111501,
1485115105, 1485118701, 1485122306, 1485125905, 1485129506, 1485133103,
1485136701, 1485140306, 1485143906, 1485147503, 1485151105, 1485154703,
1485158303, 1485161904, 1485165481, 1485169077, 1485172682, 1485176276,
1485179879, 1485183479, 1485187080, 1485190681, 1485194277, 1485197877,
1485201478, 1485205077, 1485208680, 1485212281, 1485215878, 1485219477,
1485223082, 1485226680, 1485230278, 1485233881, 1485237478, 1485241076,
1485244677, 1485248282, 1485251882, 1485255482, 1485259196, 1485262680,
1485265335, 1485312724, 1485316675, 1485320277, 1485323879, 1485327478,
1485331075, 1485334678, 1485338280, 1485341881, 1485345479, 1485349077,
1485352684, 1485356277, 1485359878, 1485363478, 1485367079, 1485370672,
1485374276, 1485377885, 1485381477, 1485385080, 1485388679, 1485392280,
1485395878, 1485399478, 1485403082, 1485406677, 1485410284, 1485413877,
1485417477, 1485421081, 1485424679, 1485428278, 1485431880, 1485435481,
1485439080, 1485442680, 1485446279, 1485449884, 1485453481, 1485457082,
1485460680, 1485464280, 1485467876, 1485471483, 1485475081, 1485478680,
1485482280, 1485485880, 1485489485, 1485493082, 1485496679, 1485500280,
1485503879, 1485507485, 1485511079, 1485514682, 1485518285, 1485521885,
1485525482, 1485529085, 1485532684, 1485536281, 1485539882, 1485543484,
1485547081, 1485550679, 1485554281, 1485557884, 1485561483, 1485565082,
1485568685, 1485572287, 1485575886, 1485579483, 1485583083, 1485586685,
1485590282, 1485593886, 1485597487, 1485601085, 1485604681, 1485608285,
1485611885, 1485615484, 1485619082, 1485622681, 1485626287, 1485629882,
1485633484, 1485637083, 1485640681, 1485644283, 1485647889, 1485651484,
1485655086, 1485658686, 1485662288, 1485665889, 1485669486, 1485673085,
1485676685, 1485680283, 1485683886, 1485687488, 1485691085, 1485694687,
1485698288, 1485701886, 1485705489, 1485709089, 1485712685, 1485716287,
1485719884, 1485723484, 1485727084, 1485730688, 1485734287, 1485737884,
1485741487, 1485745088, 1485748690, 1485752291, 1485755885, 1485759487,
1485763085, 1485766686, 1485770289, 1485773889, 1485777486, 1485781093,
1485784691, 1485788287, 1485791887, 1485795492, 1485799088, 1485802689,
1485806287, 1485809890, 1485813491, 1485817088, 1485820693, 1485824289,
1485827888, 1485831491, 1485835093, 1485838688, 1485842289, 1485845889,
1485849489, 1485853090, 1485856691, 1485860290, 1485863888, 1485867490,
1485871089, 1485874693, 1485878289, 1485881888, 1485885488, 1485889091,
1485892688, 1485896288, 1485899890, 1485903494, 1485907096, 1485910694,
1485914292, 1485917890, 1485921490, 1485925090, 1485928695, 1485932291,
1485935888, 1485939492, 1485943093, 1485946690, 1485950293, 1485953895,
1485957493, 1485961096, 1485964692, 1485968291, 1485971892, 1485975492,
1485979084, 1485982689, 1485986289, 1485989895, 1485993493, 1485997092,
1486000694, 1486004292, 1486006761), tzone = "", tclass = c("POSIXlt",
"POSIXt")), .indexCLASS = c("POSIXlt", "POSIXt"), .indexTZ = "", tclass = c("POSIXlt",
"POSIXt"), tzone = "", class = c("xts", "zoo"), frequency = 24)
Now, I want to decompose this timeseries and look at the seasonality, so I run:
dec <- decompose(as.ts(hourplot))
plot(dec)
The plot I get looks like:
Now, I want to have the timestamps mentioned under the individual trend and seasonality plots too, but R seems to strip off the timestamp and put 1-14 in the X axis instead.
How can I preserve the timestamps in the plots?
Here's a function that will decompose an xts series and return an object of class "decomposed.xts".
decompose.xts <-
function (x, type = c("additive", "multiplicative"), filter = NULL)
{
dts <- decompose(as.ts(x), type, filter)
dts$x <- .xts(dts$x, .index(x))
dts$seasonal <- .xts(dts$seasonal, .index(x))
dts$trend <- .xts(dts$trend, .index(x))
dts$random <- .xts(dts$random, .index(x))
with(dts,
structure(list(x = x, seasonal = seasonal, trend = trend,
random = if (type == "additive") x - seasonal - trend else x/seasonal/trend,
figure = figure, type = type), class = "decomposed.xts"))
}
And here's a plot.decomposed.xts() method:
plot.decomposed.xts <-
function(x, ...)
{
xx <- x$x
if (is.null(xx))
xx <- with(x,
if (type == "additive") random + trend + seasonal
else random * trend * seasonal)
p <- cbind(observed = xx, trend = x$trend, seasonal = x$seasonal, random = x$random)
plot(p, main = paste("Decomposition of", x$type, "time series"), multi.panel = 4,
yaxis.same = FALSE, major.ticks = "days", grid.ticks.on = "days", ...)
}
And an example of using it on your data:
dex <- decompose.xts(hourplot)
plot(dex)
I would translate the decomposed data into a dataframe:
n = length(index(hourplot))
df1 = data.frame(date = index(hourplot), name = rep("random", n), data = as.numeric(dec$random))
df2 = data.frame(date = index(hourplot), name = rep("seasonal", n), data = as.numeric(dec$seasonal))
df3 = data.frame(date = index(hourplot), name = rep("trend", n), data = as.numeric(dec$trend))
df4 = data.frame(date = index(hourplot), name = rep("observed", n), data = as.numeric(dec$x))
df = rbind(df1, df2, df3, df4)
And then use ggplot2:
library(ggplot2)
ggplot(df, aes(x = date, y = data)) +
geom_line() +
facet_wrap(~name, ncol = 1, scales = "free") +
scale_x_datetime(date_labels = "%d", date_breaks = "24 hours")
I'll leave it to you to fine-tune the scales and labels in scale_x_datetime.