Related
I'm facing a problem that
rq(y ~ x, tau = 0.50, method = "br")
doesn't complete calculation. There is no error and warning.
I traced code and found .Fortran() in rq.fit.br() does not finish calculation. I'm not familiar with Fortran so let me ask this question here.
I would like to know boundary condition that causes this infinite loop. I can avoid this endless calculation if I know it.
Thank you for your help in advance.
# The problematic input data :
y <- c(
0, 0, 0, 0, 0, 0, 0.234, 0.117, 0.351, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0,
0.117, 0, 0, 0, 0, 0.117, 0.117, 0, 0.117, 0.117, 0, 0, 0.117, 0, 0, 0, 0,
0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.234, 0, 0.117, 0.117, 0,
0.351, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.234, 0, 0, 0, 0.117, 0.117, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0,
0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.117, 0, 0, 0, 0, 0.117, 0.117, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.117, 0, 0, 0, 0.234, 0, 0.234, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0.234,
0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.585, 0, 0, 0.117, 0.234,
0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0.234, 0.117, 0, 0, 0.117, 0.234, 0, 0.117, 0,
0, 0, 0.234, 0, 0.117, 0, 0.117, 0.117, 0, 0, 0, 0, 0.234, 0, 0, 0.234, 0.234,
0.234, 0.234, 0.117, 0, 0, 0, 0, 0, 0, 0.117, 0.117, 0, 0, 0, 0, 0, 0, 0, 0,
0.117, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0.117,
0.117, 0, 0.117, 0, 0.117, 0, 0, 0.117, 0, 0.117, 0, 0, 0.234, 0, 0, 0, 0, 0,
0, 0, 0.234, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0.234, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0.234, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0.117, 0, 0.117, 0, 0, 0, 0.117,
0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0.117, 0, 0, 0.234, 0.351, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0,
0.117, 0, 0, 0, 0.117, 0.117, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0,
0.117, 0, 0, 0.117, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117,
0.234, 0, 0, 0, 0, 0, 0.117, 0.117, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.117, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0,
0, 0.117, 0, 0.703, 0.117, 0.117, 0.234, 0, 0.117, 0, 0.117, 0.117, 0, 0, 0,
0.117, 0, 0, 0, 0.234, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.234, 0, 0.117, 0, 0, 0,
0.117, 0, 0, 0, 0, 0, 0.117, 0.234, 0.234, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117,
0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0,
0, 0, 0, 0.117, 0.234, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0.234, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0.234, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.234, 0, 0,
0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0.351, 0, 0, 0, 0, 0, 0, 0, 0.117, 0.117, 0,
0.234, 0.117, 0, 0, 0.117, 0.234, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0.117, 0.351, 0, 0.468, 0.234, 0.234, 0, 0, 0, 0, 0.234, 0, 0.117, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117, 0, 0,
0.117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.234, 0, 0,
0, 0, 0, 0, 0, 0.117, 0, 0, 0.585, 0, 0, 0.234, 0, 0.117, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0.468, 0, 0, 0, 0, 0.117, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.234, 0, 0, 0, 0, 0, 0, 0.234, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0.351, 0.234, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117,
0.117, 0, 0, 0, 0
)
x <- c(
24.4018079, 28.8807817, 40.4440632, 22.6231939, 21.5390181, 27.6194884,
32.9537276, 23.0931019, 24.6333219, 32.2419818, 23.401991, 32.4394215,
18.980218, 36.1419718, 52.0712614, 24.5704906, 60.3779992, 31.7383285,
33.7286139, 17.3276892, 30.7991282, 50.9481105, 31.080127, 53.3908315,
17.6551953, 27.7416689, 27.2468471, 27.1576005, 23.1742951, 41.4709552,
41.967935, 27.8544524, 19.8297627, 27.620431, 15.2351169, 18.3377948,
25.0163879, 25.2405723, 33.3461355, 48.2266249, 24.3607428, 27.7216683,
26.6296912, 28.4811453, 30.1760888, 26.8148476, 25.128683, 23.8027916,
30.7355199, 30.5610606, 41.6265463, 26.175877, 27.9212294, 37.9277741,
25.1736093, 24.4808208, 28.8720055, 23.4672164, 20.0011668, 24.0894199,
28.4711092, 21.9160106, 26.516336, 16.7173241, 24.9135346, 26.2081573,
21.091112, 32.6515246, 30.2453634, 24.3514448, 27.8053967, 27.0362428,
25.8530264, 40.8166319, 21.7807126, 30.2581439, 35.0799978, 38.2687556,
46.9241502, 35.4400451, 25.0417079, 20.9683041, 18.9136417, 26.9527511,
32.5955477, 24.1518614, 33.2041696, 29.5174409, 31.9583297, 38.5216324,
46.6958841, 31.8675876, 29.4891597, 32.2909144, 27.8331436, 27.3906857,
36.2563286, 17.9700065, 26.2149315, 29.4081605, 40.4465888, 24.6274318,
32.0388361, 22.6569988, 21.6720743, 16.7256267, 16.852218, 13.8423948,
25.4936663, 26.6811577, 26.812348, 29.8681522, 27.7055245, 15.006916,
27.7348524, 26.4439379, 30.0100718, 37.0017757, 14.5881997, 31.0969282,
18.1142671, 14.4049902, 37.6274298, 44.3967524, 27.5595956, 19.4116367,
20.0019715, 16.9236606, 26.246288, 35.607556, 30.3840455, 30.3815949,
24.5976682, 37.809445, 21.577438, 21.8125242, 14.5899396, 23.6670936,
25.268452, 31.6692812, 22.690343, 24.3731279, 33.3598046, 29.6117162,
39.1297631, 15.2346245, 26.0681545, 25.4259568, 42.6836935, 23.7939986,
41.4657106, 25.8638121, 41.9544915, 26.7980014, 19.7041992, 19.8906546,
33.3622323, 28.7616006, 36.8331863, 38.3432724, 13.2298725, 18.7853833,
25.4594015, 30.0428453, 24.0999749, 28.5411825, 23.7460138, 15.7947625,
36.502406, 46.1166203, 26.2032695, 19.0759098, 25.7247696, 27.8923323,
35.3435019, 28.7847773, 28.7935475, 18.5619149, 39.1965545, 28.0817298,
26.4450131, 28.0224018, 30.3223318, 35.6540999, 28.2432116, 25.3455388,
25.9718787, 32.34494, 44.2724262, 31.45872, 29.6019651, 20.3401629,
18.4580252, 37.232467, 34.3359164, 44.1724399, 23.4782245, 27.4645108,
27.1781198, 18.6422675, 19.2272688, 30.8315997, 25.5801722, 22.4312101,
15.3473977, 27.472274, 24.4604204, 22.5680019, 14.6711321, 17.4751012,
59.8056645, 24.4605417, 22.4188914, 24.111513, 18.9521777, 29.562426,
21.1020296, 31.1739265, 24.7056341, 22.9724849, 22.0160131, 30.531877,
18.4457535, 28.0308361, 27.8577984, 23.3738145, 33.924868, 23.397325,
29.6069946, 18.181721, 31.6132161, 25.0790305, 28.2104791, 22.931505,
29.4002554, 18.5959381, 25.0670957, 18.7296132, 24.770169, 24.9230376,
29.4113636, 20.8127034, 24.3650603, 16.2016467, 24.9011517, 28.7432186,
31.7488893, 15.4118624, 32.5145935, 15.0132364, 22.2026089, 28.3184513,
30.7058983, 30.7271516, 42.0031247, 44.6998899, 36.0661786, 21.6462755,
32.1853902, 43.8380628, 27.4295037, 33.3694872, 24.8508583, 26.4525086,
27.7657697, 28.7656539, 27.8993627, 24.1259411, 31.7960144, 20.1717771,
24.6846924, 31.3695283, 16.9871094, 27.3006097, 24.353586, 24.2973128,
22.2143346, 32.5385164, 46.238405, 15.87873, 24.8137702, 31.6289988,
16.7381839, 28.4660331, 38.6910685, 34.4105544, 26.0612181, 21.1271684,
26.6423251, 24.5583478, 23.493619, 26.539979, 28.6731828, 28.239095,
18.9509248, 24.0585342, 27.0629323, 25.4465092, 26.7561273, 32.5888733,
34.5808151, 29.2271733, 39.7868991, 25.690253, 39.4753614, 15.3304099,
43.0529772, 26.977613, 12.5825155, 19.4012753, 21.718714, 26.4869929,
26.1744893, 18.1392682, 27.6152192, 21.0045136, 28.0224205, 26.5692797,
28.9952192, 35.6324021, 27.6756153, 24.1850861, 27.8542562, 30.6187182,
25.4835887, 27.6732922, 27.6414935, 19.2370806, 26.8892099, 23.5372461,
26.8561243, 31.2019592, 42.2955339, 30.0204426, 32.4021614, 20.7569456,
27.2942332, 23.9750141, 26.2850638, 16.7624382, 33.0947322, 24.0859599,
32.1946124, 29.4301999, 30.2732967, 21.4577098, 49.1818696, 51.8139063,
38.768469, 35.5710685, 26.370072, 30.6496823, 39.3829068, 15.2329246,
26.0432293, 35.6630347, 29.6303257, 41.2432668, 45.2378368, 34.3707241,
30.5024637, 19.9775718, 17.801213, 26.1688732, 28.4875885, 29.7568102,
29.1107832, 36.2328291, 31.8214314, 38.6204676, 36.8834063, 42.6788448,
28.7272819, 28.626164, 34.291455, 13.2104893, 46.3331664, 25.6895379,
15.6121566, 24.8888988, 15.5789419, 26.7369002, 46.104393, 23.2856908,
20.3333047, 30.5485, 15.6038882, 28.7037872, 32.3490014, 28.0980547,
30.1026873, 15.7965292, 23.5073005, 23.9334505, 19.6343642, 22.1035284,
24.9623722, 17.4625706, 19.5039162, 28.129359, 26.2476225, 26.7068015,
30.9865519, 26.9848648, 25.3245441, 27.8976892, 27.4579313, 41.9166111,
17.2998378, 29.7315102, 25.7406406, 22.2793813, 36.3643588, 20.9277299,
44.1597183, 27.5133605, 25.6170785, 50.9843355, 26.9664186, 49.9873232,
20.9170467, 24.1125628, 36.8751858, 34.3871658, 20.4164672, 25.8413742,
40.4310466, 21.2163069, 36.3361406, 38.0197324, 33.3294001, 21.4165966,
28.4923429, 22.1280619, 22.4756409, 30.2215385, 41.7292709, 36.8907943,
31.5413965, 26.1837229, 37.0218326, 25.6081831, 28.1776375, 29.8671555,
29.3736138, 35.4604974, 18.7050421, 22.255564, 33.6674798, 29.5201891,
24.0745051, 31.8653496, 27.0345565, 26.0525699, 23.0252921, 28.5281226,
33.71177, 29.0951349, 21.7951779, 31.0860777, 22.6393045, 19.0780616,
24.3621088, 29.5905486, 26.5586841, 42.5486628, 33.6081618, 25.1165999,
17.9542253, 24.9186609, 38.5966133, 17.0141855, 23.0060186, 30.1943135,
27.1080814, 28.6187819, 24.1290464, 41.5916513, 43.4616517, 21.3195541,
26.1493297, 20.9077832, 25.7692305, 29.4261399, 29.2463376, 28.8000356,
25.1911071, 35.8697203, 22.2615425, 31.4633835, 24.1877693, 22.8952581,
15.1601842, 32.3567013, 27.3661214, 39.0040261, 32.0344394, 18.0945006,
23.8093766, 31.8086423, 40.3121132, 30.3588501, 25.9724817, 44.6408476,
43.4099194, 30.3728664, 33.5631618, 23.0705636, 27.4936808, 36.8187291,
32.9492242, 29.5742094, 25.5605033, 23.7780532, 25.3959579, 38.1226182,
24.9726063, 46.6698573, 51.4411263, 25.9290155, 31.1392917, 52.7543492,
31.7838842, 30.0237536, 25.819024, 29.7256644, 28.6428477, 32.1406846,
42.8654594, 33.7451799, 37.9804767, 28.7298898, 26.8582201, 46.1198892,
32.7408768, 55.5203714, 31.495731, 51.0941729, 24.314699, 33.4828115,
24.4570093, 29.5138786, 21.5354954, 30.317556, 24.6148894, 17.7956648,
31.7810597, 30.4743448, 19.8238082, 32.3247417, 27.3858365, 29.8964156,
29.7147584, 25.2144471, 43.6346636, 28.5816332, 26.0747485, 39.6629311,
29.0223235, 28.5093962, 26.6550715, 30.5273411, 20.8669945, 29.1348172,
21.710507, 24.8998055, 32.975078, 39.652974, 17.3980911, 40.1282322,
35.8886658, 27.829678, 49.7612583, 31.9280938, 28.5301198, 27.4779753,
25.1837491, 32.5874968, 25.1871786, 21.9365623, 26.8613984, 24.9095698,
32.8732303, 27.1493754, 22.0098701, 34.8794858, 32.4926707, 15.6674662,
24.9172861, 27.4232858, 28.6305736, 32.8272527, 20.0910068, 40.1099478,
23.6793994, 26.1303681, 32.4211306, 36.3060647, 13.3999648, 29.8795415,
42.9840395, 24.9583988, 22.7460185, 43.3399476, 25.5209943, 18.1369994,
28.7716568, 25.3741116, 30.5742097, 32.1805709, 23.9898692, 28.0885658,
30.6875465, 22.8677012, 34.825549, 29.3668426, 34.999954, 41.5279957,
22.285163, 24.0556159, 15.200475, 21.4487252, 19.0435518, 30.7447739,
15.9420322, 16.7287114, 28.6306528, 29.554746, 16.4364149, 31.3476225,
34.9007223, 30.0167197, 27.4047092, 27.0686892, 26.8545017, 16.7846713,
23.2117939, 31.6776801, 29.9123239, 32.8621553, 21.3552832, 50.75349,
23.991315, 29.7791991, 24.7992546, 27.5752878, 17.4821681, 26.911058,
25.3942484, 31.8553779, 29.6222535, 22.8196871, 31.0947927, 41.7734339,
35.4348117, 20.5096086, 28.8058813, 38.8916449, 24.7993731, 33.1395299,
21.5910174, 27.5014509, 15.826052, 27.9783595, 40.0840328, 28.2133025,
32.194761, 54.2405209, 31.9051349, 25.4699985, 25.2212782, 22.0680464,
14.9236318, 37.6760713, 33.189865, 33.3475948, 23.3487752, 26.42338,
30.3284095, 24.8590412, 15.1272312, 24.4415683, 24.3445192, 32.8710454,
43.5016354, 38.9345868, 15.6964367, 33.7304737, 21.6569327, 26.3130489,
22.4141185, 28.3356213, 31.1180693, 26.1339403, 19.5924288, 27.3062312,
70.0232447, 31.4218925, 31.6894943, 70.6923221, 27.8249275, 45.9796955,
27.5667195, 39.3534247, 25.2532679, 26.6210675, 42.0462846, 57.8231799,
32.1834225, 35.3014761, 21.4324152, 30.7373532, 29.6940796, 51.0814799,
26.3502205, 19.3649445, 24.7811548, 21.3144558, 44.4579779, 30.3942605,
24.9829469, 30.9145726, 22.6600533, 33.542845, 25.0688544, 23.3785057,
31.7691544, 30.4176433, 37.8272268, 36.0808069, 31.3964826, 18.1158916,
22.5676528, 27.3483272, 23.0186353, 20.550375, 39.3741719, 20.2418609,
21.1396468, 66.5203361, 62.4761334, 29.6869787, 26.3493349, 37.4383524,
11.9058338, 26.0618828, 27.7359828, 16.1160718, 15.9020783, 24.2150908,
25.8994792, 27.4616096, 32.8780234, 27.1384018, 14.7083443, 36.2070307,
36.4121969, 30.5070904, 29.0891587, 20.2350553, 25.2963738, 15.7358134,
16.9517607, 34.2212512, 27.7858155, 28.1410475, 29.6620824, 26.7803718,
27.2988802, 19.9575323, 30.1752288, 27.5984041, 28.1399913, 16.9137777,
32.6449213, 23.451331, 30.4658547, 21.8190914, 22.1539569, 24.4125483,
24.8686785, 26.2065556, 16.5003938, 35.8479682, 24.0469412, 40.4014116,
33.4889847, 22.8764475, 47.3298653, 18.5901704, 30.3538952, 36.3691136,
31.4970049, 19.9467581, 29.2147637, 27.4377932, 36.0034333, 37.0777657,
29.8718446, 32.4984043, 32.6625905, 35.8877784, 32.6524614, 54.8709701,
16.2960404, 25.8058264, 23.3658142, 22.3081803, 19.9078811, 63.384812,
28.1309744, 37.47588, 30.4532575, 44.6020145, 33.2618816, 24.0572143,
13.706158, 37.4377234, 26.3083542, 21.7035413, 55.5107049, 15.4091252,
41.8114748, 32.530624, 51.0474142, 46.2803631, 23.4427667, 38.9682187,
26.6724182, 21.1864403, 40.0251724, 36.3530887, 17.4979165, 35.06262,
23.74311, 36.424138, 27.5016505, 16.5533966, 21.071959, 32.0166696,
31.6319279, 58.1904527, 29.5304514, 26.0416582, 33.6718921, 40.8230363,
38.4883028, 27.2786438, 24.1567016, 18.165015, 39.4164839, 25.5206923,
29.2113946, 54.5010605, 28.7706133, 18.5841954, 44.3722705, 20.11255,
29.4044921, 30.7372406, 14.0238355, 33.6927312, 16.1737632, 38.0253688,
26.6644105, 18.53842, 27.7307437, 51.8477646, 32.6874462, 30.5680848,
30.9982972, 27.7979667, 21.532476, 24.7052444, 39.1467731, 58.5943291,
23.1092334, 36.9199715, 29.3797749, 15.8163776, 28.6779247, 33.0831701,
22.6006952, 22.7515948, 38.4527168, 31.9983674, 40.9573834, 23.3273802,
19.8011584, 35.6943063, 35.3977388, 22.0540011, 34.1045856, 27.5684501,
25.5696024, 23.621706, 21.9372108, 30.2297453, 32.0413651, 34.8618642,
37.8348272, 25.6725508, 26.6521851, 18.6582369, 35.5246539, 25.1168522,
25.1152485, 27.4422305, 27.5549447, 40.7587422, 46.997187, 16.5311856,
28.7304576, 28.3537612, 29.3829696, 30.9081955, 33.0904867, 54.6381586,
38.9128035, 35.0343507, 23.9988363, 28.0781288, 22.4621496, 32.9971513,
25.2213571, 30.1988958, 23.9096162, 46.0506056, 32.1925042, 35.8170223,
23.8509302, 38.5036865, 38.731735, 29.397908
)
Pity the poor simplex algorithm! In this example there are 934 observations, 780 of which are 0. So simplex would like to choose which two of the 780 it should pick to make a solution to say that the conditional median of y|x is 0. Any two will do, but how to choose? One could dither the y's, dither(y, value = 1e-8) -- this breaks the ties, and delivers a single precision answer, but as the other responder suggests it is better to switch to method = "fn". Even better would be to avoid using QR at all for problems like this where the response variable is discrete.
Your response vector is a bit too sparse so I am not sure how efficiently the default algorithm can solve that, if you check the help page for rq, under method:
The default method is the modified version of the Barrodale and
Roberts algorithm for l1-regression, used by ‘l1fit’ in S, and is
described in detail in Koenker and d'Orey(1987, 1994), default =
‘"br"’. This is quite efficient for problems up to several thousand
observations, and may be used to compute the full quantile
regressionprocess. [...]For larger problems it is advantageous to use
the Frisch-Newton interior point method ‘"fn"’.
If you need something specifically from the default algorithm, you can try:
library(quantreg)
fit = rq(y ~ x, tau = 0.50, method = "fn")
I have two matrices and I need to do a row name and column name match to conduct element by element calculations. The first calculation is phij/pij and the second is ((phij-pij)^2)/pij
The long matrix phij has row names separated by a dash e.g. Aaa-Baa. The column names have no dash. I need to match the part of the row name after the dash i.e. Baa and a column name in the phij matrix to the row name and column name of the smaller matrix pij.
I tried using a for loop but it's not matching the actual row names and column names but instead looks up positions in the sequence.
LR<-phij
ChiSq<-phij
ROWS <- data.frame(ROW0=rownames(phij),
ROW1=substr(rownames(phij),regexpr("-", rownames(phij))+1,nchar(rownames(phij))))
COLNAMES <- c(colnames(phij))
for(rowN in 1:length(ROWS$ROW0)){
for(colN in COLNAMES){
LR[ROWS$ROW0[rowN],colN]<-LR[ROWS$ROW0[rowN],colN]/pij[ROWS$ROW1[rowN],colN]
ChiSq[ROWS$ROW0[rowN],colN]<-((ChiSq[ROWS$ROW0[rowN],colN]-pij[ROWS$ROW1[rowN],colN])^2)/pij[ROWS$ROW1[rowN],colN]
}
}
Data:
phij:
structure(c(0.111111111111111, 0.2, 0, 0, 0, 0.25, 0, 0.666666666666667,
0.166666666666667, 0, 0, 0.666666666666667, 0.5, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 0.333333333333333, 0, 1,
0, 0.166666666666667, 0, 0.571428571428571, 0, 0, 0, 0.4, 0.272727272727273,
0, 0, 0, 0, 0, 0, 0, 0, 0.222222222222222, 0.6, 0, 0, 0.25, 0,
0, 0.333333333333333, 0.333333333333333, 0, 0, 0.333333333333333,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 1, 0, 0, 0, 0, 0, 0.2, 0.666666666666667,
0, 0, 0, 0.166666666666667, 0, 0.142857142857143, 1, 0, 0, 0.2,
0.272727272727273, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0.222222222222222, 0.2, 0, 0, 0, 0,
0, 0, 0.166666666666667, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0.333333333333333, 0, 0.142857142857143,
0, 0, 0, 0.2, 0.181818181818182, 0, 0, 0, 0, 0, 0, 0, 0, 0.444444444444444,
0, 0, 0, 0.75, 0.5, 0, 0, 0.333333333333333, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 1, 1, 0, 0.2, 0, 0, 0, 1, 0.333333333333333,
0, 0.142857142857143, 0, 0, 1, 0, 0.272727272727273, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), class = "table", .Dim = c(49L,
7L), .Dimnames = list(i = c("A-A", "A-Aa", "A-Aaa", "A-B", "A-Ba",
"A-Baa", "A-Caa", "Aa-A", "Aa-Aa", "Aa-Aaa", "Aa-B", "Aa-Ba",
"Aa-Baa", "Aa-Caa", "Aaa-A", "Aaa-Aa", "Aaa-Aaa", "Aaa-B", "Aaa-Ba",
"Aaa-Baa", "Aaa-Caa", "B-A", "B-Aa", "B-Aaa", "B-B", "B-Ba",
"B-Baa", "B-Caa", "Ba-A", "Ba-Aa", "Ba-Aaa", "Ba-B", "Ba-Ba",
"Ba-Baa", "Ba-Caa", "Baa-A", "Baa-Aa", "Baa-Aaa", "Baa-B", "Baa-Ba",
"Baa-Baa", "Baa-Caa", "Caa-A", "Caa-Aa", "Caa-Aaa", "Caa-B",
"Caa-Ba", "Caa-Baa", "Caa-Caa"), j = c("A", "Aa", "Aaa", "B",
"Ba", "Baa", "Caa")))
pij:
structure(c(0, 0, 0, 0, 0, 0, 0, 0, 0.608695652173913, 0.323529411764706,
0.129032258064516, 0.176470588235294, 0.125, 0, 0, 0.173913043478261,
0.323529411764706, 0.258064516129032, 0.294117647058824, 0.25,
0, 0, 0.0869565217391304, 0.235294117647059, 0.419354838709677,
0.352941176470588, 0.25, 0, 0, 0.130434782608696, 0.117647058823529,
0.161290322580645, 0.117647058823529, 0.25, 0, 0, 0, 0, 0.032258064516129,
0.0588235294117647, 0.125, 0, 0, 0, 0, 0, 0, 0, 0), class = "table", .Dim = c(7L,
7L), .Dimnames = list(i = c("Aaa", "Aa", "A", "Baa", "Ba", "B",
"Caa"), j = c("Aaa", "Aa", "A", "Baa", "Ba", "B", "Caa")))
You can create a new matrix of pij which is of same dimension as phij and then perform the calculations that you want.
new_pij <- pij[sub('-.*', '', rownames(phij)), colnames(phij)]
You can then do :
phij/new_pij
and
((phij-new_pij)^2)/new_pij
I've created a new data frame and the rownames got named like char(X1, X2, X3, ... Xn).
In order to merge the new data frame with an old one I need them to be num(1,2,3,...,n).
# Create DB with Topics
df_test <- data.frame(doc_topic_distr)
setDT(df_test, keep.rownames = "doc_id")
I've tried to df_test$doc_id <- as.integer(gsub('[a-zA-Z]', '', df$doc_id))them afterwards, but that's not working. :/
Any clues for this one?
/e:Here we go:
> df_test$doc_id <- gsub('[a-zA-Z]', '', df$doc_id)
Error in df$doc_id : object of type 'closure' is not subsettable
>
> dput(head(doc_topic_distr))
structure(c(0, 0, 0, 0, 0, 0.037037037037037, 0, 0.08, 0, 0,
0, 0, 0.25, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.32,
0, 0, 0.875, 0.407407407407407, 0, 0.16, 0, 0.166666666666667,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.04, 0, 0, 0, 0.0740740740740741, 0, 0.12,
0, 0, 0, 0.037037037037037, 0, 0.04, 0, 0, 0, 0, 0, 0.08, 0,
0, 0, 0.0740740740740741, 0.25, 0, 0, 0, 0.0625, 0.037037037037037,
0, 0, 0, 0.333333333333333, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0.222222222222222, 0, 0, 0, 0, 0, 0.037037037037037, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0625, 0, 0, 0, 0, 0, 0, 0.037037037037037,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.037037037037037, 0, 0, 0,
0, 0, 0, 0, 0.16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), .Dim = c(6L,
31L), .Dimnames = list(c("0", "1", "2", "3", "4", "5"), NULL))
´´´
Many thanks in advance!
Solved it like this:
df <- data.frame(doc_topic_distr)
df <- cbind(doc_id = rownames(df), df)
df$doc_id <- as.numeric(as.character(df$doc_id))
In the data below, there are values for 2005and 2006. Each year has 2 columns as h1and h24. Function myfun is using h1to aggregate hourly data to 6, 12 and 24 hourly data as h6,h12 and h24 and finding means of them as mean1, mean6, mean12 and mean24 for each year separately and listing them.
Head of the data:
X2005.h1 X2005.h24 X2006.h1 X2006.h24
1 0 0.0 0 0.0
2 0 0.0 0 8.6
3 0 3.4 0 11.2
4 0 21.4 0 8.4
5 0 1.8 0 0.0
6 0 1.4 0 0.0
And my whole data is:
myd<-structure(list(X2005.h1 = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0.6, 0.2, 0, 0, 0.6, 0.2, 0.6, 0.6, 0, 0, 0, 0.6, 1.2, 1.8,
1.8, 1.2, 1, 1.2, 1.6, 1.2, 1.4, 1, 1.2, 0.8, 0.8, 0.2, 0, 0,
1.6, 0.4, 0, 0.4, 1.2, 0.8, 0.2, 0.4, 0.2, 0.4, 0.4, 0.2, 0.2,
0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.2,
0.4, 0.4, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.2, 0.4, 1.2, 1.6, 2.2, 1.8, 0.4, 0.6, 0,
0.2, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.2, 0.2, 0, 0.6, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.4, 0.2, 0, 0, 0.8, 0, 0, 0.2, 0, 0, 0, 0, 0.2, 0.2, 0.2, 0,
0, 1.2, 4, 0.2, 0, 0, 0, 0, 0, 0, 0, 1.6, 0.6, 0, 0, 0, 1.4,
0.6, 0.4, 0.8, 0, 0, 0, 0, 0.2, 0, 0.2, 0, 0, 0, 0, 0.6, 0, 0,
0, 0, 0.2, 1.2, 1, 1.8, 2, 1.8, 0.4, 0.6, 1.2, 1.8, 1.4, 1.2,
0.6, 0.4, 0, 0.2, 0.2, 1.6, 1.6, 1.4, 0.8, 0.4, 0.2, 0.2, 0,
0.6, 0, 0.4, 0.4, 0.2, 0.2, 0.2, 0.4, 0.4, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.6, 0, 0, 0, 0.2, 0.2, 0.4,
0.4, 1.6, 1.8, 1.2, 1.4, 1.4, 1.4, 0.6, 0.6, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), X2006.h1 = c(0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 2, 4.8, 1, 0, 0, 0, 0, 0, 0, 0, 0.8, 0, 0, 0, 0,
0, 0, 6.6, 0.2, 0, 0, 0.2, 0, 0, 0.2, 0, 1.4, 0, 1.2, 0, 0.2,
0, 0.2, 1, 0, 0, 0, 0, 0, 0.2, 1.8, 2, 2.2, 0.6, 0.8, 0, 0, 0,
0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0.4, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 1.8, 5.8, 0.2, 0.4, 0,
3.8, 0.2, 1.2, 0.4, 0, 0, 1, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.6, 0.4, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.8, 0.8, 1, 0.2,
0, 0.2, 0, 0, 0.6, 2.4, 0.2, 0, 0.2, 0, 0, 1.6, 0.2, 2, 0, 1.2,
4.6, 0.6, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 0, 0, 0, 0, 0, 0.2, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.8, 0.4, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.2, 0, 0, 0, 0, 0.2, 0,
0, 0, 0, 0, 0, 0, 0, 0.2, 1.8, 0.6, 0, 0, 0, 0.4, 2, 1, 0.8,
0.4, 0.2, 0, 0.2, 0, 0.8, 0.8, 0.2, 0.2, 0, 0.4, 0.4, 0.6, 0.8,
4, 2, 0.8, 2.6, 1.4, 0.6, 1, 1, 1, 1.2, 1.2, 0.8, 1.6, 1.8, 0.4,
0.2, 0.2, 0.2, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.6, 1.2, 0.6, 0.8, 0.8, 1, 0.8, 1.2, 1.2, 0.8, 1, 2.2, 3.4,
2.6, 1.8, 1.6, 1.8, 3.6, 3, 1.8, 3.2, 1.8, 1, 0.6, 0.4, 0, 1,
0.6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3.6, 3.8, 0.2, 0.4,
0, 0.2, 0.2, 0.4, 0.2, 0.2, 0.2, 0.4, 0, 0, 0, 0.2, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.2, 0, 0.2, 0.6,
0, 0.6, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 1, 0, 0, 0, 0.2,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.6, 0.4, 0.6, 0.4, 0.6,
1, 0.8, 0.2, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.6, 1.4, 2.2, 0, 0.4, 1, 0.6, 0,
1.2, 1.6, 0.6, 0, 0.2, 0, 0, 0, 0, 0, 0.2, 0, 0, 0.2, 0, 0, 0.2,
0, 0.2, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 0.2, 0, 1.2, 0.4, 1, 0.6,
0.8, 1, 0, 0, 0, 0, 0), X2005.h24 = c(0, 0, 3.4, 21.4, 1.8, 1.4,
0, 0, 0, 0, 0, 0, 0, 0, 0, 8.6, 1.4, 0, 0, 0, 0, 0.6, 7, 6.4,
21.8, 3.2, 0, 0, 0.6, 11.2, 0), X2006.h24 = c(0, 8.6, 11.2, 8.4,
0, 0, 0, 0, 0, 7.8, 7.6, 1.2, 10.2, 7, 0, 0, 1.6, 7.6, 23.6,
3.2, 0, 9, 31.4, 10, 2.2, 1.6, 0, 4.8, 10, 1.2, 4.6)), .Names = c("X2005.h1",
"X2006.h1", "X2005.h24", "X2006.h24"))
The function:
myfun<- function(x) {
y<-as.matrix(sapply(x, as.numeric))
#Aggregating hourly (h1) data to 6,12 hourly data
h<-sapply(c(1, 6, 12), function(hrs) colSums(matrix(y, ncol=length(x)/hrs)))
hours<-sapply(list(h[[1]], h[[2]], h[[3]]), function(qq) list(qq))
names(hours)<-c("h1", "h6", "h12")
#Calculating means of 1, 6, 12 hourly data
means<-sapply(list(hours[[1]], hours[[2]], hours[[3]]), function(ss) mean(ss, na.rm = T))
#means of 1, 6, 12 hourly data
mean1<-means[[1]]
mean6<-means[[2]]
mean12<-means[[3]]
mean24<- means[[4]]
meanlist<-list(mean1, mean6, mean12, mean24)
names(meanlist) <- c("mean1", "mean6", "mean12", "mean24")
return(meanlist)
}
What I want is:
As aformentioned before, 6-12-24 hourly values are obtained with aggregation of hourly data h1 as h6,h12 and h24. mean1, mean6, mean12 and mean24 are calculating in loop with lapply function for each year.
Means mean1, mean6, mean12 are okey. But to obtain mean24, I want to use h24 data (x2005.h24, x2006.h24,...) instead of aggregated h24 data. How can I achieve this?
I have to say I'm not used to analysing data represented in lists, usually data.frames or time series, this doesn't seem very efficient, but since I don't have any better suggestion, here's one way this could be done, create a new list which includes the data you want (every six and 12 hours), and then apply the mean on each element
# list with 1h data
l1 <- myd[grepl('h1', names(myd))]
# list with 6h and 12h data
l6_12 <- names(myd)[grepl('h1', names(myd))] %>%
lapply(function(x) {
setNames(lapply(c(6,12),
function(h) rowSums(matrix(myd[[x]], ncol = h))),
c(gsub('h1', 'h6', x), gsub('h1', 'h12', x)))
}) %>%
unlist(recursive = FALSE)
# list with 24h data
l24 <- myd[grepl('h24', names(myd))]
# combine all three together and then take the mean of each element (in this case 8 elements in total)
c(l1, l6_12, l24) %>% lapply(mean)
I have a data like here. The data contains rainfall values for 3 years (2005-2006-2007) separately.
Data:
mydata<-structure(list(X2005.hourly = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.6, 0.2, 0, 0, 0.6, 0.2, 0.6, 0.6, 0, 0, 0, 0.6, 1.2, 1.8, 1.8,
1.2, 1, 1.2, 1.6, 1.2, 1.4, 1, 1.2, 0.8, 0.8, 0.2, 0, 0, 1.6,
0.4, 0, 0.4, 1.2, 0.8, 0.2, 0.4, 0.2, 0.4, 0.4, 0.2, 0.2, 0.2,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.2, 0.4, 0.4,
0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0.2, 0.4, 1.2, 1.6, 2.2, 1.8, 0.4, 0.6, 0, 0.2, 0.2,
0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.2, 0.2, 0, 0.6, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 0.2,
0, 0, 0.8, 0, 0, 0.2, 0, 0, 0, 0, 0.2, 0.2, 0.2, 0, 0, 1.2, 4,
0.2, 0, 0, 0, 0, 0, 0, 0, 1.6, 0.6, 0, 0, 0, 1.4, 0.6, 0.4, 0.8,
0, 0, 0, 0, 0.2, 0, 0.2, 0, 0, 0, 0, 0.6, 0, 0, 0, 0, 0.2, 1.2,
1, 1.8, 2, 1.8, 0.4, 0.6, 1.2, 1.8, 1.4, 1.2, 0.6, 0.4, 0, 0.2,
0.2, 1.6, 1.6, 1.4, 0.8, 0.4, 0.2, 0.2, 0, 0.6, 0, 0.4, 0.4,
0.2, 0.2, 0.2, 0.4, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.6, 0, 0, 0, 0.2, 0.2, 0.4, 0.4, 1.6, 1.8,
1.2, 1.4, 1.4, 1.4, 0.6, 0.6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0), X2006.hourly = c(0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 4.8,
1, 0, 0, 0, 0, 0, 0, 0, 0.8, 0, 0, 0, 0, 0, 0, 6.6, 0.2, 0, 0,
0.2, 0, 0, 0.2, 0, 1.4, 0, 1.2, 0, 0.2, 0, 0.2, 1, 0, 0, 0, 0,
0, 0.2, 1.8, 2, 2.2, 0.6, 0.8, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0.2, 0, 0, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0.2, 1.8, 5.8, 0.2, 0.4, 0, 3.8, 0.2, 1.2, 0.4, 0,
0, 1, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0.2, 0.6, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.8, 0.8, 1, 0.2, 0, 0.2, 0, 0, 0.6, 2.4,
0.2, 0, 0.2, 0, 0, 1.6, 0.2, 2, 0, 1.2, 4.6, 0.6, 0, 0, 0, 0,
0, 0, 0, 0, 0.4, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0.8, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0.2, 0.2, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0,
0.2, 1.8, 0.6, 0, 0, 0, 0.4, 2, 1, 0.8, 0.4, 0.2, 0, 0.2, 0,
0.8, 0.8, 0.2, 0.2, 0, 0.4, 0.4, 0.6, 0.8, 4, 2, 0.8, 2.6, 1.4,
0.6, 1, 1, 1, 1.2, 1.2, 0.8, 1.6, 1.8, 0.4, 0.2, 0.2, 0.2, 0.4,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.6, 1.2, 0.6, 0.8,
0.8, 1, 0.8, 1.2, 1.2, 0.8, 1, 2.2, 3.4, 2.6, 1.8, 1.6, 1.8,
3.6, 3, 1.8, 3.2, 1.8, 1, 0.6, 0.4, 0, 1, 0.6, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 3.6, 3.8, 0.2, 0.4, 0, 0.2, 0.2, 0.4, 0.2,
0.2, 0.2, 0.4, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0.2, 0.2, 0, 0.2, 0.6, 0, 0.6, 0.4, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.4, 1, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.6, 0.4, 0.6, 0.4, 0.6, 1, 0.8, 0.2, 0, 0,
0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0.2, 0.6, 1.4, 2.2, 0, 0.4, 1, 0.6, 0, 1.2, 1.6, 0.6, 0,
0.2, 0, 0, 0, 0, 0, 0.2, 0, 0, 0.2, 0, 0, 0.2, 0, 0.2, 0, 0,
0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.4, 0.2, 0, 1.2, 0.4, 1, 0.6, 0.8, 1, 0,
0, 0, 0, 0), X2007.hourly = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0.6, 0.4, 1, 2.2, 2.6, 1.8, 0.8, 0.6, 0, 0, 0.2, 0, 0, 0,
0, 0.4, 0, 0.2, 0.2, 0, 0, 0.2, 0, 0, 0, 0, 0.6, 0.4, 0.4, 0.6,
0.4, 0.4, 0.2, 0, 0, 0.2, 0.2, 0, 0, 0, 0.2, 0, 0, 0.2, 0.4,
0.4, 0, 1.4, 0.4, 0.8, 0.4, 0.2, 0.4, 0.6, 0.8, 1.6, 0.6, 0.4,
0.6, 0.8, 1, 0.8, 1.4, 1.4, 2.2, 0.8, 0.2, 0.2, 0.2, 0, 0, 0.6,
0.6, 0.4, 0.2, 0.4, 0.6, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 0.6, 1,
1, 0.6, 1.4, 1.4, 0.6, 0.4, 0.6, 0.6, 0.8, 1, 1.8, 1.6, 0.6,
0.4, 0.2, 0.2, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 0.8, 1.6, 1.8, 1.6,
1, 1, 0.4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.2, 0, 0, 0, 0, 0, 0.2, 0.4, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.4, 0, 0.2, 0, 0, 0, 0.2, 0.2,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0.2, 0, 0.6, 0.6, 0.6, 0.8, 0.2, 0, 1, 0.2, 0.6, 1.6, 1, 0.6,
0.6, 0.4, 0.6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 3.4, 0, 0, 0, 0, 1.6, 1.4, 0.2, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 2.4, 1.2, 0, 0.2, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.6, 3, 1.4, 0, 0, 0.2, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.6, 0.2, 2.6, 3, 0.4, 0, 0, 0, 0, 0, 0, 0,
0.2, 0, 0, 0, 0.2, 0, 0, 0, 0, 0.8, 0.2, 0.2, 0, 0, 0, 0, 0,
0.2, 0, 0.2, 0, 0.4, 0.4, 0.8, 0.8, 1.2, 1, 0.6, 0.6, 0.4, 0.2,
0.2, 0.4, 0.2, 0.2, 0.4, 0.2, 0.8, 0, 0, 0.2, 0, 0, 0.2, 0, 0.2,
0, 0.2, 0, 0.2, 0.2, 0, 0, 0, 0)), .Names = c("X2005.hourly", "X2006.hourly",
"X2007.hourly"), class = "data.frame", row.names = c(NA, -744L))
And the function I used before:
myfun<- function(x) {
y<-as.matrix(sapply(x, as.numeric))
#Aggregating hourly data to 6,12 and 24 hourly data
h<-sapply(c(1, 6, 12, 24), function(hrs) colSums(matrix(y, ncol=length(x)/hrs)))
hours<-sapply(list(h[[1]], h[[2]], h[[3]], h[[4]]), function(qq) list(qq))
names(hours)<-c("hourly", "h.6", "h.12", "h.24")
#Calculating means of 1, 6, 12 and 24 hourly data
means<-sapply(list(hours[[1]], hours[[2]], hours[[3]], hours[[4]]), function(ss) mean(ss, na.rm = T))
#means of 1, 6, 12 and 24 hourly data
mean1<-means[[1]]
mean6<-means[[2]]
mean12<-means[[3]]
mean24<-means[[4]]
meanlist<-list(mean1, mean6, mean12, mean24)
names(meanlist) <- c("mean1", "mean6", "mean12", "mean24")
return(meanlist)
}
What I did:
I am applying myfun with lapply function and I am getting results separately for each year. And there is no trouble here.
results<-lapply(mydata, myfun)
What I want:
As you see, there is h.24 term inside of names(hours) in the myfun function . To calculate mean of 24 hourly data, function is using h.24 data aggregated from hourly data. But I want to use external 24 hourly daily data instead of h.24. In the data below, there are daily values separately for 2005,2006,2007
adddata<-structure(list(X2005.daily = c(0, 0, 3.4, 21.4, 1.8, 1.4, 0,
0, 0, 0, 0, 0, 0, 0, 0, 8.6, 1.4, 0, 0, 0, 0, 0.6, 7, 6.4, 21.8,
3.2, 0, 0, 0.6, 11.2, 0), X2006.daily = c(0, 8.6, 11.2, 8.4,
0, 0, 0, 0, 0, 7.8, 7.6, 1.2, 10.2, 7, 0, 0, 1.6, 7.6, 23.6,
3.2, 0, 9, 31.4, 10, 2.2, 1.6, 0, 4.8, 10, 1.2, 4.6), X2007.daily = c(0.6,
9.4, 4.2, 7.2, 15, 15, 0.4, 0, 8.6, 0, 0.2, 0.8, 0, 0.6, 0.6,
1.4, 8.2, 0.2, 0, 0, 0, 0, 5, 1.6, 4, 5.2, 0, 0, 7, 4.2, 7.2)), .Names = c("X2005.daily",
"X2006.daily", "X2007.daily"), class = "data.frame", row.names = c(NA,
-31L))
Briefly: I want to add external daily data for each year, and then I want to calculate means of 1, 6, 12 hourly values and also means of added daily data for each year. So, the results of mean1, mean6, mean12 will be the same but mean24 will be the mean of added daily values.
If I understood your question correctly, this should work:
results <- do.call(cbind,results) #convert your list to a data.frame for better overview
colnames(results) <- c("X2005","X2006","X2007")
results <- rbind(results,meanDaily=colMeans(adddata)) #rbind the colMeans of you adddata
results
X2005 X2006 X2007
mean1 0.1193548 0.2336022 0.1430108
mean6 0.716129 1.401613 0.8580645
mean12 1.432258 2.803226 1.716129
mean24 2.864516 5.606452 3.432258
meanDaily 2.864516 5.574194 3.43871
Additional answer to request in comment:
library(zoo)
myfun <- function(df_hourly,df_daily)
{
DF <- sapply(df_hourly,function(col){c(mean(rollapply(col,width=1,sum,by=1,align="left")),
mean(rollapply(col,width=6,sum,by=6,align="left")),
mean(rollapply(col,width=12,sum,by=12,align="left")),
mean(rollapply(col,width=24,sum,by=24,align="left")))})
DF <- rbind(DF,colMeans(adddata))
colnames(DF) <- c("yr2005","yr2006","yr2007")
rownames(DF) <- c("mean.1hr","mean.6hr","mean.12hr","mean.24hr","mean.daily")
return(DF)
}
myfun(mydata,adddata)
yr2005 yr2006 yr2007
mean.1hr 0.1193548 0.2336022 0.1430108
mean.6hr 0.7161290 1.4016129 0.8580645
mean.12hr 1.4322581 2.8032258 1.7161290
mean.24hr 2.8645161 5.6064516 3.4322581
mean.daily 2.8645161 5.5741935 3.4387097