How to optimize a repetitive string using dictionary comprehension? - python-3.6

Trying to optimize my very repetitive code for my own edification.
I have attempted to implement a for loop in the dictionary function, but I did not succeed. Is there a quicker and more efficient to write this:
blah = dict({'S1_Cycle1': ['S1/S1_Cycle1', 'S1'],
'S1_Cycle2': ['S1/S1_Cycle2', 'S1'],
'S1_Cycle3': ['S1/S1_Cycle3', 'S1'],
'S1_Cycle4': ['S1/S1_Cycle4', 'S1'],
'S1_Cycle5': ['S1/S1_Cycle5', 'S1'],
'S1_Cycle6': ['S1/S1_Cycle6', 'S1'],
'S1_Cycle7': ['S1/S1_Cycle7', 'S1'],
'S1_Cycle8': ['S1/S1_Cycle8', 'S1'],
'S1_Cycle9': ['S1/S1_Cycle9', 'S1'],
'S1_Cycle10': ['S1/S1_Cycle10', 'S1'],
'S1_Cycle11': ['S1/S1_Cycle11', 'S1'],
'S1_Cycle12': ['S1/S1_Cycle12', 'S1'],
'S1_Cycle13': ['S1/S1_Cycle13', 'S1'],
'S1_Cycle14': ['S1/S1_Cycle14', 'S1'],
'S1_Cycle15': ['S1/S1_Cycle15', 'S1'],
'S1_Cycle16': ['S1/S1_Cycle16', 'S1'],
'S1_Cycle17': ['S1/S1_Cycle17', 'S1'],
'S1_Cycle18': ['S1/S1_Cycle18', 'S1'],
'S1_Cycle19': ['S1/S1_Cycle19', 'S1'],
'S1_Cycle20': ['S1/S1_Cycle20', 'S1'],
'S1_Cycle21': ['S1/S1_Cycle21', 'S1'],
'S1_Cycle22': ['S1/S1_Cycle23', 'S1'],
'S1_Cycle23': ['S1/S1_Cycle23', 'S1'],
'S2_Cycle1': ['S1/S1_Cycle1', 'S2'],
'S2_Cycle2': ['S1/S1_Cycle2', 'S2'],
'S2_Cycle3': ['S1/S1_Cycle3', 'S2'],
'S2_Cycle4': ['S1/S1_Cycle4', 'S2'],
'S2_Cycle5': ['S1/S1_Cycle5', 'S2'],
'S2_Cycle6': ['S1/S1_Cycle6', 'S2'],
'S2_Cycle7': ['S1/S1_Cycle7', 'S2'],
'S2_Cycle8': ['S1/S1_Cycle8', 'S2'],
'S2_Cycle9': ['S1/S1_Cycle9', 'S2'],
'S2_Cycle10': ['S1/S1_Cycle10', 'S2'],
'S2_Cycle11': ['S1/S1_Cycle11', 'S2'],
'S2_Cycle12': ['S1/S1_Cycle12', 'S2'],
'S2_Cycle13': ['S1/S1_Cycle13', 'S2'],
'S2_Cycle14': ['S1/S1_Cycle14', 'S2'],
'S2_Cycle15': ['S1/S1_Cycle15', 'S2'],
'S2_Cycle16': ['S1/S1_Cycle16', 'S2'],
'S2_Cycle17': ['S1/S1_Cycle17', 'S2'],
'S2_Cycle18': ['S1/S1_Cycle18', 'S2'],
'S2_Cycle19': ['S1/S1_Cycle19', 'S2'],
'S2_Cycle20': ['S1/S1_Cycle20', 'S2'],
'S2_Cycle21': ['S1/S1_Cycle21', 'S2'],
'S2_Cycle22': ['S1/S1_Cycle23', 'S2'],
'S2_Cycle23': ['S1/S1_Cycle23', 'S2']
})

Use a dictionary-comprehension:
{f'S{x}_Cycle{y}': [f'S1/S1_Cycle{y}', f'S{x}'] for x in range(1, 3) for y in range(1, 24)}
which outputs:
{'S1_Cycle1': ['S1/S1_Cycle1', 'S1'],
'S1_Cycle2': ['S1/S1_Cycle2', 'S1'],
...
...
'S2_Cycle22': ['S1/S1_Cycle23', 'S2'],
'S2_Cycle23': ['S1/S1_Cycle23', 'S2']}

Related

From arrival time to Avg. number of tasks per hour vs. hour of day

I am trying to analyze real-world Google traces for my research. I have the arrival time of tasks in milliseconds and I would like to represent in y-axis -> avg. number of tasks per hour vs. x-axis -> hour of day. I am struggling so far. I put only 2000 values of arrival time for simplicity. Any help would be really appreciate.
Python code to translate to R:
one_hour_in_milliseconds = 1000*60*60
hours_in_a_day = 24
def get_tasks_per_hour_of_day(tasks_df):
tasks_per_hour_df = tasks_df.groupby(F.floor(F.col("ts_submit") / F.lit(one_hour_in_milliseconds)).alias("hour")).count()
tasks_per_hour_of_week = tasks_per_hour_df.groupby((F.col("hour") % hours_in_a_day).alias("hour_of_day")).agg(F.avg("count"))
return tasks_per_hour_of_week
"ts_submit" correspond to the arrival time and it is like this:
arrival_time <- c(757956, 757956, 757956, 760949, 760949, 760949, 760949, 760949,
760949, 760949, 760950, 760950, 760950, 760950, 760950, 760950,
772013, 870116, 870116, 870116, 870117, 870117, 870117, 870118,
870118, 919618, 1107986, 1372983, 1372983, 1372983, 1372984,
1372984, 1398938, 1398938, 1398938, 1398939, 1423337, 1423337,
1423337, 1423337, 1423337, 1423338, 1423338, 1423338, 1423339,
1423339, 1437049, 1437049, 1437049, 1437050, 1437050, 1437050,
1437050, 1437050, 1437051, 1437051, 1437051, 1437051, 1599232,
1599232, 1599232, 2073894, 2100114, 2100114, 2100114, 2100114,
2100114, 2100115, 2112582, 2122978, 2135463, 2135463, 2135464,
2135464, 2135464, 2135464, 2135464, 2135466, 2135466, 2140221,
2149856, 2149857, 2149857, 2175430, 2175431, 2175431, 2175431,
2175431, 2175432, 2175432, 2175432, 2199343, 2199343, 2199343,
2199344, 2199344, 2199345, 2199345, 2199345, 2199346, 2488835,
2639241, 2639241, 2639241, 2639241, 2639241, 2639242, 2639242,
2639242, 2639242, 2788104, 2788104, 2788104, 2788105, 2788105,
2851444, 2851444, 2851444, 2851444, 2851445, 2851445, 2851445,
2851445, 2851445, 3094321, 3094321, 3094321, 3238164, 3238164,
3238164, 3238165, 3238165, 3238165, 3238165, 3410843, 3410843,
3413803, 3525977, 3525978, 3525978, 3525978, 3525978, 3525978,
3549873, 3549873, 3549873, 3549873, 3549873, 3549873, 3549873,
3549873, 3549873, 3851600, 3851601, 3938009, 3938009, 3938009,
3986760, 4037632, 4186660, 4186660, 4186660, 4186660, 4205532,
4205532, 4223089, 4223090, 4223090, 4223090, 4277981, 4277981,
4314340, 4314340, 4365067, 4365067, 4365068, 4365068, 4365068,
4365068, 4365068, 4365068, 4365068, 4365068, 4365068, 4607608,
4839388, 4839388, 4852011, 4852011, 4852011, 4852011, 4852011,
4852011, 4948953, 4948954, 4948954, 4950308, 4950308, 4950309,
4950309, 4950309, 5269837, 5286999, 5334312, 5475300, 5548613,
5797195, 5797195, 5797196, 5797196, 6003342, 6144761, 6144761,
6209245, 6232036, 6385718, 6497910, 6675175, 6675175, 6726517,
6751350, 6751350, 6751350, 6751351, 7146772, 7561379, 7625130,
7879679, 8109251, 8183407, 8183407, 8183407, 8183408, 8183408,
8291747, 8341073, 8341910, 8341911, 8341911, 8341912, 8341912,
8341912, 8341912, 8341912, 8341913, 8341913, 8341913, 8341914,
8341914, 8559979, 8559979, 8559979, 8597532, 8597532, 8736044,
8736044, 8847283, 8847283, 8847283, 8847283, 8886313, 8886313,
8923410, 9386029, 9386030, 9386030, 9683967, 10109483, 10125754,
10154979, 10190049, 10214849, 10288755, 10298767, 10698932, 10698933,
10698933, 10911758, 10937271, 10937271, 11408067, 11408067, 12213612,
12311840, 12311840, 12311840, 12311840, 12472906, 12509542, 12509542,
12911217, 12911226, 13163002, 13163002, 13163002, 13185420, 13185420,
13185420, 13185421, 13185421, 13185421, 13185421, 13252574, 13374280,
13398830, 13398830, 13400736, 13475167, 13475168, 13475169, 13701338,
13701339, 13701339, 13701339, 13701339, 13701339, 13701340, 13701340,
13701340, 13701340, 13701340, 13701340, 13826250, 13826250, 13826250,
13954187, 13954187, 13954187, 13954188, 13962612, 14072634, 14294046,
14294047, 14294047, 14294047, 14294047, 14294048, 14294048, 14294048,
14294049, 14294049, 14294049, 14364720, 14364720, 14364720, 14364720,
14663840, 14874149, 14874149, 14874149, 14874149, 14937730, 15314089,
15326106, 15326108, 15326108, 15326108, 15326108, 15326108, 15326109,
15326109, 15326109, 15326110, 15326110, 15326110, 15326111, 15326111,
15338576, 15486616, 15486616, 15647460, 15647460, 15647460, 15647460,
15806441, 15806441, 15844483, 15844483, 15844483, 15844483, 15844483,
15844483, 15844483, 15844483, 15918805, 15918805, 15986737, 15986738,
15986738, 15986738, 15986739, 15986740, 16079743, 16216549, 16216549,
16216549, 16216549, 16254921, 16254921, 16254921, 16254922, 16254922,
16254922, 16254922, 16254922, 16254923, 16254923, 16469030, 16530256,
16530257, 16617988, 16801458, 16907652, 17301184, 17301185, 17301186,
17301186, 17458282, 17458283, 17458283, 17458284, 17494866, 17507368,
17507368, 17583714, 17583714, 17921147, 18069403, 18069404, 18069404,
18094921, 18094921, 18244693, 18244693, 18244693, 18244693, 18244693,
18391425, 18472661, 18472661, 18708037, 18708037, 18797563, 18821442,
18844215, 18981864, 18981864, 18981864, 19007073, 19009142, 19084466,
19084466, 19084466, 19109977, 19198215, 19198215, 19198216, 19348976,
19410777, 19410777, 19410777, 19549339, 19582661, 19660325, 19709086,
19758327, 19821920, 19909761, 19909761, 20080135, 20080135, 20080135,
20080135, 20143181, 20296794, 20485668, 20485668, 20485668, 20485668,
20485668, 20485669, 20485669, 20485669, 20522133, 20524785, 20524785,
20538764, 20538764, 20787352, 20834431, 21135456, 21135456, 21268106,
21268106, 21307519, 21695402, 21695402, 21695402, 21695402, 21695402,
21695402, 21695403, 21695403, 21695403, 21700449, 21708509, 21708509,
21846639, 21869462, 21869463, 21869463, 21911309, 21911309, 22113517,
22113518, 22113518, 22132507, 22175044, 22175044, 22175044, 22175044,
22175044, 22175045, 22175045, 22224907, 22274211, 22291331, 22349386,
22349386, 22349388, 22349388, 22452173, 22464686, 22464686, 22464687,
22502278, 22571928, 22571928, 22588242, 22714864, 22866236, 23227341,
23227341, 23250030, 23251797, 23265533, 23298161, 23353828, 23353828,
23353829, 23353829, 23353829, 23363507, 23363507, 23456219, 23467556,
23528672, 23528672, 23528672, 23528673, 23595173, 23605403, 23605403,
23664892, 23689971, 23804575, 24039832, 24095776, 24095776, 24095776,
24095776, 24095776, 24095776, 24095776, 24095776, 24095776, 24095777,
24104204, 24115500, 24119849, 24120916, 24352267, 24352267, 24352267,
24442081, 24670784, 24705243, 24705243, 24724065, 24864393, 24949334,
25007713, 25007713, 25010975, 25110349, 25187708, 25187708, 25498002,
25667013, 25667013, 25667013, 25667014, 25699719, 25707399, 25740850,
26043560, 26064571, 26064571, 26136232, 26136233, 26136233, 26136234,
26136234, 26150821, 26230497, 26259966, 26291610, 26291610, 26644150,
26644150, 26644150, 26644150, 26785031, 26785032, 26785042, 26785042,
26785042, 26785042, 26785042, 26785043, 26785043, 26842512, 26842512,
26920823, 27083447, 27083448, 27083448, 27083449, 27083449, 27083449,
27083450, 27083450, 27083450, 27083450, 27083450, 27083534, 27083534,
27083534, 27083534, 27083535, 27083536, 27083536, 27083537, 27179751,
27204895, 27514060, 27681148, 27755972, 27755980, 27755986, 27758456,
28093450, 28093450, 28093450, 28142347, 28210772, 28363317, 28363317,
28497902, 28497902, 28497902, 28497902, 28497903, 28497903, 28497903,
28497904, 28497904, 28497904, 28497904, 28497904, 28497904, 28497904,
28497904, 28497904, 28497905, 28497905, 28497905, 28497905, 28497905,
28546659, 28546660, 28546660, 28546660, 28546660, 28546661, 28546661,
28551688, 28551688, 28551688, 28551689, 28551689, 29248859, 29248859,
29248859, 29309935, 29535662, 29535662, 29541149, 29707778, 29848098,
29848098, 29983569, 29983569, 29983569, 29983569, 29983570, 29983570,
30194883, 30442041, 30442041, 30442041, 30525968, 30525969, 30604622,
30823608, 30885528, 31159416, 31345499, 32354843, 32354844, 32354844,
32354844, 32354845, 32354845, 32354845, 32354846, 32354846, 32354846,
32354846, 32354846, 32443597, 32443597, 32443597, 32443597, 32443597,
32443598, 32443598, 32443598, 32443598, 32443599, 32443599, 32443599,
32443600, 32443600, 32443600, 32443601, 32443601, 32443603, 32443603,
32443604, 32443604, 32443605, 32443605, 32443606, 32443607, 32443608,
32443608, 32443609, 32443609, 32443609, 32443610, 32443610, 32443611,
32443611, 32443612, 32443613, 32443613, 32443613, 32443613, 32443614,
32443615, 32443616, 32443617, 32443617, 32443617, 32491340, 32493301,
32951556, 33037800, 33098521, 33235764, 34113105, 34336104, 34409634,
34418071, 34418071, 34418071, 34418071, 34418072, 34418072, 34418072,
34418073, 34418073, 34418074, 34418074, 34418074, 34418074, 34637093,
34637093, 34637093, 34637094, 34637094, 34761083, 34905880, 34979175,
34979175, 35016369, 35016370, 35016370, 35016371, 35016371, 35016371,
35019674, 35019674, 35166054, 35850468, 35859368, 36014996, 36317663,
36317663, 36317663, 36317664, 36317665, 36317666, 36317666, 36863673,
36863674, 36929379, 37088849, 37279075, 37309596, 37309596, 37309597,
37504453, 37528551, 37530874, 37530874, 37530875, 37530875, 37530876,
37530876, 37614185, 37667591, 37702858, 37704580, 37704580, 37714912,
37855804, 38189537, 38213184, 38308386, 38308386, 38308386, 38308387,
38308387, 38308387, 38308387, 38308387, 38308387, 38308387, 38308387,
38308387, 38308388, 38308388, 38308388, 38308388, 38308389, 38308389,
38308389, 38308389, 38308389, 38308390, 38308390, 38308390, 38308391,
38308391, 38308391, 38308392, 38308392, 38308392, 38308392, 38308392,
38308393, 38308393, 38308393, 38308394, 38308394, 38308395, 38308395,
38308395, 38308395, 38308396, 38308396, 38308396, 38308396, 38308396,
38430415, 39140178, 39140178, 39140179, 39140179, 39140179, 39415380,
39575481, 39575481, 39575481, 39575481, 39575482, 39575482, 39575483,
39575483, 39575483, 39575483, 39583572, 39670699, 39670699, 39913944,
39919941, 39941344, 39941344, 39941344, 39941344, 39941345, 39941345,
39941345, 39941345, 39941345, 40200753, 40699497, 40699497, 41230877,
41230877, 41230878, 41230878, 41312702, 41423823, 42135444, 42135444,
42135460, 42148090, 42148150, 42148162, 42148162, 42212294, 42390686,
42390686, 42390687, 42390687, 42390687, 42390688, 42390689, 42390690,
42635231, 42635231, 42638029, 42638029, 42638029, 42980989, 43344928,
43485736, 44401393, 44542352, 44542352, 44736757, 44736757, 44736757,
44736757, 44772454, 44774029, 44774030, 44808621, 44915963, 45212960,
45637062, 45637062, 45637062, 45637062, 45637062, 45776584, 46184655,
46357726, 46357726, 46357726, 46357727, 46657424, 46657425, 46657425,
46657425, 46657425, 46657425, 46657425, 46657425, 46657426, 46657426,
46657426, 46657426, 46657426, 46657426, 47360778, 47433852, 47433852,
47433852, 47433984, 47433985, 47604913, 47662504, 47662504, 47662505,
47662505, 47662505, 47662505, 47662505, 47662506, 47662506, 47814440,
47814440, 47814440, 47849206, 47849207, 47849207, 47849207, 47849208,
47861403, 47861403, 47861403, 47861403, 47861403, 47861404, 47861404,
47861404, 47861404, 47861404, 47861405, 47861405, 47861405, 47861405,
47861406, 47861406, 47861406, 47861407, 47861407, 47861407, 47861407,
47861408, 48463583, 48469546, 48524758, 48661693, 48673536, 48673536,
48673536, 48673537, 48689005, 48699797, 48699798, 48699798, 48699799,
48699799, 48699800, 48699800, 48699801, 48699801, 48699802, 48699802,
48723653, 48723653, 48723653, 49037695, 49037699, 49163896, 49189852,
49189852, 49286549, 49286549, 49297165, 49325507, 49334090, 49337118,
49749268, 49749269, 49758997, 49800456, 49800456, 49800456, 49800457,
49800457, 49800457, 49800457, 49800457, 49800457, 49800457, 49800458,
49800458, 49800458, 49800458, 50114159, 50265451, 50469956, 50482424,
50482424, 50482424, 50482424, 50482424, 50482424, 50482425, 50482425,
50482425, 50482425, 50482426, 50482426, 50482426, 50482426, 50482427,
50482427, 50482427, 50482427, 50482427, 50482427, 50482428, 50482428,
50482428, 50482428, 50482429, 50482429, 50482429, 50482429, 50482429,
50482431, 50482431, 50482431, 50482431, 50482431, 50482431, 50482432,
50482432, 50482433, 50482433, 50482433, 50482433, 50482433, 50482433,
50482434, 50636008, 50636008, 50639808, 50639809, 50639809, 50639809,
50639810, 50680249, 51425487, 51568769, 51582475, 51855743, 52094367,
52406956, 52406956, 52481978, 52550437, 52560092, 52560092, 52754531,
53021992, 53021992, 53021992, 53021993, 53021993, 53021994, 53021994,
53021994, 53021995, 53021995, 53037426, 53037427, 53037427, 53060227,
53281608, 53281608, 53281608, 53307653, 53355669, 53355669, 53647271,
53647272, 53709981, 53709982, 53720672, 53720690, 53795612, 53795612,
53795613, 53795613, 53795613, 53795615, 53795615, 53795616, 53795616,
53795616, 53795618, 53795618, 53795619, 53795619, 53795619, 53925439,
54010507, 54537694, 54637861, 54738366, 54845159, 54873097, 54892639,
54950924, 54979981, 54979981, 54979981, 55107460, 55390353, 55390353,
55390353, 55390353, 55390353, 55390354, 55390354, 55390354, 55390354,
55390354, 55545710, 55545710, 55560762, 55560763, 55571473, 55571473,
55571474, 55574240, 55574241, 55574241, 55574241, 55574241, 55574241,
55574241, 55574241, 55638292, 55668469, 55832897, 55860912, 55860913,
55860913, 55860914, 55860914, 55887756, 56198611, 56198612, 56198612,
56198612, 56198612, 56198613, 56198613, 56198613, 56494310, 56494310,
56517633, 56517633, 56517633, 56517642, 56519945, 56530111, 56530112,
56793944, 56793944, 56793944, 56793944, 56893498, 56897176, 57142484,
57253989, 57329324, 57369776, 57369776, 57369776, 57369776, 57889818,
57889818, 57889818, 57889819, 57889819, 57889819, 57889819, 57891918,
57891919, 57891919, 57891919, 57961257, 58176575, 58176575, 58176576,
58176576, 58176576, 58176577, 58176577, 58377258, 58377258, 58377259,
58377259, 58377260, 58443163, 58627503, 58741438, 58931876, 59079448,
59079448, 59079448, 59079448, 59079449, 59079449, 59079450, 59079450,
59079451, 59084904, 59084905, 59112957, 59129501, 59129501, 59129501,
59129501, 59129501, 59129502, 59129502, 59176863, 59225796, 59225796,
59268481, 59268481, 59371229, 59371230, 59439688, 59447236, 59447236,
59447236, 59447237, 59447237, 59447238, 59683626, 59940114, 60325859,
60411051, 60634569, 60634569, 60729839, 60824385, 60829336, 60988482,
61011801, 61011801, 61090753, 61205430, 61245545, 61336568, 61657853,
61657853, 61657854, 61657854, 61657855, 61657855, 61657855, 61682564,
61685227, 61912488, 61912488, 61912488, 62012104, 62012104, 62012104,
62012105, 62058074, 62058919, 62513624, 62528305, 62575355, 62671367,
62694310, 62806735, 62806735, 62871168, 62885177, 62885179, 62886202,
62886202, 62886202, 62886203, 62886204, 62886204, 62886205, 62886205,
62933437, 63125213, 63171350, 63171350, 63214146, 63214146, 63262433,
63360290, 63444920, 63447598, 63546986, 63751354, 63787894, 63787895,
63833676, 63981489, 63981489, 63997975, 64384806, 64522143, 64522143,
64522143, 64522143, 64522144, 64536594, 64536594, 64660657, 64796620,
64796620, 64796620, 64999801, 65233922, 65267983, 65267983, 65405554,
65539371, 65625725, 65704385, 65760736, 66161330, 66316074, 66364353,
66364353, 66364353, 66466991, 66488145, 66488145, 66506952, 66506953,
66579607, 66740301, 66740302, 66740302, 66740302, 66740302, 66740302,
66740303, 66740303, 66820009, 67091236, 67091236, 67091236, 67113527,
67151838, 67248954, 67248955, 67248955, 67248955, 67248955, 67408976,
67573552, 67573552, 67666935, 67881803, 67916236, 67971425, 68246431,
68471507, 68495414, 68780188, 68837867, 69068470, 69068470, 69068470,
69068470, 69068471, 69068472, 69068472, 69068472, 69068473, 69068473,
69273410, 69273410, 69478571, 69545443, 69545444, 69545444, 69907898,
70189600, 70189601, 70718202, 70724278, 70724278, 70796341, 70796341,
70826560, 70857008, 70862056, 70863680, 70863681, 70863681, 70943370,
71004502, 71004503, 71053539, 71059974, 71059974, 71075017, 71079823,
71097524, 71100997, 71100997, 71100997, 71134237, 71134237, 71134237,
71134238, 71134238, 71134238, 71134238, 71134238, 71134238, 71134238,
71134238, 71134239, 71134239, 71134239, 71134239, 71134239, 71134239,
71290708, 71290708, 71290708, 71329874, 71329874, 71370725, 71370725,
71370725, 71370725, 71370726, 71370726, 71469848, 71469848, 71539790,
71539790, 71563614, 71622432, 71806530, 71806530, 71806531, 72135750,
72178223, 72178223, 72274414, 72351729, 72675060, 72675061, 72675061,
72675061, 72675061, 72675062, 72675062, 72675062, 72675062, 72675063,
72675063, 72675063, 72675063, 72675063, 72675063, 72777183, 72777183,
72777183, 72777184, 72777184, 72777184, 72777185, 72777186, 72777186,
72780189, 72780189, 72780189, 72780189, 72780190, 72780190, 72780190,
72780190, 72788659, 72796723, 72978870, 73207448, 73401342, 73522703,
73522703, 73566900, 73566901, 73566901, 73566901, 73572839, 73585497,
73585497, 73585498, 73585619, 73585619, 73683372, 73735106, 73755927,
73755927, 73755928, 73762112, 73833778, 73833778, 74026418, 74029109,
74060244, 74097558, 74143707, 74143707, 74143707, 74143707, 74143708,
74143708, 74143708, 74143720, 74143721, 74143721, 74143721, 74143722,
74143722, 74143723, 74143723, 74143723, 74143724, 74163648, 74163648,
74163648, 74163649, 74163650, 74163650, 74163650, 74163650, 74163650,
74163651, 74163651, 74225586, 74229234, 74229234, 74244927, 74244927,
74267674, 74267674, 74468446, 74529190, 74546378, 74550484, 74608548,
74680812, 74680812, 74680812, 74680813, 74680813, 74680813, 74680814,
74722815, 74872271, 74895622, 74895623, 74952896, 75123194, 75123194,
75287349, 75396027, 75396027, 75396028, 75396028, 75694684, 75697179,
75788767, 75806456, 76004572, 76175984, 76178539, 76178540, 76665433,
76789974, 76789974, 76789975, 76789975, 76984916, 77088158, 77088158,
77211792, 77220562, 77269965, 77398749, 77398749, 77460675, 77553760,
77737735, 77773127, 77848104, 78098123, 78186359, 78278050, 78278050,
78278050, 78292132, 78315016, 78361410, 78361410, 78361410, 78361448,
78361448, 78361448, 78385967, 78385967, 78385967, 78385968, 78410826,
78510022, 78531930, 78535277, 78535277, 78572226, 78611469, 78639538,
78657266, 78688108, 78787315, 78787315, 78857986, 78857986, 78894564,
78894564, 79051278, 79070637, 79174030, 79246177, 79350675, 79406571,
79639643, 79639643, 79639643, 79707252, 79746611, 79985433, 79985436,
79985436, 80008991, 80041214, 80085547, 80144783, 80311060, 80491693,
80491693, 80491693, 80491693, 80491694, 80505779, 80559619, 80644479,
80880743, 80880744, 80920419, 80920419, 80920420, 81018404, 81022920,
81110021, 81152159, 81152160, 81154987, 81154988, 81154988, 81196294,
81201358, 81201358, 81201358, 81204946, 81305550, 81347707, 81347708,
81347708, 81434323, 81898669, 81898669, 81898669, 81898669, 81898669,
81979010, 82174601, 82174601, 82174601, 82174601, 82174602, 82174602,
82174602, 82174602, 82174602, 82174602, 82174602, 82174602, 82249955,
82296355, 82320090, 82520048, 82747607, 82747607, 82819990, 82819990,
82851264, 82873136, 82873136, 82873136, 82873136, 82873137, 82873138,
82993012, 83207843, 83285153, 83288998, 83288998, 83288998, 83288998,
83294190, 83294190, 83294191, 83371710, 83822767, 83915529, 84075828,
84075829, 84075829, 84075829, 84075830, 84228502, 84228503, 84331280,
84429542, 84429542, 84475188, 84538851, 84590030, 84590030, 84590031,
84662177, 84697070, 84943765, 85039997, 85047461, 85047461, 85047462,
85076835, 85125425, 85125426, 85125426, 85125426, 85166777, 85166777,
85186810, 85221710, 85314545, 85317218, 85352519, 85352519, 85352521,
85352521, 85352522, 85352522, 85392206, 85392206, 85392206, 85392207,
85392207, 85392207, 85392207, 85392207, 85392208, 85392208, 85392208,
85392208, 85392209, 85392209, 85392209, 85392210, 85398185, 85398185,
85398186, 85398186, 85398187, 85398187, 85398187, 85426511, 85426511,
85426511, 85426513, 85426513, 85426514, 85426514, 85426514, 85426514,
85466060, 85466060, 85466060, 85466060, 85466060, 85466061, 85466061,
85612637, 85630078, 85630078, 85630079, 85758504, 85758504, 85758504,
85758504, 85758504, 85758505, 85865854, 85865854, 85865854, 85869891,
85869891, 85869891, 85869891, 85869891, 85869892, 85869892, 85869892,
85869892, 85869892, 85869893, 85969662, 86022326, 86022327, 86022327,
86022327, 86022327, 86022327, 86022327, 86022327, 86022327, 86045129,
86062688, 86062688, 86062688, 86062689, 86062689, 86062689, 86062689,
86062689, 86062689, 86062689, 86062689, 86207257, 86365835, 86365835,
86365835, 86365835, 86365836, 86365837, 86365837, 86365837, 86365837,
86365838, 86365838, 86707487, 86867203, 86867204, 86867204, 86867204,
86867204, 86886456, 87008520, 87008520, 87082425, 87082425, 87082425,
87082425, 87082426, 87082426, 87082426, 87082426, 87082427, 87082427,
87342981, 87342981)
The desired output should be like this for Google:
Uploading two approaches here:
First, converting the list to data frame:
arrival_time <- data.frame(arrival_time)
If you want to get the exact hour of the day, in that case, the number of observations per distinct hour will be very less and the plot will be a bit congested:
library(ggplot2)
arrival_time %>%
mutate(hour = (arrival_time/(1000*60*60))) %>%
group_by(hour) %>%
summarise(count=n()) %>%
ggplot()+
geom_point(aes(x=hour, y=count))+
geom_line(aes(x=hour, y=count, group=1))+
xlab('Hour of day')+
ylab('Avg. num. tasks per hour')+
theme_bw()
Now if you consider the arrival as I mentioned in the comment, that is, if the arrival is between 0 to 15 mins, it will be considered as hour 0. If it is between 1 hr 1 min and 2 hour, it will be considered as hour 1 and like this:
arrival_time %>%
mutate(hour = floor(arrival_time/(1000*60*60))) %>%
group_by(hour) %>%
summarise(count=n()) %>%
ggplot()+
geom_point(aes(x=hour, y=count))+
geom_line(aes(x=hour, y=count))+
xlab('Hour of day')+
ylab('Avg. num. tasks per hour')+
theme_bw()
This is something similar to what you want from the plot you've shared in the question. Though the logic from the comment doesn't match.
Adding another plot as asked in the comment.
arrival_time %>%
mutate(hour = floor(arrival_time/(1000*60*60)),
minute= floor(arrival_time/(1000*60))) %>%
group_by(hour, minute) %>%
summarise(count=n()) %>%
ungroup() %>%
ggplot()+
geom_point(aes(x=minute, y=count))+
geom_line(aes(x=minute, y=count))+
scale_x_continuous(breaks = seq(0, 1440, 60), labels = seq(0, 24, 1))+
xlab('Hour of day')+
ylab('Avg. num. tasks per hour')+
theme_bw()

merging shapefile county geodata with R dataframe with missing county data

I have a shapefile (US) and a list of counties (biz) with data that I'd like to convert to a choropleth map. There is data missing, so not every county FIPS is present in the biz dataframe. I've filtered both the shapefile and the biz R dataframe so that each contain the same countyfips, but I am still getting the following error when I try to merge the two to create a spatial dataframe.
ERROR while rich displaying an object: Error in sp::SpatialPolygonsDataFrame(polys, data = input#data): row.names of data and Polygons IDs do not match
I've tried:
library(rgdal)
merge(US, biz, by = "FIPS", match.ID = FALSE, duplicateGeoms = T)
and
sp::merge(US, biz, by = "FIPS")
What am I missing here?
Below is how I loaded the shapefile:
# download shape (a little less detail than in the other scripts)
f <- tempfile()
download.file("http://www2.census.gov/geo/tiger/GENZ2010/gz_2010_us_050_00_20m.zip", destfile = f)
unzip(f, exdir = ".")
US <- shapefile("gz_2010_us_050_00_20m.shp")
# leave out AK, HI, and PR (state FIPS: 02, 15, and 72)
US <- US[!(US$STATE %in% c("02","15","72")),]
# create key column
US$FIPS <- paste0(US$STATE, US$COUNTY)
And the following is the first 100 rows of my biz data:
structure(list(FIPS = c(10001, 10003, 10005, 11001, 12001, 12003,
12005, 12007, 12009, 12011, 12013, 12015, 12017, 12019, 12021,
12023, 12027, 12031, 12033, 12035, 12037, 12039, 12041, 12045,
12049, 12051, 12053, 12055, 12057, 12061, 12063, 12065, 12069,
12071, 12073, 12075, 12081, 12083, 12085, 12086, 12087, 12089,
12091, 12093, 12095, 12097, 12099, 12101, 12103, 12105, 12107,
12109, 12111, 12113, 12115, 12117, 12119, 12121, 12123, 12127,
12129, 12131, 12133, 13001, 13009, 13013, 13015, 13021, 13029,
13031, 13033, 13035, 13039, 13045, 13047, 13051, 13055, 13057,
13059, 13063, 13067, 13069, 13071, 13073, 13075, 13077, 13081,
13083, 13085, 13087, 13089, 13095, 13097, 13103, 13105, 13107,
13109, 13111, 13113, 13115), merch_perc_change = c(-0.999999997419426,
-0.999999960099927, -0.999999463606016, -0.999999999992747, -0.999999589415816,
-0.999776694518714, -0.999998139171249, -0.923691427414423, -0.999962755064437,
-0.999999297609172, 6.86356300955509, -0.999796407354303, -0.999999920342331,
-0.999989017442108, -0.999999987828828, -0.999687144654275, -0.999809854633837,
-0.999994022173781, -0.999997564726118, -0.999991074764553, -0.999978132819644,
-0.999048191581644, -0.999816255794303, -0.99999999999953, -0.999998368294174,
-0.999983530324202, -0.999999959737609, -0.999999999915238, -0.9999998961883,
-0.999997217365727, -0.999999486649557, -0.998247739967963, -0.999998585480854,
-0.99999734246251, -0.99998316042666, -0.999997698874625, -0.999999749503321,
-0.999999812749288, -0.999935448837586, -0.999999220586342, -0.99999956646977,
-0.999999999979337, -0.999999311106202, -0.99798206258435, -0.999999372093591,
-0.999999923772396, -0.999995867853803, -0.999997734665119, -0.999999905345573,
-0.999998463559358, -0.99999999205029, -0.999999991820707, -0.99990638439578,
-0.999999312175056, -0.999999978238681, -0.999999607559491, -0.999997940723336,
-0.999965827464078, -0.99987412039642, -0.999998284179988, -0.999788909786871,
-0.999999999391236, -0.999558269566805, -0.999999999991671, -0.999999896605606,
-0.999999811038711, -0.99957117015121, -0.999997531666568, -0.999891490436958,
-0.99996557149293, -0.999997779404573, -0.999999995034784, -0.999999862415467,
-0.999984320487476, -0.999998471029504, -0.999999475141075, -0.999989299298126,
-0.99999700303135, -0.999999983041563, -0.999976248756212, -0.999997959804029,
-0.999906120208879, -0.974494760294364, -0.999996351284903, -0.999999987539243,
-0.999314090769117, -0.999999999937565, -0.999999999996, -0.999227737065879,
-0.999999894274373, -0.999999275234666, -0.99999972998535, -0.999989925844005,
-0.999980780315507, 11.9726427425268, -0.999999845161106, -0.999985089588605,
-0.99999999506689, -0.999997272506931, -0.999999600943781), rev_perc_change = c(-0.999999868770726,
-0.999999990632098, -0.999978350417758, -1, -0.999999977063738,
-0.999999557925536, 2.96734401473508, -0.999999978115999, -0.999999876658537,
-0.999998644384056, -0.999999948925146, -0.999957383116217, -0.999999999202153,
-0.799103739676736, -0.999999999730313, 52.5419769811163, -0.989897752573793,
-0.999990771638404, -0.999999097277337, -0.999999056612073, -0.999999999997651,
-0.791258306382895, -0.835408375403338, -0.999999949255092, -0.999999995917852,
-0.995526573454658, -0.999980744495764, -0.999999999999523, -0.999999967084492,
-0.999999795346701, -0.865997270001737, -0.998547105111494, -0.999999994311607,
-0.999999986040664, -0.999991969242583, -0.999644160435803, -0.999999761949245,
-0.999999615053214, -0.439117774567191, -0.999999973586027, -0.999999999999614,
-0.999997934384089, -0.999999490767057, -0.904747175661281, -0.999999996424736,
-0.999999995884735, -0.999999970209871, -0.999993644951214, -0.999999928595513,
-0.999997760171338, -0.999999999201456, -0.999999998546284, -0.999996541137073,
-0.999999998767801, -0.999999999803137, -0.999999882081287, -0.999998232474645,
-0.999999998321198, -0.999999999134723, -0.999999877739019, -0.999938452181625,
-0.999999999696075, -0.999971062558217, -1, -0.999999999998913,
-0.999999936705897, -0.999999972777953, -0.999993970088101, -0.767994542540064,
-0.999997956578155, -0.999999999999991, -0.999999999504284, -0.999999988489851,
-0.999998275694059, -0.999978229417813, -0.999999893084442, -0.99999998787768,
-0.999974986845167, -0.999999999116068, -0.999998633242769, -0.999998804829792,
-0.982278889848666, -0.364341840741643, -0.999999997786457, -0.999999976614482,
-0.998382508964322, -0.999999999999671, -0.999999196325065, -0.999975011497777,
-0.998599571268165, -0.999999979094997, -0.997753198487314, -0.999999926318692,
-0.154416470074132, -0.928951601468295, -0.994011394007064, -0.999999999462366,
-0.999999999998392, -0.999999919376053, -0.99999990340994), religion = c(0.041286647,
0.02607471, 0.066732876, 0.044134054, 0.05952625, 0.06738507,
0.059015673, 0.111649066, 0.025701044, 0.01340009, 0.15596099,
0.020686807, 0.041033328, 0.028240733, 0.017688353, 0.10048646,
0.09002863, 0.045010816, 0.0599233, 0.023459796, 0.18008809,
0.15380068, 0.10875068, 0.16463679, 0.12366201, 0.08062692, 0.020737456,
0.072489396, 0.02980364, 0.03522147, 0.18431833, 0.2241213, 0.04967877,
0.020767821, 0.042845365, 0.114026226, 0.025252016, 0.050515506,
0.028480459, 0.011976497, 0.051216464, 0.071065746, 0.060021035,
0.0671975, 0.022543803, 0.026054224, 0.012819406, 0.012850202,
0.019244831, 0.052286506, 0.104308784, 0.021517357, 0.026059527,
0.04549559, 0.026263602, 0.018891444, 0.04764373, 0.14865479,
0.17848434, 0.032876275, 0.074900545, 0.13018908, 0.14712808,
0.22409584, 0.06142032, 0.04922596, 0.058575258, 0.080876544,
0.054859933, 0.05560649, 0.14968075, 0.09415126, 0.076041564,
0.07222162, 0.04100541, 0.059602886, 0.19613168, 0.022500644,
0.035217304, 0.023242751, 0.01764376, 0.12785666, 0.1234028,
0.03073952, 0.13650659, 0.060998723, 0.12673242, 0.11131127,
0.10484315, 0.20243417, 0.024939083, 0.07396702, 0.03522846,
0.07814219, 0.21069641, 0.14236231, 0.14371029, 0.15000819, 0.050455987,
0.099605545), civic = c(0.09236666, 0.10866666, 0.08693332, 0.10866666,
0.10323332, 0.048899993, 0.09779999, 0.048899993, 0.10866666,
0.10866666, 0.02173333, 0.081499994, 0.081499994, 0.09779999,
0.09779999, 0.05433333, 0.05976666, 0.10866666, 0.10866666, 0.081499994,
0.06519999, 0.06519999, 0.02173333, 0.06519999, 0.05976666, 0.08693332,
0.10323332, 0.08693332, 0.10866666, 0.09779999, 0.06519999, 0.03803333,
0.09236666, 0.10866666, 0.10866666, 0.05976666, 0.09779999, 0.09779999,
0.09779999, 0.10866666, 0.09779999, 0.09236666, 0.09236666, 0.07063332,
0.10866666, 0.09236666, 0.10866666, 0.09779999, 0.10866666, 0.10866666,
0.07606666, 0.09779999, 0.07606666, 0.08693332, 0.10866666, 0.10866666,
0.08693332, 0.05976666, 0.048899993, 0.09779999, 0.06519999,
0.07063332, 0.05976666, 0.05433333, 0.07606666, 0.06519999, 0.07063332,
0.10866666, 0.06519999, 0.081499994, 0.032599997, 0.02173333,
0.081499994, 0.05976666, 0.048899993, 0.10866666, 0.03803333,
0.08693332, 0.10866666, 0.09236666, 0.10866666, 0.07063332, 0.07063332,
0.07606666, 0.03803333, 0.07606666, 0.05433333, 0.05433333, 0.05433333,
0.05433333, 0.09779999, 0.081499994, 0.08693332, 0.06519999,
0.03803333, 0.05976666, 0.02173333, 0.048899993, 0.10323332,
0.081499994), ethnic = c(0.3970396, 0.37897056, 0.5357585, 0.24768388,
0.40286815, 0.6357579, 0.6016036, 0.5114627, 0.6255658, 0.3186922,
0.5825614, 0.7621388, 0.8317045, 0.6124228, 0.6632664, 0.52478975,
0.49357843, 0.30896062, 0.39672756, 0.6146619, 0.662264, 0.283978,
0.79085565, 0.5306848, 0.6103033, 0.35079825, 0.7638138, 0.64863324,
0.44819558, 0.6953889, 0.40650934, 0.3398254, 0.61111623, 0.6295637,
0.3380776, 0.67512846, 0.61698794, 0.5896463, 0.721169, 0.4455908,
0.7558898, 0.75864416, 0.5891671, 0.65199465, 0.29523754, 0.43643963,
0.46105385, 0.7583519, 0.62372357, 0.50434506, 0.53839517, 0.74449384,
0.45155495, 0.7162366, 0.7817946, 0.5408662, 0.68203306, 0.6332905,
0.49937886, 0.6118857, 0.626454, 0.6951653, 0.5778341, 0.41941065,
0.32683736, 0.5616461, 0.5903009, 0.28805655, 0.5589603, 0.38802063,
0.33231544, 0.43056065, 0.47309256, 0.5226548, 0.8516469, 0.29380715,
0.66506696, 0.6927326, 0.33732903, 0.3049189, 0.29078054, 0.35410964,
0.32911754, 0.5134419, 0.37947673, 0.51836216, 0.3146671, 0.8988029,
0.89620274, 0.29536778, 0.24357241, 0.38732207, 0.308964, 0.61849403,
0.39014798, 0.3550223, 0.30616927, 0.91790694, 0.43637908, 0.5408756
), language = c(0.9028321, 0.8534756, 0.9003864, 0.8477236, 0.85861427,
0.9763271, 0.91966915, 0.96824694, 0.89786947, 0.6191133, 0.9333193,
0.9033248, 0.9319655, 0.9029625, 0.6809509, 0.93964654, 0.70345926,
0.86834, 0.91468996, 0.8441426, 0.92808974, 0.9063144, 0.9645202,
0.9443971, 0.5804937, 0.56604165, 0.8854753, 0.81346387, 0.7352802,
0.8515354, 0.94880205, 0.93205106, 0.877309, 0.7811706, 0.9077789,
0.9318178, 0.8326978, 0.8790868, 0.8538892, 0.25064504, 0.7663783,
0.9555273, 0.907167, 0.7620087, 0.66066307, 0.5361982, 0.72005004,
0.8630133, 0.8662945, 0.81044525, 0.9097693, 0.9163808, 0.7922232,
0.92694986, 0.8709022, 0.81088173, 0.90886694, 0.92950064, 0.9560861,
0.8681097, 0.96221495, 0.92667896, 0.9416113, 0.8984488, 0.9538415,
0.8885773, 0.9254957, 0.94662046, 0.9305949, 0.9372623, 0.98557174,
0.96830726, 0.94496214, 0.923303, 0.9672361, 0.9219519, 0.96036005,
0.87589335, 0.8607454, 0.7873163, 0.7968753, 0.89746803, 0.83082134,
0.8995851, 0.9295714, 0.9215403, 0.96248287, 0.9729186, 0.955991,
0.9710674, 0.8113927, 0.9619904, 0.8924399, 0.96472365, 0.9513764,
0.94352907, 0.86856884, 0.9584267, 0.88784045, 0.9044569), voteage = c(0.5415052,
0.5250972, 0.5858911, 0.57861066, 0.60942423, 0.558716, 0.5892219,
0.6370349, 0.6141516, 0.3952495, 0.5966508, 0.69935966, 0.70640314,
0.52056634, 0.43017876, 0.59642285, 0.37977982, 0.53641015, 0.5947154,
0.59636533, 0.68313015, 0.524484, 0.60493284, 0.6814822, 0.26147988,
0.24641654, 0.62900823, 0.60015863, 0.4710629, 0.59511346, 0.6425276,
0.66375273, 0.5933129, 0.52887684, 0.6171358, 0.60419106, 0.53888094,
0.620374, 0.614735, 0.24189119, 0.6001134, 0.6118948, 0.57089466,
0.43977982, 0.44113994, 0.39214894, 0.4736451, 0.58449423, 0.6172989,
0.5052691, 0.56155217, 0.5756319, 0.5006779, 0.56436056, 0.64906085,
0.53386456, 0.79360616, 0.5629912, 0.64399046, 0.6279944, 0.61143416,
0.6120734, 0.6178282, 0.51169556, 0.63740486, 0.46528107, 0.49653965,
0.5406704, 0.49688926, 0.6171718, 0.51588005, 0.60201925, 0.540264,
0.52046335, 0.5638834, 0.55689573, 0.5832646, 0.45817313, 0.58187026,
0.37612963, 0.4259325, 0.49137145, 0.39633512, 0.49645153, 0.49827608,
0.48816267, 0.5490801, 0.61691815, 0.58953744, 0.5359645, 0.43509322,
0.54484385, 0.44624743, 0.5085251, 0.5765146, 0.53727007, 0.45505175,
0.6727599, 0.493701, 0.52082986), local = c(0.11346269, 0.0821763,
0.09865325, 0.13039602, 0.16706811, 0.1925357, 0.18262136, 0.20276488,
0.16904168, 0.18558133, 0.23628975, 0.19656563, 0.22064485, 0.19406708,
0.13554244, 0.18911618, 0.1964137, 0.1288788, 0.1642498, 0.2320314,
0.2406299, 0.23134439, 0.2193604, 0.21352763, 0.18684117, 0.2663458,
0.18552017, 0.1936251, 0.15774018, 0.16654506, 0.2464689, 0.2213913,
0.19595641, 0.17701498, 0.16920264, 0.18539564, 0.19038261, 0.18990363,
0.17630382, 0.16714683, 0.16716042, 0.18136276, 0.14041798, 0.21676183,
0.13718824, 0.1693134, 0.16400187, 0.19431508, 0.15682065, 0.20015918,
0.22922051, 0.15314136, 0.20060387, 0.17444389, 0.17039652, 0.1454538,
0.234232, 0.16478789, 0.1899709, 0.18571664, 0.26345906, 0.2111755,
0.27364796, 0.16966473, 0.16955985, 0.21253699, 0.22058052, 0.15532793,
0.15032071, 0.15741009, 0.2304608, 0.250423, 0.20964451, 0.17636389,
0.16328937, 0.16403592, 0.16296567, 0.1623544, 0.17546876, 0.17806159,
0.13644126, 0.23888248, 0.1776321, 0.15040636, 0.16471201, 0.17668277,
0.24506003, 0.14990062, 0.19695675, 0.20724353, 0.15140234, 0.19517402,
0.18235727, 0.19484001, 0.18699574, 0.16355312, 0.121629976,
0.21160316, 0.1918851, 0.16756512), education = c(0.6537678,
0.6364562, 0.5315682, 0.6130346, 0.72199595, 0.45010185, 0.68533605,
0.51221997, 0.6629328, 0.6099796, 0.4124236, 0.65784115, 0.56619143,
0.67209774, 0.52138495, 0.57331973, 0.1934827, 0.66496944, 0.700611,
0.67413443, 0.45112014, 0.43380857, 0.50610995, 0.37576377, 0.24847251,
0.3940937, 0.6252546, 0.47352344, 0.6262729, 0.5875764, 0.55906314,
0.5366599, 0.59063137, 0.59572303, 0.7443992, 0.55397147, 0.55906314,
0.59979635, 0.58350307, 0.56109977, 0.5682281, 0.62729126, 0.7209776,
0.44093686, 0.6405295, 0.62118125, 0.5712831, 0.6099796, 0.65478617,
0.5549898, 0.50814664, 0.65478617, 0.57026476, 0.65580446, 0.62729126,
0.68329936, 0.4857434, 0.4562118, 0.42260695, 0.6863544, 0.51221997,
0.54887986, 0.4154786, 0.49694502, 0.5875764, 0.5641548, 0.49083507,
0.5397149, 0.6201629, 0.7739307, 0.49694502, 0.33604884, 0.70162934,
0.59063137, 0.63441956, 0.62729126, 0.48472506, 0.6120163, 0.71588594,
0.598778, 0.6171079, 0.49083507, 0.48472506, 0.5916497, 0.58655804,
0.59775966, 0.54276985, 0.6822811, 0.65173113, 0.4582485, 0.6120163,
0.62729126, 0.5967413, 0.55600816, 0.36863542, 0.45723015, 0.401222,
0.6466395, 0.65784115, 0.5875764), employ = c(0.5028835, 0.57900804,
0.47750866, 0.5213379, 0.5121107, 0.4463668, 0.5074971, 0.32987314,
0.449827, 0.5397924, 0.28950405, 0.2802768, 0.23644753, 0.5374856,
0.44059977, 0.33679354, 0.35178775, 0.5363322, 0.4152249, 0.37600923,
0.37139562, 0.32064593, 0.34256056, 0.3252595, 0.37831604, 0.37600923,
0.30449826, 0.28835064, 0.5432526, 0.3690888, 0.30680507, 0.34256056,
0.42214534, 0.4152249, 0.5432526, 0.3656286, 0.43367937, 0.3287197,
0.39792386, 0.49942333, 0.57324106, 0.49019605, 0.5098039, 0.38523644,
0.56747407, 0.5224914, 0.46597463, 0.4106113, 0.48212227, 0.44059977,
0.32641295, 0.5374856, 0.38985008, 0.46251443, 0.37946942, 0.5859285,
0.118800454, 0.4083045, 0.34371397, 0.44059977, 0.49250287, 0.4544406,
0.36678204, 0.5340254, 0.38177624, 0.5201846, 0.5363322, 0.42791235,
0.5940023, 0.46251443, 0.39677048, 0.43137255, 0.44059977, 0.47866204,
0.5524798, 0.5098039, 0.37024224, 0.61937714, 0.4913495, 0.4948097,
0.6297578, 0.4394464, 0.4936563, 0.55709344, 0.4544406, 0.5870819,
0.38292962, 0.48096886, 0.54094577, 0.461361, 0.5282584, 0.37024224,
0.5340254, 0.5524798, 0.4175317, 0.38869664, 0.48096886, 0.40138412,
0.5928489, 0.46943483), socialcap = c(1.2465897, 1.1796607, 0.99583554,
1.213838, 1.2140356, 1.0433682, 1.1867946, 1.0998459, 1.1968888,
0.9863519, 1.0743337, 1.2542336, 1.2588876, 1.167894, 1.0432669,
1.0626454, 0.88172776, 1.0901438, 1.1371455, 1.1487094, 1.0707088,
1.1323296, 1.0424548, 1.3017348, 0.7089147, 0.99225324, 1.006372,
1.0797297, 1.0777289, 1.0685663, 1.1497757, 1.0380225, 1.1510758,
1.0681523, 1.1533836, 1.1480858, 1.109215, 1.0647533, 1.108393,
0.86751366, 1.1352568, 1.250612, 1.1412742, 0.9820876, 1.0141772,
0.9327179, 0.9930749, 1.0881922, 1.2074403, 1.0714755, 1.1362286,
1.1118767, 0.96056145, 1.2471367, 1.2069551, 0.9745213, 1.0848411,
1.1748116, 1.166508, 1.1300496, 1.1630167, 1.1959319, 1.1653817,
1.0205215, 1.0896816, 1.0423381, 1.0528433, 1.1218456, 1.1675899,
1.1556351, 1.0114185, 1.0285435, 1.2299659, 1.0582013, 1.1651775,
1.124054, 1.0567757, 1.1237862, 1.135855, 0.9389708, 1.0796958,
0.9262693, 0.8858328, 1.1860313, 0.9620687, 1.1273029, 1.0348538,
1.1883315, 1.1845229, 1.0326669, 1.0741078, 1.1087412, 1.0355263,
1.1264266, 1.0276779, 0.95914507, 0.9550716, 1.1294575, 1.1779915,
1.1125866)), row.names = c(NA, -100L), class = c("tbl_df", "tbl",
"data.frame"))
This is how I filtered the two dataframes to have the same county fips:
US <- US[which(US$FIPS %in% c(test$FIPS)),]
biz <- biz[which(biz$FIPS %in% c(US$FIPS)),]
Any help would be greatly appreciated!

Finding increases from 'baseline' in the graph, not sure how to do

I want to write an algorithm that spits out the points highlighted by arrows. I've tried using a second derivative but it returns a similar plot to the one above and not sure how to use it.
Hi, sorry about that, I don't want the peaks, I want the point where the graph starts to increase - ie I want the point where the gradient changes from ~0 to something larger, does that make sense
Example data is below.
df = structure(list(X1 = c("2729", "2730", "2731", "2732", "2733",
"2734", "2735", "2736", "2737", "2738", "2739", "2740", "2741",
"2742", "2743", "2744", "2745", "2746", "2747", "2748", "2749",
"2750", "2751", "2752", "2753", "2754", "2755", "2756", "2757",
"2758", "2759", "2760", "2761", "2762", "2763", "2764", "2765",
"2766", "2767", "2768", "2769", "2770", "2771", "2772", "2773",
"2774", "2775", "2776", "2777", "2778", "2779", "2780", "2781",
"2782", "2783", "2784", "2785", "2786", "2787", "2788", "2789",
"2790", "2791", "2792", "2793", "2794", "2795", "2796", "2797",
"2798", "2799", "2800", "2801", "2802", "2803", "2804", "2805",
"2806", "2807", "2808", "2809", "2810", "2811", "2812", "2813",
"2814", "2815", "2816", "2817", "2818", "2819", "2820", "2821",
"2822", "2823", "2824", "2825", "2826", "2827", "2828", "2829",
"2830", "2831", "2832", "2833", "2834", "2835", "2836", "2837",
"2838", "2839", "2840", "2841", "2842", "2843", "2844", "2845",
"2846", "2847", "2848", "2849", "2850", "2851", "2852", "2853",
"2854", "2855", "2856", "2857", "2858", "2859", "2860", "2861",
"2862", "2863", "2864", "2865", "2866", "2867", "2868", "2869",
"2870", "2871", "2872", "2873", "2874", "2875", "2876", "2877",
"2878", "2879", "2880", "2881", "2882", "2883", "2884", "2885",
"2886", "2887", "2888", "2889", "2890", "2891", "2892", "2893",
"2894", "2895", "2896", "2897", "2898", "2899", "2900", "2901",
"2902", "2903", "2904", "2905", "2906", "2907", "2908", "2909",
"2910", "2911", "2912", "2913", "2914", "2915", "2916", "2917",
"2918", "2919", "2920", "2921", "2922", "2923", "2924", "2925",
"2926", "2927", "2928", "2929", "2930", "2931", "2932", "2933",
"2934", "2935", "2936", "2937", "2938", "2939", "2940", "2941",
"2942", "2943", "2944", "2945", "2946", "2947", "2948", "2949",
"2950", "2951", "2952", "2953", "2954", "2955", "2956", "2957",
"2958", "2959", "2960", "2961", "2962", "2963", "2964", "2965",
"2966", "2967", "2968", "2969", "2970", "2971", "2972", "2973",
"2974", "2975", "2976", "2977", "2978", "2979", "2980", "2981",
"2982", "2983", "2984", "2985", "2986", "2987", "2988", "2989",
"2990", "2991", "2992", "2993", "2994", "2995", "2996", "2997",
"2998", "2999", "3000", "3001", "3002", "3003", "3004", "3005",
"3006", "3007", "3008", "3009", "3010", "3011", "3012", "3013",
"3014", "3015", "3016", "3017", "3018", "3019", "3020", "3021",
"3022", "3023", "3024", "3025", "3026", "3027", "3028", "3029",
"3030", "3031", "3032", "3033", "3034", "3035", "3036", "3037",
"3038", "3039", "3040", "3041", "3042", "3043", "3044", "3045",
"3046", "3047", "3048", "3049", "3050", "3051", "3052", "3053",
"3054", "3055", "3056", "3057", "3058", "3059", "3060", "3061",
"3062", "3063", "3064", "3065", "3066", "3067", "3068", "3069",
"3070", "3071", "3072", "3073", "3074", "3075", "3076", "3077",
"3078", "3079", "3080", "3081", "3082", "3083", "3084", "3085",
"3086", "3087", "3088", "3089", "3090", "3091", "3092", "3093",
"3094", "3095", "3096", "3097", "3098", "3099", "3100", "3101",
"3102", "3103", "3104", "3105", "3106", "3107", "3108", "3109",
"3110", "3111", "3112", "3113", "3114", "3115", "3116", "3117",
"3118", "3119", "3120", "3121", "3122", "3123", "3124", "3125",
"3126", "3127", "3128", "3129", "3130", "3131", "3132", "3133",
"3134", "3135", "3136", "3137", "3138", "3139", "3140", "3141",
"3142", "3143", "3144", "3145", "3146", "3147", "3148", "3149",
"3150", "3151", "3152", "3153", "3154", "3155", "3156", "3157",
"3158", "3159", "3160", "3161", "3162", "3163", "3164", "3165",
"3166", "3167", "3168", "3169", "3170", "3171", "3172", "3173",
"3174", "3175", "3176", "3177", "3178", "3179", "3180", "3181",
"3182", "3183", "3184", "3185", "3186", "3187", "3188", "3189",
"3190", "3191", "3192", "3193", "3194", "3195", "3196", "3197",
"3198", "3199", "3200", "3201", "3202", "3203", "3204", "3205",
"3206", "3207", "3208", "3209", "3210", "3211", "3212", "3213",
"3214", "3215", "3216", "3217", "3218", "3219", "3220", "3221",
"3222", "3223", "3224", "3225", "3226", "3227", "3228", "3229",
"3230", "3231", "3232", "3233", "3234", "3235", "3236", "3237",
"3238", "3239", "3240", "3241", "3242", "3243", "3244", "3245",
"3246", "3247", "3248", "3249", "3250", "3251", "3252", "3253",
"3254", "3255", "3256", "3257", "3258", "3259", "3260", "3261",
"3262", "3263", "3264", "3265", "3266", "3267", "3268", "3269",
"3270", "3271", "3272", "3273", "3274", "3275", "3276", "3277",
"3278", "3279", "3280", "3281", "3282", "3283", "3284", "3285",
"3286", "3287", "3288", "3289", "3290", "3291", "3292", "3293",
"3294", "3295", "3296", "3297", "3298", "3299", "3300", "3301",
"3302", "3303", "3304", "3305", "3306", "3307", "3308", "3309",
"3310", "3311", "3312", "3313", "3314", "3315", "3316", "3317",
"3318", "3319", "3320", "3321", "3322", "3323", "3324", "3325",
"3326", "3327", "3328", "3329", "3330", "3331", "3332", "3333",
"3334", "3335", "3336", "3337", "3338", "3339", "3340", "3341",
"3342", "3343", "3344", "3345", "3346", "3347", "3348", "3349",
"3350", "3351", "3352", "3353", "3354", "3355", "3356", "3357",
"3358", "3359", "3360", "3361", "3362", "3363", "3364", "3365",
"3366", "3367", "3368", "3369", "3370", "3371", "3372", "3373",
"3374", "3375", "3376", "3377", "3378", "3379", "3380", "3381",
"3382", "3383", "3384", "3385", "3386", "3387", "3388", "3389",
"3390", "3391", "3392", "3393", "3394", "3395", "3396", "3397",
"3398", "3399", "3400", "3401", "3402", "3403", "3404", "3405",
"3406", "3407", "3408", "3409", "3410", "3411", "3412", "3413",
"3414", "3415", "3416", "3417", "3418", "3419", "3420", "3421",
"3422", "3423", "3424", "3425", "3426", "3427", "3428", "3429",
"3430", "3431", "3432", "3433", "3434", "3435", "3436", "3437",
"3438", "3439", "3440", "3441", "3442", "3443", "3444", "3445"
), X2 = c(-0.00385000000001254, -0.0154500000000484, -0.0277600000000007,
-0.0154500000000279, -0.0386000000000704, -0.0154500000000329,
-0.0115500000000053, 2.5238009638656e-15, -0.00385000000000757,
3.60475000000867, -0.470850000000881, -0.347350000000663, -0.173700000000328,
-0.139699999999998, -0.096500000000187, -0.0617500000001111,
-0.0579000000001016, -0.0424500000000768, -0.050150000000105,
-0.0579000000001191, -0.0540000000000976, -0.0579000000001924,
-0.0270000000000563, -0.0309000000000539, -0.0231500000000468,
-0.0270500000000538, -0.00775000000002209, -0.0193000000000404,
-0.0131199999999931, 0.219999999999842, 0.0579000000001427, -0.061750000000126,
-0.0617500000002055, -0.0309000000000726, -0.050150000000105,
-0.042450000000091, -0.0193000000000293, -0.0309000000000144,
-0.0115500000000196, -0.0116000000000154, -0.0154500000000366,
-0.00385000000000946, -0.0193000000000305, -0.00390000000000946,
-0.00390000000000639, -0.00771000000000015, -0.000789999999999225,
-4.97400384373025e-15, -0.00619000000000085, -0.0116000000000265,
-0.011550000000014, -0.00385000000000504, -0.00538999999999987,
-0.0116000000000203, -0.011550000000014, 0.00385000000001136,
-0.00230999999999795, 2.86419210237446e-15, -0.00230999999999954,
-0.00770000000002508, -0.00770000000001703, -0.00390000000000449,
-0.0085000000000008, -0.0193000000000529, -8.05101707233625e-15,
-0.00385000000001751, -0.0146699999999988, -0.00619000000000085,
-0.0116000000000265, 0.00153999999999996, 0.00385000000000546,
-0.00231000000000233, -0.000780000000000314, -0.00230999999999884,
0.0015400000000021, -8.05101707233625e-15, -0.00848000000000013,
-0.00385000000001751, -0.00775000000003729, -0.00769999999999792,
-1.1787959787484e-15, -0.00384999999999692, 0.00385000000001136,
-0.00384999999999762, 0.00385000000000639, -0.00385000000001161,
-0.000440000000001542, -0.00390000000000639, -0.000769999999999981,
0, -0.0154500000000091, -0.0077500000000059, -0.0154500000000335,
-0.0115500000000165, -0.00385000000000567, -0.00311000000000092,
0.0116000000000272, -0.00230999999999994, 0.0116000000000172,
0.00770000000001277, -0.00385000000000377, -0.00385000000001254,
0.00385000000001136, -0.00385000000000411, -0.0038499999999997,
-0.0116000000000215, -0.0154300000000006, -6.15348059644161e-15,
-0.00849999999999866, -0.0015500000000003, 0.00154000000000174,
-3.07674029821757e-15, -0.0115500000000345, -0.0115500000000165,
-6.15348059644161e-15, -0.00385000000002247, 0.0077000000000059,
-0.00385000000001254, -0.0115500000000315, -0.0154500000000107,
-0.0154500000000229, -0.0309000000000733, -1.65190000000256,
-0.258600000000477, -0.111900000000204, -0.0640499999999989,
-0.0579000000001016, -0.0270000000000494, -0.02393, -0.0193000000000324,
-0.0115500000000165, -0.0270000000000624, -0.0193000000000598,
-0.0309000000000733, -0.0463000000001036, -2.19220000000482,
-0.524900000000959, -0.189100000000636, -0.11580000000022, -0.0717700000000001,
-0.0424500000001407, -0.057900000000101, -0.0386000000000673,
-0.0193000000000449, -0.0277899999999995, -0.0077500000000276,
-0.0208600000000011, -0.0193000000000293, -0.0463000000000912,
-0.0386000000000716, -0.0501500000001031, -0.0347500000000728,
-0.0502000000000926, -0.0424500000000836, -0.00307999999999993,
-0.0116000000000234, 0.00389999999999833, -0.000769999999999981,
-0.00153999999999996, -0.00153999999999996, 0.00153999999999783,
-0.0162100000000009, -0.0386000000000797, -0.0432300000000026,
-0.038600000000117, -0.050200000000097, -0.0309000000000527,
-0.0231500000000593, 0.00461999999999989, -0.00385000000001064,
-0.00385000000000757, -0.0116000000000215, 0.00770000000004104,
0.00385000000000639, -0.941700000001459, -0.169850000000308,
-0.100350000000196, -0.0933799999999984, -0.0617500000001154,
-0.0579000000001165, -0.0386000000000822, -0.019300000000043,
-0.0231500000000629, -0.0115500000000165, -0.0270000000000464,
-0.0116000000000284, -0.00769999999999982, -2.76340000000441,
-0.270200000000513, -0.119650000000229, -0.108100000000387, -0.0540000000001033,
-0.0772000000001527, -0.0579000000001345, -0.0656000000001255,
-0.0540500000001704, -0.0386000000000716, -0.0270500000000663,
-0.0116000000000284, -0.0216200000000043, -0.00770000000001206,
-0.0308500000000552, -0.0115500000000265, -2.4190463576414e-14,
-0.00770000000003006, -0.0115900000000011, -0.0231500000000985,
-0.0193000000000293, -0.033979999999999, -0.00775000000002643,
-0.0478400000000022, -0.0231500000000412, -0.019300000000043,
-0.00233000000000134, -0.00390000000002501, 0.00154999999999958,
0.00384999999999991, 0.0077000000000059, -0.00770000000003193,
-0.0200899999999983, -0.0193000000000423, -0.0347000000000634,
-0.0540000000000927, -0.0733500000001364, -0.0501500000001637,
-0.0424500000000886, -0.050200000000087, -0.0308500000000459,
0.00384999999999834, -0.00231000000000208, -0.00387000000000167,
0.0030799999999978, -0.00385000000000757, -0.00385000000001064,
-0.0192500000000504, -0.0115500000000296, -0.0231500000001104,
-0.0579000000001085, -0.0733500000001314, -0.0386000000000697,
-0.0386000000000754, -0.0347500000000935, -0.00775000000001395,
0.00385000000000881, 0.000769999999999982, 0.0115500000000203,
0.00390000000001095, 0.00154000000000294, -0.00385000000001497,
-0.00385000000000567, -0.0309000000001234, -0.0347500000000728,
-0.0193000000000814, -0.0424500000000992, -0.0347500000000678,
0.274000000000822, 0.463150000000818, 1.03820000000353, 0.636800000000563,
-0.13663, -0.87225000000281, 0.644550000001354, -0.0579000000003174,
-0.72560000000209, -0.115800000000169, 2.08025000000553, -0.208400000000342,
-0.227700000000415, -0.328050000000636, -0.169850000000303, -0.104200000000212,
-0.0656500000001349, -0.0656500000001373, -0.0424500000000712,
-0.0347500000000697, -0.0285600000000002, -0.0193000000000324,
-0.0270000000000538, -0.0193000000000498, -0.0270000000000513,
-0.00849999999999724, -0.00770000000001513, -0.0162100000000009,
-0.0339800000000025, -0.0502000000001566, -0.0501500000000907,
-0.0193000000000454, -0.00770000000001893, 0.00385000000001136,
0.00390000000001402, 0.00153999999999996, -0.00307999999999993,
0.00390000000000023, 0.00384999999999834, 0.00384999999999644,
0.00385000000002943, -0.0138899999999971, -0.0223899999999993,
-0.0270500000000588, -0.00618999999999943, -0.0270500000000669,
0.00153999999999892, -0.000779999999999603, -2.5238009638656e-15,
0.00465000000000089, -0.00770000000001703, -2.91289464345889e-16,
0.00461999999999805, -0.0115900000000011, -0.00390000000001506,
-0.019300000000043, -0.0115899999999989, -0.0115900000000011,
-0.00770000000003258, 0, 0.00390000000000331, 0.0193000000000281,
0.00385000000002044, 0.00770000000002145, 0.00770000000000148,
0.0077000000000078, 0, 0.00308000000000135, -6.15348059644161e-15,
-0.015450000000036, -0.0309000000000726, -0.00385000000001254,
-0.0154000000000341, -1.11274169835756e-14, -0.00923999999999978,
-0.00234000000000107, -0.00770999999999944, 0.00385000000003251,
0.00461999999999429, 0.00385999999999811, -0.00770000000000798,
-0.023150000000093, -0.0154500000000348, -0.0424500000000737,
-0.019300000000043, -0.0308500000000125, -0.0309000000001054,
-0.0231500000000394, -1.1787959787484e-15, 0.000790000000000646,
-0.00231000000000036, 0, -0.00307999999999851, -0.00390000000002326,
-0.00230999999999753, -0.0193100000000022, -0.042450000000016,
-0.0385500000000679, -0.057900000000106, -0.0347000000000627,
-0.0386000000000922, -0.00385000000000445, 0.0077500000000097,
0.00230999999999995, -0.00385000000000352, 0.00307999999999948,
-0.000769999999999381, -1.1787959787484e-15, -0.015440000000001,
-0.0193000000000099, -0.0425000000000806, -0.0386000000000829,
-0.0424500000001675, -0.0386000000000773, -0.0463000000000192,
-0.00385000000001562, 0, 0.00769999999999875, -3.07674029821757e-15,
-0.00307999999999922, -0.0030799999999978, -0.0154000000000493,
-0.00385000000001254, -0.0231500000000079, -0.0347500000000802,
-0.0231500000000319, -0.0355200000000003, -0.0386000000000829,
-0.0463500000000801, -0.0347500000000678, 0.00155999999999792,
0.00385000000000639, -0.00385000000000231, 0, -0.00385000000000946,
-0.00153999999999966, 0, -0.0285600000000002, -0.0309000000000546,
-0.069450000000125, -0.0502000000000889, -0.0502000000000896,
0.3898000000001, 0.0540500000001028, 0.0115500000000253, 0.0116000000000142,
0.000769999999999981, -0.00385000000000504, -7.40090066366128e-15,
-0.00230999999999995, 0.00385000000000141, 0.00385000000000639,
-0.00385000000001254, -0.0270199999999981, -0.0502000000000896,
0.409100000000679, 0.0386000000000674, -0.0116000000000445, -0.00775000000004081,
-0.00307999999999993, -0.00385000000000757, -0.00770000000000208,
-0.00385000000001562, -0.00385000000001064, -0.00153999999999783,
-8.05101707233625e-15, -0.0177499999999995, -0.0424500000001796,
-0.0509500000000003, -0.0694500000001324, -0.0424500000001587,
-0.061750000000121, -0.0232000000000369, -0.0131299999999989,
-0.00153999999999886, 0.0077000000000059, -0.00310999999999879,
0.00769999999999982, 7.49841812496252e-15, -8.05101707233625e-15,
-0.0077000000000406, -0.0424500000000787, -0.0502000000001032,
-0.0347500000000747, -0.0656000000001262, -0.0733000000001494,
-0.034700000000074, -0.0193000000000869, 0.0231500000000662,
-0.00385000000000757, 0.00770000000001088, 0.0115600000000001,
-0.957150000001501, -0.14670000000027, -0.0772000000001383, -0.0617500000002002,
-0.0463000000000981, -0.0617500000001229, -0.0270000000000544,
-0.0347500000000597, -0.0386000000001412, -0.0694500000001331,
-0.0887500000001619, -0.0386000000000747, -0.0077500000000295,
0.015400000000064, 0.355050000000611, 0.0478699999999975, -6.15348059644161e-15,
-0.0177800000000019, -0.00385000000001064, -0.0116000000000674,
-0.0154500000000435, -0.0524900000000017, -0.0540500000002006,
-0.0540500000001021, -0.0617500000002394, -0.0308500000000688,
-0.0193000000000355, -0.0154000000000216, -0.000770000000000204,
0.00770000000001775, 0.00694000000000017, -0.00385000000001254,
0.0116000000000123, -0.00385000000002063, -0.00385000000000757,
-0.0270000000000743, -0.0309000000000763, -0.0926500000001775,
-0.0887500000003161, -0.0656500000001199, -0.042450000000078,
-0.0270000000000588, -0.0116000000000364, -0.000769999999999982,
-0.00385000000001254, -0.015400000000026, 0.00385000000002041,
0.0115500000000253, 0, -0.00385000000001161, -0.0386000000000723,
-0.0154500000000693, -0.100350000000196, -0.0849000000001704,
-0.0926500000001751, -0.0115500000000116, 0.00385000000000546,
-0.0116000000000154, 6.87160777622118e-15, -0.00384999999999991,
0.00231999999999886, -3.07674029821757e-15, 0.00390000000003514,
0.000779999999996745, -3.07674029821757e-15, -0.0231500000000617,
-0.0270500000000527, -0.0517200000000003, -0.050150000000105,
-0.0347500000000721, -0.0347500000000142, -0.00385000000001161,
0.00770000000000401, -0.00385000000000197, 0.000769999999999982,
-0.00385000000001372, 0.00385000000000141, 0.0116000000000278,
-3.71670324204166e-15, -0.0116000000000584, -0.00385000000001064,
-0.00464999999999875, -0.00775000000004982, -0.00390000000001506,
0.277900000000906, 0.119650000000208, 0.054000000000013, 0.0463000000000931,
0.0154500000000168, 0.00775000000000384, 0.0115500000000154,
0.00769999999999875, 1.89760393249092e-15, 0.00231999999999957,
0.000769999999999304, -0.0231500000000085, -0.0270500000000402,
0.351200000000562, -0.0231500000000833, -0.0270500000000588,
-0.0463500000000216, -0.0139000000000062, -9.23022089465272e-15,
-8.05101707233625e-15, 0.00385000000000546, 0.000759999999998229,
-0.0115500000000395, 0.000769999999999982, -0.011600000000024,
-0.00770000000001206, -0.0540500000001929, -0.0772000000001558,
-0.0656000000000217, -0.0772000000001484, -0.0579000000001128,
-0.0347000000000764, -0.0193000000000461, -0.00385000000000352,
-0.00385000000002122, -0.00696000000000083, 0.000789999999999225,
0.00384999999999834, -0.000800000000000978, -0.0116000000000234,
-0.00775000000001088, -0.0115900000000055, -0.0193000000000218,
-0.0347500000000808, -0.0386000000000897, -0.0501500000000858,
-0.00233999999999881, -0.00385000000000757, 2.00000000009208e-05,
0.308750000000515, 0.092650000000154, 0.0424500000000756, 0.0231500000000227,
0.0154500000000312, -0.00385000000001469, 0.00538999999999237,
0.474750000000936, 0.212300000000357, -0.0030699999999996, -0.0309000000000739,
-0.0115500000000265, -0.0116000000000265, -3.57390000000716,
-0.293350000001048, -0.119650000000226, -0.104200000000194, -0.0926500000001831,
-0.0540500000001096, -0.0694500000002714, -0.0772000000001527,
-0.0965000000001976, -0.0694500000001375, -0.100350000000182,
-0.084950000000289, -0.061750000000121, -0.0425000000000912,
-0.0424500000000662, -0.00770000000002011, -0.0154500000000422,
-0.00307999999999993, -0.00230999999999994, 0.00385000000001447,
-0.00154, -0.00385000000000567, -0.0386000000000747, -0.0695000000002463,
-0.0772000000001664, -0.0849000000002961, -0.0887500000001668,
-0.0193000000000504, -0.0578500000001047, -0.00775000000000708,
-1.2095231788207e-14, 0.00848999999999485, -3.07674029821757e-15,
-0.00541000000000057, -0.00390000000002247, 0.000769999999999981,
-0.0293300000000002, -0.050200000000087, -0.0656000000002546,
-0.0540500000001096, -0.069450000000138, 0.123500000000375, 0.0849000000001387,
0.00384999999999644, 0.023200000000042, 0.0115500000000123, 0.00775000000000473,
0.0115500000000203, 0.00385000000001447, -0.00775000000002506,
0.00466000000000122, -0.0254699999999978, -0.054799999999998,
-0.0231500000000444, 0.0116000000000454, 0.115800000000206, 0.030900000000046,
0.00385000000000331, -0.00153999999999996, 0.00384999999999084,
-0.00385000000000757, 0.00770000000001088, 1.7849988639723e-14,
0.00230999999999994, 0.00385000000001326, -0.00153999999999882,
-0.038600000000126, -0.0309000000000553, -0.00692999999999628,
-0.0154000000000403, -0.0579000000001097, -0.0347500000000678,
-0.0100400000000054, 0.00385000000000023, -0.00385000000001994,
-2.17923926727129e-14, 0.00389999999999028, 0.00390000000001402,
0.00384999999999084, -0.00385000000001751, 0.00770000000001399,
-0.0308500000000632, -0.0502000000001986, -0.0695000000001394,
-0.0501799999999982, -0.0309000000000752, -0.0270500000000557,
-0.0100500000000011, 0.00389999999999596, 0.0116000000000117,
1.89760393249092e-15, 0.0115500000000123, 0.00384999999998841,
-0.00385000000002965, 0.0077000000000078, 0.00385000000000639,
0.00770000000000283, -0.0501500000001132, -0.0617500000002242,
-0.0710100000000004, -0.0810500000000306, -0.0540500000001891,
-0.0386000000000617, -0.019300000000043, 0.00775000000000473,
0.00847000000000282, 0.00462999999999951, -2.11128370304365e-14,
0.00770000000001088, 0.00384999999999858, 9.99999999962123e-06,
-0.00770000000001206, -0.0733000000000254, -0.0656000000001967,
-0.111900000000213, -0.100350000000323, -0.0579000000001141,
-0.0385500000000131, -0.0116000000000215, 0.0193000000000318,
0.00390000000001402, 0.0270000000000452, 0.00770000000000182,
-8.05101707233625e-15)), row.names = c(NA, -717L), class = "data.frame")
As others have said, it is not clear what you are looking for.
specifically, it's not clear how high above "baseline" is too high.
Here's a shot at it:
df_prime <- df$X2[-1] - df$X2[-length(df$X2)]
large_rise <- which(df_prime > sd(df_prime) & df$X2[-length(df$X2)] > -sd(df$X2))
df$X1[large_rise]
It's difficult to know from the question, but aren't you just looking for something like this?
spikes <- as.numeric(df$X1[df$X2 > 0.1])
spikes <- spikes[which(diff(c(0, spikes)) > 3)]
spikes
#> [1] 2738 2758 2984 2994 3126 3139 3190 3260 3273 3309 3316 3363 3377
So, for example if you did
plot(df$X1, df$X2, type = "l")
points(spikes, rep(1, length(spikes)), col="red")
You would get

unable to parse xml file in R

here is the file that need to parse and eventually convert in dataframe-
<?xml version="1.0" encoding="UTF-8"?>
-<message hash="fb73481d3f3d2b9a70733d69268de71c84f151f8" type="xml" sessionid="https" connector_id="4510010" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<comment/>
-<drive utc_timestamp="2015-09-03T00:09:01.362058" report_name="HTTP Logging to RSP" device_type="ACS 800" sw_appl="ASARF012" sw_type="AS7R7322" serial_number="-">
-<signals timezone="UTC +00:00">
<signal unit="rpm" scale="1" timestamp="2015-09-03T00:07:28.006" name="02_02_01_speed_ref_2" value="0"/>
<signal unit="%" scale="1" timestamp="2015-09-03T00:07:28.511" name="02_01_05_torque" value="0"/>
<signal unit="C" scale="1" timestamp="2015-09-03T00:07:45.352" name="02_01_32_pp_3_temp" value="0"/>
<signal unit="C" scale="1" timestamp="2015-09-03T00:07:46.203" name="02_01_33_pp_4_temp" value="0"/>
<signal unit="C" scale="1" timestamp="2015-09-03T00:07:48.155" name="02_01_35_motor_1_temp" value="0"/>
<signal unit="C" scale="1" timestamp="2015-09-03T00:07:48.911" name="02_01_36_motor_2_temp" value="0"/>
<signal unit="C" scale="1" timestamp="2015-09-03T00:07:52.723" name="02_01_37_motor_temp_est" value="29.999969"/>
<signal unit="" scale="1" timestamp="2015-09-03T00:07:53.638" name="02_03_18_alarm_word_5" value="0"/>
<signal unit="" scale="1" timestamp="2015-09-03T00:07:54.747" name="02_03_19_int_init_fault" value="0"/>
<signal unit="" scale="1" timestamp="2015-09-03T00:07:56.884" name="02_03_11_follower_mcw" value="0"/>
<signal unit="" scale="1" timestamp="2015-09-03T00:07:58.405" name="02_03_13_aux_status_word_3" value="1030"/>
<signal unit="" scale="1" timestamp="2015-09-03T00:07:59.806" name="02_03_14_aux_status_word_4" value="0"/>
<signal unit="" scale="1" timestamp="2015-09-03T00:08:00.485" name="02_03_16_alarm_word_4" value="0"/>
</signals>
</drive>
</message>
I have tried multiple options (XML package is used) but I am not able to convert it into a dataframe/structured form.
## method 1
result <- xmlParse(file = "test.xml")
print(result)
# method 2
xmldataframe <- xmlToDataFrame("test.xml")
print(xmldataframe)
# method 3
xmldoc <- xmlParse(file = test.xml)
rootNode <- xmlRoot(xmldoc)
xmlSApply(rootNode,function(x) xmlSApply(x, xmlValue))
cd.catalog <- data.frame(t(data),row.names=NULL)
none of results/xmldataframe/cd.catalog has dataframe/list in structured form that i can use for further analysis.
Hope you are looking for something like this:
library(xml2)
library(dplyr)
xml_doc <- read_xml("test.xml")
df <- xml_doc %>%
xml_find_all("//signal") %>%
xml_attrs() %>%
unlist() %>%
matrix(ncol=5, byrow=T) %>%
as.data.frame(stringsAsFactors=FALSE)
colnames(df) <- c('unit','scale','timestamp','name','value')
df

Calculating Time Weighted Rate of Return in R

Is there an R function or library that will give me the monthly (or any other specified timeframe) time weighted rate of return (twrr) for my portfolio?
I am including a dput dump of sample data below of the date and portfolio ending balance below. Not sure why the dates were dput'ed the way they were, but the first date 12053 is '2003-01-01' and the last date 12195 is '2003-05-23'.
portfolio.df <- structure(
list(
Date = structure(c(12053, 12054, 12055, 12058,
12059, 12060, 12061, 12062, 12065, 12066, 12067, 12068, 12069,
12073, 12074, 12075, 12076, 12079, 12080, 12081, 12082, 12083,
12086, 12087, 12088, 12089, 12090, 12093, 12094, 12095, 12096,
12097, 12101, 12102, 12103, 12104, 12107, 12108, 12109, 12110,
12111, 12114, 12115, 12116, 12117, 12118, 12121, 12122, 12123,
12124, 12125, 12128, 12129, 12130, 12131, 12132, 12135, 12136,
12137, 12138, 12139, 12142, 12143, 12144, 12145, 12146, 12149,
12150, 12151, 12152, 12153, 12156, 12157, 12158, 12159, 12163,
12164, 12165, 12166, 12167, 12170, 12171, 12172, 12173, 12174,
12177, 12178, 12179, 12180, 12181, 12184, 12185, 12186, 12187,
12188, 12191, 12192, 12193, 12194, 12195),
class = "Date"),
Ending_Balance = c(56250000L,
56852500L, 57080000L, 57355000L, 57477500L, 56817500L, 57885000L,
57810000L, 57732500L, 57670000L, 57520000L, 57285000L, 57270000L,
56655000L, 55802500L, 56337500L, 55642500L, 54510000L, 54987500L,
55802500L, 56065000L, 56865000L, 56635000L, 56497500L, 56640000L,
56155000L, 55757500L, 55972500L, 55865000L, 55535000L, 55885000L,
56840000L, 56902500L, 56945000L, 56622500L, 57012500L, 57200000L,
58072500L, 57612500L, 57447500L, 57157500L, 57032500L, 57405000L,
57502500L, 56785000L, 57007500L, 56342500L, 55697500L, 56655000L,
56900000L, 57002500L, 57465000L, 57467500L, 57382500L, 57982500L,
56562500L, 58065000L, 58935000L, 58502500L, 58200000L, 57767500L,
57757500L, 58055000L, 58305000L, 58277500L, 58295000L, 59047500L,
58907500L, 59125000L, 59072500L, 59107500L, 59315000L, 59690000L,
58957500L, 59407500L, 59385000L, 59965000L, 60297500L, 59890000L,
59822500L, 60367500L, 60407500L, 60380000L, 60815000L, 61155000L,
61080000L, 61132500L, 61265000L, 60912500L, 61107500L, 61445000L,
61345000L, 61137500L, 61035000L, 60707500L, 61340000L, 61365000L,
61402500L, 61640000L, 61675000L)),
.Names = c("Date", "Ending_Balance"),
row.names = c(NA, 100L),
class = "data.frame")

Resources