Extracting and averaging monthly data in R using grep? - r
In advance I have to say that I'm truly sorry for poor formatting! I just can not seem to get a hang of it...
I am also an R noob.
So,
I have a 80 files containing information on 80 individual fish's movements;
> head(PAVdata)
X meanlat meanlong meandepth mindepth maxdepth depthrange rec numhits datetime mon X.1 Y Z
1 23 58.39601 8.708952 6.493529 5.72 7.47 1.75 4 17 2012-05-11 11:15:00 1 482986.7 6472838 6.493529
2 24 58.39618 8.710993 4.035588 1.76 7.47 5.71 6 34 2012-05-11 11:45:00 1 483106.1 6472855 4.035588
3 25 58.39615 8.710919 6.037692 3.08 8.35 5.27 6 26 2012-05-11 12:15:00 1 483101.8 6472853 6.037692
4 26 58.39556 8.710654 3.987500 2.20 6.16 3.96 5 16 2012-05-11 12:45:00 1 483086.0 6472787 3.987500
5 27 58.39617 8.711363 2.944615 2.20 3.96 1.76 6 26 2012-05-11 13:15:00 1 483127.8 6472855 2.944615
6 28 58.39628 8.710904 2.860000 2.20 3.52 1.32 6 24 2012-05-11 13:45:00 1 483101.0 6472867 2.860000
But the columns in interest is datetime and Z, where Z is depth.
What I am aiming to do is;
Get a daily and monthly average of depth
Get a daily and monthly min & max (look at amplitude)
What I've tried so far is;
june<-c(PAVdata$datetime[grep("^2012-07",as.character(PAVdata$datetime))])
Which yield something thats completely farfetched to me
head(june)
[1] 2390 2391 2392 2393 2394 2395
I was reading about subset and aggregate and POSIXlt and tapply and zoo and wanted to try these, but I seemed to get the error that there was an length diff ..
Heres an attempt with the POSIXlt;
PAVdata$datetime <- strftime(as.POSIXlt(PAVdata$datetime,format="%Y-%m-%d %H:%M:%S"),format='%b')
june_Monthly_Avg <- ave(PAVdata$Z,PAVdata$datetime,FUN=mean)
head(june_Monthly_Avg)
[1] 9.341862 9.341862 9.341862 9.341862 9.341862 9.341862
And the whole thing consisted of the same numbers...
Think I'm most def using the commando very wrong.
But then I tried something simpler;
juni_Monthly_Avg <- ave(PAVdata$Z,PAVdata$datetime,FUN=mean)
head(juni_Monthly_Avg)
[1] 6.493529 4.035588 6.037692 3.987500 2.944615 2.860000
Confusing to me, seeing that for June I should only get ONE value.
So far I reckon grep looks most promising, but I can't seem to get my head around it.
Been trawling The R Book and the interwebs and of course SO for a couple days now and decided to cry out for help before I pull out all my hair (and before consulting my supervisors).
Thanks ^_^
> dput(head(PAVdata))
structure(list(X = 23:28, meanlat = c(58.3960117647059, 58.3961761764706,
58.3961507692308, 58.395558125, 58.3961726923077, 58.3962804166667
), meanlong = c(8.70895176470588, 8.71099294117647, 8.71091923076923,
8.71065375, 8.71136346153846, 8.71090416666667), meandepth = c(6.49352941176471,
4.03558823529412, 6.03769230769231, 3.9875, 2.94461538461538,
2.86), mindepth = c(5.72, 1.76, 3.08, 2.2, 2.2, 2.2), maxdepth = c(7.47,
7.47, 8.35, 6.16, 3.96, 3.52), depthrange = c(1.75, 5.71, 5.27,
3.96, 1.76, 1.32), rec = c(4L, 6L, 6L, 5L, 6L, 6L), numhits = c(17L,
34L, 26L, 16L, 26L, 24L), datetime = structure(1:6, .Label = c("2012-06-01 00:15:00", "2012-06-01 00:45:00",
"2012-06-01 01:15:00", "2012-06-01 01:45:00", "2012-06-01 02:15:00",
"2012-06-01 02:45:00", "2012-06-01 03:15:00", "2012-06-01 03:45:00",
"2012-06-01 04:15:00", "2012-06-01 04:45:00", "2012-06-01 05:45:00",
"2012-06-01 06:15:00", "2012-06-01 06:45:00", "2012-06-01 07:15:00",
"2012-06-01 07:45:00", "2012-06-01 08:15:00", "2012-06-01 08:45:00",
"2012-06-01 09:45:00", "2012-06-01 10:15:00", "2012-06-01 10:45:00",
"2012-06-01 11:15:00", "2012-06-01 11:45:00", "2012-06-01 12:15:00",
"2012-06-01 12:45:00", "2012-06-01 13:15:00", "2012-06-01 13:45:00",
"2012-06-01 14:45:00", "2012-06-01 15:15:00", "2012-06-01 15:45:00",
"2012-06-01 16:15:00", "2012-06-01 16:45:00", "2012-06-01 17:45:00",
"2012-06-01 18:15:00", "2012-06-01 18:45:00", "2012-06-01 19:15:00",
"2012-06-01 19:45:00", "2012-06-01 20:15:00", "2012-06-01 20:45:00",
"2012-06-01 21:15:00", "2012-06-01 22:15:00", "2012-06-01 22:45:00",
"2012-06-01 23:15:00", "2012-06-01 23:45:00", "2012-06-02 00:15:00",
"2012-06-02 00:45:00", "2012-06-02 01:15:00", "2012-06-02 01:45:00",
"2012-06-02 02:15:00", "2012-06-02 02:45:00", "2012-06-02 03:15:00",
"2012-06-02 03:45:00", "2012-06-02 04:15:00", "2012-06-02 04:45:00",
"2012-06-02 05:15:00", "2012-06-02 05:45:00", "2012-06-02 06:15:00",
"2012-06-02 06:45:00", "2012-06-02 07:15:00", "2012-06-02 07:45:00",
"2012-06-02 08:15:00", "2012-06-02 08:45:00", "2012-06-02 09:15:00",
"2012-06-02 09:45:00", "2012-06-02 10:15:00", "2012-06-02 10:45:00",
"2012-06-02 11:15:00", "2012-06-02 11:45:00", "2012-06-02 12:15:00",
"2012-06-02 13:15:00", "2012-06-02 13:45:00", "2012-06-02 14:15:00",
"2012-06-02 14:45:00", "2012-06-02 15:15:00", "2012-06-02 15:45:00",
"2012-06-02 16:15:00", "2012-06-02 16:45:00", "2012-06-02 17:15:00",
"2012-06-02 18:15:00", "2012-06-02 19:15:00", "2012-06-02 19:45:00",
"2012-06-02 20:15:00", "2012-06-02 20:45:00", "2012-06-02 21:15:00",
"2012-06-02 21:45:00", "2012-06-02 22:15:00", "2012-06-02 22:45:00",
"2012-06-02 23:15:00", "2012-06-02 23:45:00", "2012-06-03 00:15:00",
"2012-06-03 00:45:00", "2012-06-03 01:15:00", "2012-06-03 01:45:00",
"2012-06-03 02:15:00", "2012-06-03 02:45:00", "2012-06-03 03:15:00",
"2012-06-03 03:45:00", "2012-06-03 04:15:00", "2012-06-03 04:45:00",
"2012-06-03 05:15:00", "2012-06-03 05:45:00", "2012-06-03 06:15:00",
"2012-06-03 06:45:00", "2012-06-03 07:15:00", "2012-06-03 07:45:00",
"2012-06-03 08:15:00", "2012-06-03 08:45:00", "2012-06-03 09:15:00",
"2012-06-03 09:45:00", "2012-06-03 10:15:00", "2012-06-03 10:45:00",
"2012-06-03 11:15:00", "2012-06-03 11:45:00", "2012-06-03 12:15:00",
"2012-06-03 12:45:00", "2012-06-03 13:15:00", "2012-06-03 13:45:00",
"2012-06-03 14:15:00", "2012-06-03 15:45:00", "2012-06-03 16:15:00",
"2012-06-03 16:45:00", "2012-06-03 17:15:00", "2012-06-03 17:45:00",
"2012-06-03 18:15:00", "2012-06-03 18:45:00", "2012-06-03 19:15:00",
"2012-06-03 19:45:00", "2012-06-03 20:15:00", "2012-06-03 20:45:00",
"2012-06-03 21:15:00", "2012-06-03 21:45:00", "2012-06-03 22:15:00",
"2012-06-03 22:45:00", "2012-06-03 23:15:00", "2012-06-03 23:45:00",
"2012-06-04 00:15:00", "2012-06-04 00:45:00", "2012-06-04 01:15:00",
"2012-06-04 01:45:00", "2012-06-04 02:15:00", "2012-06-04 02:45:00",
"2012-06-04 03:15:00", "2012-06-04 03:45:00", "2012-06-04 04:45:00",
"2012-06-04 05:15:00", "2012-06-04 05:45:00", "2012-06-04 06:15:00",
"2012-06-04 06:45:00", "2012-06-04 07:15:00", "2012-06-04 07:45:00",
"2012-06-04 08:15:00", "2012-06-04 08:45:00", "2012-06-04 09:15:00",
"2012-06-04 09:45:00", "2012-06-04 10:15:00", "2012-06-04 10:45:00",
"2012-06-04 11:15:00", "2012-06-04 11:45:00", "2012-06-04 12:15:00",
"2012-06-04 12:45:00", "2012-06-04 13:15:00", "2012-06-04 13:45:00",
"2012-06-04 14:15:00", "2012-06-04 14:45:00", "2012-06-04 15:15:00",
"2012-06-04 15:45:00", "2012-06-04 16:15:00", "2012-06-04 16:45:00",
"2012-06-04 17:15:00", "2012-06-04 17:45:00", "2012-06-04 18:15:00",
"2012-06-04 18:45:00", "2012-06-04 19:15:00", "2012-06-04 19:45:00",
"2012-06-04 20:15:00", "2012-06-04 20:45:00", "2012-06-04 21:15:00",
"2012-06-04 21:45:00", "2012-06-04 22:15:00", "2012-06-04 22:45:00",
"2012-06-04 23:15:00", "2012-06-04 23:45:00", "2012-06-05 00:15:00",
"2012-06-05 00:45:00", "2012-06-05 01:15:00", "2012-06-05 01:45:00",
"2012-06-05 02:15:00", "2012-06-05 02:45:00", "2012-06-05 03:15:00",
"2012-06-05 03:45:00", "2012-06-05 04:15:00", "2012-06-05 04:45:00",
"2012-06-05 05:15:00", "2012-06-05 05:45:00", "2012-06-05 06:15:00",
"2012-06-05 06:45:00", "2012-06-05 07:15:00", "2012-06-05 07:45:00",
"2012-06-05 08:15:00", "2012-06-05 08:45:00", "2012-06-05 09:15:00",
"2012-06-05 09:45:00", "2012-06-05 10:15:00", "2012-06-05 10:45:00",
"2012-06-05 11:15:00", "2012-06-05 11:45:00", "2012-06-05 12:15:00",
"2012-06-05 12:45:00", "2012-06-05 13:15:00", "2012-06-05 13:45:00",
"2012-06-05 14:15:00", "2012-06-05 14:45:00", "2012-06-05 15:15:00",
"2012-06-05 15:45:00", "2012-06-05 16:15:00", "2012-06-05 16:45:00",
"2012-06-05 17:15:00", "2012-06-05 17:45:00", "2012-06-05 18:15:00",
"2012-06-05 18:45:00", "2012-06-05 19:15:00", "2012-06-05 19:45:00",
"2012-06-05 20:15:00", "2012-06-05 20:45:00", "2012-06-05 21:15:00",
"2012-06-05 21:45:00", "2012-06-05 22:15:00", "2012-06-05 22:45:00",
"2012-06-05 23:15:00", "2012-06-05 23:45:00", "2012-06-06 00:15:00",
"2012-06-06 00:45:00", "2012-06-06 01:15:00", "2012-06-06 01:45:00",
"2012-06-06 02:15:00", "2012-06-06 02:45:00", "2012-06-06 03:15:00",
"2012-06-06 03:45:00", "2012-06-06 04:15:00", "2012-06-06 04:45:00",
"2012-06-06 05:15:00", "2012-06-06 05:45:00", "2012-06-06 06:45:00",
"2012-06-06 07:15:00", "2012-06-06 07:45:00", "2012-06-06 08:15:00",
"2012-06-06 08:45:00", "2012-06-06 09:45:00", "2012-06-06 10:15:00",
"2012-06-06 10:45:00", "2012-06-06 11:15:00", "2012-06-06 11:45:00",
"2012-06-06 12:15:00", "2012-06-06 12:45:00", "2012-06-06 13:15:00",
"2012-06-06 13:45:00", "2012-06-06 14:15:00", "2012-06-06 14:45:00",
"2012-06-06 15:15:00", "2012-06-06 15:45:00", "2012-06-06 16:15:00",
"2012-06-06 16:45:00", "2012-06-06 17:15:00", "2012-06-06 17:45:00",
"2012-06-06 18:15:00", "2012-06-06 18:45:00", "2012-06-06 19:15:00",
"2012-06-06 19:45:00", "2012-06-06 20:15:00", "2012-06-06 20:45:00",
"2012-06-06 21:45:00", "2012-06-06 22:15:00", "2012-06-06 22:45:00",
"2012-06-06 23:15:00", "2012-06-06 23:45:00", "2012-06-07 00:15:00",
"2012-06-07 00:45:00", "2012-06-07 01:15:00", "2012-06-07 01:45:00",
"2012-06-07 02:15:00", "2012-06-07 02:45:00", "2012-06-07 03:15:00",
"2012-06-07 03:45:00", "2012-06-07 04:15:00", "2012-06-07 04:45:00",
"2012-06-07 05:15:00", "2012-06-07 05:45:00", "2012-06-07 06:15:00",
"2012-06-07 06:45:00", "2012-06-07 07:15:00", "2012-06-07 07:45:00",
"2012-06-07 08:15:00", "2012-06-07 08:45:00", "2012-06-07 09:15:00",
"2012-06-07 09:45:00", "2012-06-07 10:15:00", "2012-06-07 10:45:00",
"2012-06-07 11:15:00", "2012-06-07 11:45:00", "2012-06-07 12:15:00",
"2012-06-07 12:45:00", "2012-06-07 13:15:00", "2012-06-07 13:45:00",
"2012-06-07 14:15:00", "2012-06-07 14:45:00", "2012-06-07 15:15:00",
"2012-06-07 15:45:00", "2012-06-07 16:15:00", "2012-06-07 16:45:00",
"2012-06-07 17:15:00", "2012-06-07 17:45:00", "2012-06-07 18:15:00",
"2012-06-07 18:45:00", "2012-06-07 19:15:00", "2012-06-07 19:45:00",
"2012-06-07 20:15:00", "2012-06-07 20:45:00", "2012-06-07 21:15:00",
"2012-06-07 21:45:00", "2012-06-07 22:15:00", "2012-06-07 22:45:00",
"2012-06-07 23:15:00", "2012-06-07 23:45:00", "2012-06-08 00:15:00",
"2012-06-08 00:45:00", "2012-06-08 01:15:00", "2012-06-08 01:45:00",
"2012-06-08 02:15:00", "2012-06-08 02:45:00", "2012-06-08 03:15:00",
"2012-06-08 03:45:00", "2012-06-08 04:15:00", "2012-06-08 04:45:00",
"2012-06-08 05:15:00", "2012-06-08 05:45:00", "2012-06-08 06:15:00",
"2012-06-08 06:45:00", "2012-06-08 07:15:00", "2012-06-08 07:45:00",
"2012-06-08 08:15:00", "2012-06-08 08:45:00", "2012-06-08 09:15:00",
"2012-06-08 09:45:00", "2012-06-08 10:15:00", "2012-06-08 10:45:00",
"2012-06-08 11:15:00", "2012-06-08 11:45:00", "2012-06-08 12:15:00",
"2012-06-08 12:45:00", "2012-06-08 13:15:00", "2012-06-08 13:45:00",
"2012-06-08 14:15:00", "2012-06-08 14:45:00", "2012-06-08 15:15:00",
"2012-06-08 15:45:00", "2012-06-08 16:15:00", "2012-06-08 16:45:00",
"2012-06-08 17:15:00", "2012-06-08 17:45:00", "2012-06-08 18:15:00",
"2012-06-08 18:45:00", "2012-06-08 19:15:00", "2012-06-08 19:45:00",
"2012-06-08 20:15:00", "2012-06-08 20:45:00", "2012-06-08 21:15:00",
"2012-06-08 21:45:00", "2012-06-08 22:15:00", "2012-06-08 22:45:00",
"2012-06-08 23:15:00", "2012-06-08 23:45:00", "2012-06-09 00:15:00",
"2012-06-09 00:45:00", "2012-06-09 01:15:00", "2012-06-09 01:45:00",
"2012-06-09 02:15:00", "2012-06-09 02:45:00", "2012-06-09 03:15:00",
"2012-06-09 03:45:00", "2012-06-09 04:15:00", "2012-06-09 04:45:00",
"2012-06-09 05:15:00", "2012-06-09 05:45:00", "2012-06-09 06:45:00",
"2012-06-09 07:15:00", "2012-06-09 07:45:00", "2012-06-09 08:15:00",
"2012-06-09 08:45:00", "2012-06-09 09:15:00", "2012-06-09 09:45:00",
"2012-06-09 10:15:00", "2012-06-09 10:45:00", "2012-06-09 11:15:00",
"2012-06-09 11:45:00", "2012-06-09 12:15:00", "2012-06-09 12:45:00",
"2012-06-09 13:15:00", "2012-06-09 13:45:00", "2012-06-09 14:15:00",
"2012-06-09 14:45:00", "2012-06-09 15:15:00", "2012-06-09 15:45:00",
"2012-06-09 16:15:00", "2012-06-09 16:45:00", "2012-06-09 17:15:00",
"2012-06-09 17:45:00", "2012-06-09 18:15:00", "2012-06-09 18:45:00",
"2012-06-09 19:15:00", "2012-06-09 19:45:00", "2012-06-09 20:15:00",
"2012-06-09 20:45:00", "2012-06-09 21:15:00", "2012-06-09 21:45:00",
"2012-06-09 22:15:00", "2012-06-09 22:45:00", "2012-06-09 23:15:00",
"2012-06-09 23:45:00", "2012-06-10 00:15:00", "2012-06-10 00:45:00",
"2012-06-10 01:15:00", "2012-06-10 01:45:00", "2012-06-10 02:15:00",
"2012-06-10 02:45:00", "2012-06-10 03:15:00", "2012-06-10 03:45:00",
"2012-06-10 04:15:00", "2012-06-10 04:45:00", "2012-06-10 05:15:00",
"2012-06-10 05:45:00", "2012-06-10 06:15:00", "2012-06-10 06:45:00",
"2012-06-10 07:15:00", "2012-06-10 07:45:00", "2012-06-10 08:15:00",
"2012-06-10 08:45:00", "2012-06-10 09:15:00", "2012-06-10 09:45:00",
"2012-06-10 10:15:00", "2012-06-10 10:45:00", "2012-06-10 11:15:00",
"2012-06-10 11:45:00", "2012-06-10 12:15:00", "2012-06-10 12:45:00",
"2012-06-10 13:15:00", "2012-06-10 13:45:00", "2012-06-10 14:15:00",
"2012-06-10 14:45:00", "2012-06-10 15:15:00", "2012-06-10 15:45:00",
"2012-06-10 16:15:00", "2012-06-10 16:45:00", "2012-06-10 17:15:00",
"2012-06-10 17:45:00", "2012-06-10 18:15:00", "2012-06-10 18:45:00",
"2012-06-10 19:15:00", "2012-06-10 19:45:00", "2012-06-10 20:15:00",
"2012-06-10 20:45:00", "2012-06-10 21:15:00", "2012-06-10 21:45:00",
"2012-06-10 22:15:00", "2012-06-10 22:45:00", "2012-06-10 23:15:00",
"2012-06-10 23:45:00",
"2012-07-01 00:15:00", "2012-07-01 00:45:00", "2012-07-01 01:15:00",
"2012-07-01 01:45:00", "2012-07-01 02:15:00", "2012-07-01 02:45:00",
"2012-07-01 03:15:00", "2012-07-01 03:45:00", "2012-07-01 04:15:00",
"2012-07-01 04:45:00", "2012-07-01 05:15:00", "2012-07-01 05:45:00",
"2012-07-01 06:15:00", "2012-07-01 06:45:00", "2012-07-01 07:15:00",
"2012-07-01 07:45:00", "2012-07-01 08:15:00", "2012-07-01 08:45:00",
"2012-07-01 09:15:00", "2012-07-01 09:45:00", "2012-07-01 10:15:00",
"2012-07-01 10:45:00", "2012-07-01 11:15:00", "2012-07-01 11:45:00",
"2012-07-01 12:15:00", "2012-07-01 12:45:00", "2012-07-01 13:15:00",
"2012-07-01 13:45:00", "2012-07-01 14:15:00", "2012-07-01 14:45:00",
"2012-07-01 15:15:00", "2012-07-01 15:45:00", "2012-07-01 16:15:00",
"2012-07-01 16:45:00", "2012-07-01 17:15:00", "2012-07-01 17:45:00",
"2012-07-01 18:15:00", "2012-07-01 18:45:00", "2012-07-01 19:15:00",
"2012-07-01 19:45:00", "2012-07-01 20:15:00", "2012-07-01 20:45:00",
"2012-07-01 21:15:00", "2012-07-01 21:45:00", "2012-07-01 22:15:00",
"2012-07-01 22:45:00", "2012-07-01 23:15:00", "2012-07-01 23:45:00",
"2012-07-02 00:15:00", "2012-07-02 00:45:00", "2012-07-02 01:15:00",
"2012-07-02 01:45:00", "2012-07-02 02:15:00", "2012-07-02 02:45:00",
"2012-07-02 03:15:00", "2012-07-02 03:45:00", "2012-07-02 04:15:00",
"2012-07-02 04:45:00", "2012-07-02 05:15:00", "2012-07-02 05:45:00",
"2012-07-02 06:15:00", "2012-07-02 06:45:00", "2012-07-02 07:15:00",
"2012-07-02 07:45:00", "2012-07-02 08:15:00", "2012-07-02 08:45:00",
"2012-07-02 09:15:00", "2012-07-02 09:45:00", "2012-07-02 10:15:00",
"2012-07-02 10:45:00", "2012-07-02 11:15:00", "2012-07-02 11:45:00",
"2012-07-02 12:15:00", "2012-07-02 12:45:00", "2012-07-02 13:15:00",
"2012-07-02 13:45:00", "2012-07-02 14:15:00", "2012-07-02 14:45:00",
"2012-07-02 15:15:00", "2012-07-02 15:45:00", "2012-07-02 16:15:00",
"2012-07-02 16:45:00", "2012-07-02 17:15:00", "2012-07-02 17:45:00",
"2012-07-02 18:15:00", "2012-07-02 18:45:00", "2012-07-02 19:15:00",
"2012-07-02 19:45:00", "2012-07-02 20:15:00", "2012-07-02 20:45:00",
"2012-07-02 21:15:00", "2012-07-02 21:45:00", "2012-07-02 22:15:00",
"2012-07-02 22:45:00", "2012-07-02 23:15:00", "2012-07-02 23:45:00",
"2012-07-03 00:15:00", "2012-07-03 00:45:00", "2012-07-03 01:15:00",
"2012-07-03 01:45:00", "2012-07-03 02:15:00", "2012-07-03 02:45:00",
"2012-07-03 03:15:00", "2012-07-03 03:45:00", "2012-07-03 04:15:00",
"2012-07-03 04:45:00", "2012-07-03 05:15:00", "2012-07-03 05:45:00",
"2012-07-03 06:15:00", "2012-07-03 06:45:00", "2012-07-03 07:15:00",
"2012-07-03 07:45:00", "2012-07-03 08:15:00", "2012-07-03 08:45:00",
"2012-07-03 09:15:00", "2012-07-03 09:45:00", "2012-07-03 10:15:00",
"2012-07-03 10:45:00", "2012-07-03 11:15:00", "2012-07-03 11:45:00",
"2012-07-03 12:15:00", "2012-07-03 12:45:00", "2012-07-03 13:15:00",
"2012-07-03 13:45:00", "2012-07-03 14:15:00", "2012-07-03 15:15:00",
"2012-07-03 15:45:00", "2012-07-03 16:15:00", "2012-07-03 16:45:00",
"2012-07-03 17:15:00", "2012-07-03 17:45:00", "2012-07-03 18:15:00",
"2012-07-03 18:45:00", "2012-07-03 19:15:00", "2012-07-03 19:45:00",
"2012-07-03 20:15:00", "2012-07-03 20:45:00", "2012-07-03 21:15:00",
"2012-07-03 21:45:00", "2012-07-03 22:15:00", "2012-07-03 22:45:00",
"2012-07-03 23:15:00", "2012-07-03 23:45:00", "2012-07-04 00:15:00",
"2012-07-04 00:45:00", "2012-07-04 01:15:00", "2012-07-04 01:45:00",
"2012-07-04 02:15:00", "2012-07-04 02:45:00", "2012-07-04 03:15:00",
"2012-07-04 03:45:00", "2012-07-04 04:15:00", "2012-07-04 04:45:00",
"2012-07-04 05:15:00", "2012-07-04 05:45:00", "2012-07-04 06:15:00",
"2012-07-04 06:45:00", "2012-07-04 07:15:00", "2012-07-04 07:45:00",
"2012-07-04 08:45:00", "2012-07-04 09:15:00", "2012-07-04 09:45:00",
"2012-07-04 10:15:00", "2012-07-04 10:45:00", "2012-07-04 11:15:00",
"2012-07-04 11:45:00", "2012-07-04 12:15:00", "2012-07-04 12:45:00",
"2012-07-04 13:15:00", "2012-07-04 13:45:00", "2012-07-04 14:15:00",
"2012-07-04 14:45:00", "2012-07-04 15:15:00", "2012-07-04 15:45:00",
"2012-07-04 16:15:00", "2012-07-04 16:45:00", "2012-07-04 17:15:00",
"2012-07-04 17:45:00", "2012-07-04 18:15:00", "2012-07-04 18:45:00",
"2012-07-04 19:15:00", "2012-07-04 19:45:00", "2012-07-04 20:15:00",
"2012-07-04 20:45:00", "2012-07-04 21:15:00", "2012-07-04 21:45:00",
"2012-07-04 22:15:00", "2012-07-04 22:45:00", "2012-07-04 23:15:00",
"2012-07-04 23:45:00", "2012-07-05 00:15:00", "2012-07-05 00:45:00",
"2012-07-05 01:15:00", "2012-07-05 01:45:00", "2012-07-05 02:15:00",
"2012-07-05 02:45:00", "2012-07-05 03:15:00", "2012-07-05 03:45:00",
"2012-07-05 04:15:00", "2012-07-05 04:45:00", "2012-07-05 05:15:00",
"2012-07-05 05:45:00", "2012-07-05 06:15:00", "2012-07-05 06:45:00",
"2012-07-05 07:15:00", "2012-07-05 07:45:00", "2012-07-05 08:15:00",
"2012-07-05 08:45:00", "2012-07-05 09:15:00", "2012-07-05 09:45:00",
"2012-07-05 10:15:00", "2012-07-05 10:45:00", "2012-07-05 11:15:00",
"2012-07-05 11:45:00", "2012-07-05 12:15:00", "2012-07-05 12:45:00",
"2012-07-05 13:15:00", "2012-07-05 13:45:00", "2012-07-05 14:15:00",
"2012-07-05 14:45:00", "2012-07-05 15:15:00", "2012-07-05 15:45:00",
"2012-07-05 16:45:00", "2012-07-05 17:15:00", "2012-07-05 17:45:00",
"2012-07-05 18:15:00", "2012-07-05 18:45:00", "2012-07-05 19:15:00",
"2012-07-05 19:45:00", "2012-07-05 20:15:00", "2012-07-05 20:45:00",
"2012-07-05 21:15:00", "2012-07-05 21:45:00", "2012-07-05 22:15:00",
"2012-07-05 22:45:00", "2012-07-05 23:15:00", "2012-07-05 23:45:00",
"2012-07-06 00:15:00", "2012-07-06 00:45:00", "2012-07-06 01:15:00",
"2012-07-06 01:45:00", "2012-07-06 02:15:00", "2012-07-06 02:45:00",
"2012-07-06 03:15:00", "2012-07-06 03:45:00", "2012-07-06 04:15:00",
"2012-07-06 04:45:00", "2012-07-06 05:15:00", "2012-07-06 05:45:00",
"2012-07-06 06:15:00", "2012-07-06 06:45:00", "2012-07-06 07:15:00",
"2012-07-06 07:45:00", "2012-07-06 08:15:00", "2012-07-06 08:45:00",
"2012-07-06 09:15:00", "2012-07-06 09:45:00", "2012-07-06 10:15:00",
"2012-07-06 10:45:00", "2012-07-06 11:15:00", "2012-07-06 11:45:00",
"2012-07-06 12:15:00", "2012-07-06 12:45:00", "2012-07-06 13:15:00",
"2012-07-06 13:45:00", "2012-07-06 14:15:00", "2012-07-06 14:45:00",
"2012-07-06 15:15:00", "2012-07-06 15:45:00", "2012-07-06 16:15:00",
"2012-07-06 16:45:00", "2012-07-06 17:15:00", "2012-07-06 17:45:00",
"2012-07-06 18:15:00", "2012-07-06 18:45:00", "2012-07-06 19:15:00",
"2012-07-06 19:45:00", "2012-07-06 20:15:00", "2012-07-06 20:45:00",
"2012-07-06 21:15:00", "2012-07-06 21:45:00", "2012-07-06 22:15:00",
"2012-07-06 22:45:00", "2012-07-06 23:15:00", "2012-07-06 23:45:00",
"2012-07-07 00:15:00", "2012-07-07 00:45:00", "2012-07-07 01:15:00",
"2012-07-07 01:45:00", "2012-07-07 02:15:00", "2012-07-07 02:45:00",
"2012-07-07 03:15:00", "2012-07-07 03:45:00", "2012-07-07 04:15:00",
"2012-07-07 04:45:00", "2012-07-07 05:15:00", "2012-07-07 05:45:00",
"2012-07-07 06:15:00", "2012-07-07 06:45:00", "2012-07-07 07:15:00",
"2012-07-07 07:45:00", "2012-07-07 08:15:00", "2012-07-07 08:45:00",
"2012-07-07 09:15:00", "2012-07-07 09:45:00", "2012-07-07 10:15:00",
"2012-07-07 10:45:00", "2012-07-07 11:45:00", "2012-07-07 12:15:00",
"2012-07-07 12:45:00", "2012-07-07 13:15:00", "2012-07-07 13:45:00",
"2012-07-07 14:15:00", "2012-07-07 14:45:00", "2012-07-07 15:15:00",
"2012-07-07 15:45:00", "2012-07-07 16:15:00", "2012-07-07 16:45:00",
"2012-07-07 17:15:00", "2012-07-07 17:45:00", "2012-07-07 18:15:00",
"2012-07-07 18:45:00", "2012-07-07 19:15:00", "2012-07-07 19:45:00",
"2012-07-07 20:15:00", "2012-07-07 20:45:00", "2012-07-07 21:15:00",
"2012-07-07 21:45:00", "2012-07-07 22:15:00", "2012-07-07 22:45:00",
"2012-07-07 23:15:00", "2012-07-07 23:45:00", "2012-07-08 00:15:00",
"2012-07-08 00:45:00", "2012-07-08 01:15:00", "2012-07-08 01:45:00",
"2012-07-08 02:15:00", "2012-07-08 02:45:00", "2012-07-08 03:15:00",
"2012-07-08 03:45:00", "2012-07-08 04:15:00", "2012-07-08 04:45:00",
"2012-07-08 05:15:00", "2012-07-08 05:45:00", "2012-07-08 06:15:00",
"2012-07-08 06:45:00", "2012-07-08 07:45:00", "2012-07-08 08:15:00",
"2012-07-08 08:45:00", "2012-07-08 09:15:00", "2012-07-08 09:45:00",
"2012-07-08 10:15:00", "2012-07-08 10:45:00", "2012-07-08 11:15:00",
"2012-07-08 11:45:00", "2012-07-08 12:15:00", "2012-07-08 12:45:00",
"2012-07-08 13:15:00", "2012-07-08 13:45:00", "2012-07-08 14:15:00",
"2012-07-08 14:45:00", "2012-07-08 15:15:00", "2012-07-08 16:15:00",
"2012-07-08 16:45:00", "2012-07-08 17:15:00", "2012-07-08 17:45:00",
"2012-07-08 18:15:00", "2012-07-08 18:45:00", "2012-07-08 19:15:00",
"2012-07-08 19:45:00", "2012-07-08 20:15:00", "2012-07-08 20:45:00",
"2012-07-08 21:15:00", "2012-07-08 21:45:00", "2012-07-08 22:15:00",
"2012-07-08 22:45:00", "2012-07-08 23:15:00", "2012-07-08 23:45:00",
"2012-07-09 00:15:00", "2012-07-09 00:45:00", "2012-07-09 01:15:00",
"2012-07-09 01:45:00", "2012-07-09 02:15:00", "2012-07-09 02:45:00",
"2012-07-09 03:15:00", "2012-07-09 03:45:00", "2012-07-09 04:15:00",
"2012-07-09 04:45:00", "2012-07-09 05:15:00", "2012-07-09 05:45:00",
"2012-07-09 06:15:00", "2012-07-09 06:45:00", "2012-07-09 07:15:00",
"2012-07-09 07:45:00", "2012-07-09 08:15:00", "2012-07-09 08:45:00",
"2012-07-09 09:15:00", "2012-07-09 09:45:00", "2012-07-09 10:15:00",
"2012-07-09 10:45:00", "2012-07-09 11:15:00", "2012-07-09 11:45:00",
"2012-07-09 12:15:00", "2012-07-09 12:45:00", "2012-07-09 13:15:00",
"2012-07-09 13:45:00", "2012-07-09 14:15:00", "2012-07-09 14:45:00",
"2012-07-09 15:15:00", "2012-07-09 15:45:00", "2012-07-09 16:15:00",
"2012-07-09 16:45:00", "2012-07-09 17:15:00", "2012-07-09 17:45:00",
"2012-07-09 18:15:00", "2012-07-09 18:45:00", "2012-07-09 19:15:00",
"2012-07-09 19:45:00", "2012-07-09 20:15:00", "2012-07-09 21:15:00",
"2012-07-09 21:45:00", "2012-07-09 22:15:00", "2012-07-09 22:45:00",
"2012-07-09 23:15:00", "2012-07-09 23:45:00", "2012-07-10 00:15:00",
"2012-07-10 00:45:00", "2012-07-10 01:15:00", "2012-07-10 01:45:00",
"2012-07-10 02:15:00", "2012-07-10 02:45:00", "2012-07-10 03:15:00",
"2012-07-10 03:45:00", "2012-07-10 04:15:00", "2012-07-10 04:45:00",
"2012-07-10 05:15:00", "2012-07-10 05:45:00", "2012-07-10 06:15:00",
"2012-07-10 06:45:00", "2012-07-10 07:15:00", "2012-07-10 07:45:00",
"2012-07-10 08:15:00", "2012-07-10 08:45:00", "2012-07-10 09:15:00",
"2012-07-10 09:45:00", "2012-07-10 10:15:00", "2012-07-10 10:45:00",
"2012-07-10 11:15:00", "2012-07-10 11:45:00", "2012-07-10 12:15:00",
"2012-07-10 12:45:00", "2012-07-10 13:15:00", "2012-07-10 13:45:00",
"2012-07-10 14:15:00", "2012-07-10 14:45:00", "2012-07-10 15:15:00",
"2012-07-10 15:45:00", "2012-07-10 16:15:00", "2012-07-10 17:15:00",
"2012-07-10 17:45:00", "2012-07-10 18:15:00", "2012-07-10 18:45:00",
"2012-07-10 19:15:00", "2012-07-10 19:45:00", "2012-07-10 20:15:00",
"2012-07-10 20:45:00", "2012-07-10 21:15:00", "2012-07-10 21:45:00",
"2012-07-10 22:15:00", "2012-07-10 22:45:00", "2012-07-10 23:15:00",
"2012-07-10 23:45:00"), class = "factor"), mon = c(1L, 1L, 1L,
1L, 1L, 1L), X.1 = c(482986.747312444, 483106.142664631, 483101.821796104,
483086.019405256, 483127.799620954, 483101.003261731), Y = c(6472837.55691427,
6472855.34802117, 6472852.53770682, 6472786.61976395, 6472854.86710904,
6472866.97636095), Z = c(6.49352941176471, 4.03558823529412,
6.03769230769231, 3.9875, 2.94461538461538, 2.86)), .Names = c("X",
"meanlat", "meanlong", "meandepth", "mindepth", "maxdepth", "depthrange",
"rec", "numhits", "datetime", "mon", "X.1", "Y", "Z"), row.names = c(NA,
6L), class = "data.frame")
>
You look that You tray many things but you don't try one thing in a good way.
For example using POSIXlt, dat here is your PAVdata object
ll <- as.POSIXlt(dat$datetime)
dat$hours <- ll$hour ## I create a variable hour
dat$days <- ll$mday ## I create a variable day
dat$months <- ll$mon ## I create a variable month
Now i can use this variable to average by group.
here a solution using plyr package.
ddply(dat,.(months,days),summarise, m = mean(meandepth)) ## I group by months and days
months days m
1 4 11 4.393154
> ddply(dat,.(months,days,hours),summarise, m = mean(meandepth))
months days hours m
1 4 11 11 5.264558
2 4 11 12 5.012596
3 4 11 13 2.902308
Whatever you use, try to use one thing in a good way. Generally there are many method to do such things in R.
Related
How to get records related to highest sessioncount in a given day
I have a App Insights table like below expected output is : need to select the processDate and its related fields which has highest aggregate value of sessionCount in that day (yellow highlighted rows are the expected result) let da = datatable(id:int, processDate:datetime, message:string,col:string, sessionCount:int) [ 1,"2021-03-03 12:00:00", "a","aa",2, 1,"2021-03-03 12:00:00", "a","aa2",8, 1,"2021-03-03 09:00:00", "g","gg",20, 1,"2021-03-03 09:00:00", "g","g1",3, 1,"2021-03-03 15:00:00", "b","bb",9, 1,"2021-03-03 15:00:00", "b","bb1",1, 2,"2021-03-07 21:00:00", "f","ff",6, 2,"2021-03-07 21:00:00", "f","ff",2, 2,"2021-03-07 21:00:00", "abc","faf",21, 2,"2021-03-07 22:00:00", "abc","fav",25, 2,"2021-03-07 22:00:00", "z","zz",9 ]; da | summarize maxsessionCountperRun = sum(sessionCount) by processDate ; expected out is below
let da = datatable(id:int, processDate:datetime, message:string,col:string, sessionCount:int) [ 1,"2021-03-03 12:00:00", "a","aa",2, 1,"2021-03-03 12:00:00", "a","aa2",8, 1,"2021-03-03 09:00:00", "g","gg",20, 1,"2021-03-03 09:00:00", "g","g1",3, 1,"2021-03-03 15:00:00", "b","bb",9, 1,"2021-03-03 15:00:00", "b","bb1",1, 2,"2021-03-07 21:00:00", "f","ff",6, 2,"2021-03-07 21:00:00", "f","ff",2, 2,"2021-03-07 21:00:00", "abc","faf",21, 2,"2021-03-07 22:00:00", "abc","fav",25, 2,"2021-03-07 22:00:00", "z","zz",9 ]; da | summarize sum(sessionCount) by processDate | summarize arg_max(sum_sessionCount, processDate) by processDate_day = bin(processDate,1d) | project-away processDate_day sum_sessionCount processDate 23 2021-03-03T09:00:00Z 34 2021-03-07T22:00:00Z Fiddle
Plot timelines without date
I have a carbon dioxide sensor that captures the concentration during working time. Now I'd like to create a line plot that shows one line for each working day over time (from 8 am to 6 pm). Some sample data: co2data <- data.frame( dateTime = c( "2021-08-18 08:00:00", "2021-08-18 09:00:00", "2021-08-18 10:00:00", "2021-08-18 11:00:00", "2021-08-18 12:00:00", "2021-08-18 13:00:00", "2021-08-18 14:00:00", "2021-08-18 15:00:00", "2021-08-18 16:00:00", "2021-08-18 17:00:00", "2021-08-18 18:00:00", "2021-08-19 08:00:00", "2021-08-19 09:00:00", "2021-08-19 10:00:00", "2021-08-19 11:00:00", "2021-08-19 12:00:00", "2021-08-19 13:00:00", "2021-08-19 14:00:00", "2021-08-19 15:00:00", "2021-08-19 16:00:00", "2021-08-19 17:00:00", "2021-08-19 18:00:00" ), ppm = c( 400, 450, 456, 560, 670, 690, 810, 900, 600, 650, 700, 410, 470, 480, 590, 700, 710, 810, 900, 1010, 1000, 1100 ) ) Now I can plot the concentration over time, but I don't know how to plot times only on the x-axis (no dates). co2data <- co2data %>% mutate(dateTime = as.POSIXct(dateTime)) co2data %>% ggplot(aes(x = dateTime, y = ppm)) + geom_line() + labs(title = "CO2-Concentration", y = "CO2-concentration ppm", x = "Time") How can I plot one line for each day?
With the help of the data.table package (or lubridate) you can extract temporal information form Date/Time strings. require(data.table) require(ggplot2) setDT(co2data) co2data[ , time := hour(as.ITime(dateTime)) ] co2data[ , yday := as.factor(yday(as.IDate(dateTime))) ] ggplot(co2data, aes(x = time, y = ppm, col = yday)) + geom_line() + labs(title = "CO2-Concentration", y = "CO2-concentration ppm", x = "Time") + theme_bw()
How to establish if the dates in a column are unique?
I have a simple question to ask. I couldn't find a solution on SO. I have a column vector of dates and I would like to see whether months are unique or not. I tried to use unique but I may have used it in the wrong way. An example: Date = structure(c(11690, 11725, 11753, 11781, 11809, 11844, 11872, 11900, 11942, 11970, 11998, 12026, 12061, 12089, 12117, 12145, 12180, 12208, 12243, 12264, 12265, 12299, 12327, 12362, 12390, 12425, 12453, 12481, 12509, 12544, 12572, 12600, 12635, 12663, 12698, 12726, 12754, 12796, 12817, 12845, 12880, 12907, 12936, 12971, 12999, 13027, 13062, 13090, 13118, 13160, 13181, 13209, 13244, 13272, 13307, 13335, 13363, 13392, 13426, 13454, 13489, 13524, 13552, 13580, 13615, 13643, 13670, 13699, 13727, 13762, 13790, 13825, 13853, 13888, 13916, 13944, 13979, 14007, 14035, 14063, 14098, 14126, 14154, 14160, 14189, 14217, 14259, 14280, 14308, 14336, 14371, 14399, 14427, 14462, 14490, 14525, 14553, 14581, 14623, 14644, 14672, 14707, 14735, 14770, 14798, 14826, 14854, 14889, 14917, 14945, 14987, 15008, 15036, 15071, 15099, 15134, 15162, 15190, 15225, 15253, 15281, 15316, 15351, 15379, 15407, 15434, 15463, 15497, 15526, 15554, 15589, 15617, 15652, 15680, 15715, 15743, 15771, 15799, 15827, 15862, 15890, 15918, 15953, 15980, 16016, 16044, 16079, 16107, 16135, 16163, 16198, 16226, 16254, 16289, 16317, 16345, 16380, 16408, 16457, 16467, 16499, 16540, 16556, 16589, 16632, 16648, 16681, 16730, 16740, 16772, 16821, 16832, 16870, 16912, 16922, 16954, 17003, 17014, 17052, 17094, 17106, 17143, 17185, 17198, 17234, 17283, 17287, 17325, 17367, 17379, 17416, 17465, 17471, 17514, 17556, 17563, 17598, 17647, 17652, 17696, 17738, 17744, 17787, 17829, 17836, 17878, 17920, 17928, 17962, 17996, 18017, 18053, 18102, 18109, 18151, 18193, 18201, 18242), class = "Date") In this column vector I would like to see whether there are two observations in the same month (there are 2 for "2003-07" and "2008-10"). I can I check it with one line of command? Thanks!
In base R, we can format the Date to get only year and month, use table to count their occurrences, Filter to select only those month-year which occur more than once. names(Filter(function(x) x > 1, table(format(Date, "%Y-%m")))) #[1] "2003-07" "2008-10" Same logic using zoo::as.yearmon. names(Filter(function(x) x > 1, table(zoo::as.yearmon(Date)))) #[1] "Jul 2003" "Oct 2008"
library('lubridate')
library('tidyverse')
Date %>%
enframe() %>%
count(year(value), month(value)) %>%
filter(n > 1)
One line with as.yearmon from zoo library(zoo) Date[which(duplicated(as.yearmon(Date))==TRUE)] [1] "2003-07-31" "2008-10-08"
Use str_sub to get year-month and use table to count frequency of occurence: library(tidyverse) year_month <- str_sub(Date, 1, 7) #this will extract characters 1-7 (year-mo) result <- as_tibble(table(year_month)) #or pipe it all year_month2 <- str_sub(Date, 1, 7) %>% table() %>% as_tibble() #or filter to get only those with > 1 occurence year_month3 <- str_sub(Date, 1, 7) %>% table() %>% as_tibble() %>% filter(n > 1)
Rolling join in data.table is using wrong time zone
I am trying to join two data tables using a rolling join. My time/date stamps are POSIXct specified as eastern standard time zone (EST). During the join, values using GMT are instead retrieved, i.e., the value from EST - 5 hours. Additionally, it's making corrections for daylight savings time, retrieving values from EST - 4 hours during daylight savings time.
I've tried using fuzzyjoin and that works correctly but suffers from being very slow even for small joins and runs out of resources for larger problems (I'm guessing because it's doing a full cartesian join then filtering) so won't run my real job. I've looked through stackoverlow and I haven't seen this problem described
In the example below, the flow data is stream flow measured at 15-minute intervals (in cubic meters per second) and the chem data are results from analyzing grab samples. What I want is to find the value of stream flow (Q.cms) that is closest in time to the time the grab sample was collected. In the result, Q.cms retrieved from the flow data should match flow.EST and not flow.GMT (flow.EST and flow.GMT were obtained using the vlookup function in Excel). You can look at the resulting table or use the ggplot to visualize the results. The black symbols should fall on the 1:1 line not the red symbols. Some of this follows the examples given here https://r-norberg.blogspot.com/2016/06/understanding-datatable-rolling-joins.html. In the roll statement, I've also tried "-Inf" and "T" - neither help.
library(tidyverse)
library(data.table)
NHchem <- tibble(SampleDateTime = list("2012-03-07 15:15:00",
"2012-03-07 07:57:00",
"2012-03-07 09:00:00",
"2012-03-07 08:31:00",
"2012-03-07 08:00:00"),
Parameter = rep("HgD", 5),
Value = c(11.83, 9.09, 8.98, 10.91, 10.91),
flow.EST = c(1.734292, 1.730299, 1.711665, 1.721448, 1.729299),
flow.GMT = c(1.729082, 1.855115, 1.818062, 1.838623, 1.855263)
)
NHchem$SampleDateTime <- lubridate::ymd_hms(NHchem$SampleDateTime, tz = "EST")
NHflow <- tibble(DateTime = list("2012-03-07 00:00:00", "2012-03-07 00:15:00", "2012-03-07 00:30:00",
"2012-03-07 00:45:00", "2012-03-07 01:00:00", "2012-03-07 01:15:00",
"2012-03-07 01:30:00", "2012-03-07 01:45:00", "2012-03-07 02:00:00",
"2012-03-07 02:15:00", "2012-03-07 02:30:00", "2012-03-07 02:45:00",
"2012-03-07 03:00:00", "2012-03-07 03:15:00", "2012-03-07 03:30:00",
"2012-03-07 03:45:00", "2012-03-07 04:00:00", "2012-03-07 04:15:00",
"2012-03-07 04:30:00", "2012-03-07 04:45:00", "2012-03-07 05:00:00",
"2012-03-07 05:15:00", "2012-03-07 05:30:00", "2012-03-07 05:45:00",
"2012-03-07 06:00:00", "2012-03-07 06:15:00", "2012-03-07 06:30:00",
"2012-03-07 06:45:00", "2012-03-07 07:00:00", "2012-03-07 07:15:00",
"2012-03-07 07:30:00", "2012-03-07 07:45:00", "2012-03-07 08:00:00",
"2012-03-07 08:15:00", "2012-03-07 08:30:00", "2012-03-07 08:45:00",
"2012-03-07 09:00:00", "2012-03-07 09:15:00", "2012-03-07 09:30:00",
"2012-03-07 09:45:00", "2012-03-07 10:00:00", "2012-03-07 10:15:00",
"2012-03-07 10:30:00", "2012-03-07 10:45:00", "2012-03-07 11:00:00",
"2012-03-07 11:15:00", "2012-03-07 11:30:00", "2012-03-07 11:45:00",
"2012-03-07 12:00:00", "2012-03-07 12:15:00", "2012-03-07 12:30:00",
"2012-03-07 12:45:00", "2012-03-07 13:00:00", "2012-03-07 13:15:00",
"2012-03-07 13:30:00", "2012-03-07 13:45:00", "2012-03-07 14:00:00",
"2012-03-07 14:15:00", "2012-03-07 14:30:00", "2012-03-07 14:45:00",
"2012-03-07 15:00:00", "2012-03-07 15:15:00", "2012-03-07 15:30:00",
"2012-03-07 15:45:00", "2012-03-07 16:00:00", "2012-03-07 16:15:00",
"2012-03-07 16:30:00", "2012-03-07 16:45:00", "2012-03-07 17:00:00",
"2012-03-07 17:15:00", "2012-03-07 17:30:00", "2012-03-07 17:45:00",
"2012-03-07 18:00:00", "2012-03-07 18:15:00", "2012-03-07 18:30:00",
"2012-03-07 18:45:00", "2012-03-07 19:00:00", "2012-03-07 19:15:00",
"2012-03-07 19:30:00", "2012-03-07 19:45:00", "2012-03-07 20:00:00",
"2012-03-07 20:15:00", "2012-03-07 20:30:00", "2012-03-07 20:45:00",
"2012-03-07 21:00:00", "2012-03-07 21:15:00", "2012-03-07 21:30:00",
"2012-03-07 21:45:00", "2012-03-07 22:00:00", "2012-03-07 22:15:00",
"2012-03-07 22:30:00", "2012-03-07 22:45:00", "2012-03-07 23:00:00",
"2012-03-07 23:15:00", "2012-03-07 23:30:00", "2012-03-07 23:45:00"
),
Q.cms = c(1.9183428976162, 1.90884554590853, 1.90659614969849, 1.89946370922413,
1.89667381849492, 1.88823268972675, 1.89009845845496, 1.88925133710857,
1.89175644143891, 1.88902902166044, 1.88199493586431, 1.87318875971247,
1.86917038183388, 1.86071830324102, 1.85932613686066, 1.86093401873381,
1.87101369024615, 1.87296981545807, 1.8762632040545, 1.86625922525504,
1.86445655629997, 1.86091440792692, 1.86261797589624, 1.85849714690774,
1.85809949483732, 1.85365423835426, 1.86769219811402, 1.87137568188775,
1.86957715377645, 1.86178854932717, 1.85842153505713, 1.85511511281169,
1.85526345829864, 1.84606992374992, 1.8386232796469, 1.8306698161104,
1.81806190831676, 1.80972662643993, 1.80671560966769, 1.79692071214778,
1.79246415779217, 1.78383758090983, 1.77911151556738, 1.77540849330392,
1.76977546791765, 1.76704647349273, 1.75506646810193, 1.75116656853259,
1.74116032441497, 1.73694852237565, 1.73539800459245, 1.73029929958069,
1.72929873358336, 1.72145096214292, 1.72144822481096, 1.7137286373663,
1.71166508485562, 1.72512015058183, 1.72801058727519, 1.71946126373971,
1.72908219540057, 1.73003063754833, 1.73433620637326, 1.74704929729279,
1.75374511203315, 1.75751451176466, 1.74550984385788, 1.7374425791972,
1.73028285052801, 1.73164553811319, 1.71538821542136, 1.71111583998635,
1.69850042893639, 1.7018112620745, 1.70068743638687, 1.70624871061042,
1.70513766128531, 1.70189855875322, 1.71388172504149, 1.70687667156756,
1.72965507980508, 1.73429231190258, 1.73062829040751, 1.74161628160761,
1.74502884750137, 1.74764319440737, 1.76775550519358, 1.74926846992512,
1.75406991794209, 1.75302950118107, 1.75219840078475, 1.75821138471585,
1.74749746409036, 1.75047602988317, 1.76192575356191, 1.76360735465132)
)
NHflow$DateTime <- lubridate::ymd_hms(NHflow$DateTime, tz = "EST")
NHflowDT <- setDT(NHflow)
NHchemDT <- setDT(NHchem)
NHflowDT[, join_time := DateTime]
NHchemDT[, join_time := SampleDateTime]
setkey(NHflowDT, join_time)
setkey(NHchemDT, join_time)
NHflow_chem <- NHflowDT[ NHchemDT, roll = 'nearest']
NHflow_chem
ggplot(data = NHflow_chem, mapping = aes(x = flow.EST, y = Q.cms)) +
geom_point(size = 4) +
geom_point(aes(x = flow.GMT, y = Q.cms), color = "red", size = 4) +
geom_abline(slope = 1, intercept = 0) +
labs(x = "flow.EST (black), flow.GMT (red)")
I'm not sure what your exact problem is, but the rolling update join below seems to work fine when I look at your sample data sets...
NHflowDT <- setDT(NHflow)
NHchemDT <- setDT(NHchem)
NHchemDT[, c("nearest_flow_time", "Q.cms") :=
NHflowDT[NHchemDT, .(x.DateTime, x.Q.cms),
on = .(DateTime = SampleDateTime),
roll = "nearest"] ][]
output
SampleDateTime Parameter Value flow.EST flow.GMT nearest_flow_time Q.cms
1: 2012-03-07 15:15:00 HgD 11.83 1.734292 1.729082 2012-03-07 15:15:00 1.730031
2: 2012-03-07 07:57:00 HgD 9.09 1.730299 1.855115 2012-03-07 08:00:00 1.855263
3: 2012-03-07 09:00:00 HgD 8.98 1.711665 1.818062 2012-03-07 09:00:00 1.818062
4: 2012-03-07 08:31:00 HgD 10.91 1.721448 1.838623 2012-03-07 08:30:00 1.838623
5: 2012-03-07 08:00:00 HgD 10.91 1.729299 1.855263 2012-03-07 08:00:00 1.855263
> attributes( NHchemDT$SampleDateTime)
$class
[1] "POSIXct" "POSIXt"
$tzone
[1] "EST"
> attributes( NHchemDT$nearest_flow_time)
$class
[1] "POSIXct" "POSIXt"
$tzone
[1] "EST"
How to change the x-values displayed in ggplot and ggplotly in R
I have a timeseries X, with related timestamps, and i want to provide a graph with the hourly values. The X-axis should not show the timestamp but rather the hour, however the plot should be generated with the timestamp as the x-axis
I've already tried to plot the X with the related timestamps (hours from 0-23) using the scale_x_datetime function. However the problem arises when trying to get the x-values to show 1-24. A problem arises when you go over midnight and end up with duplicated x-values
Timestamp <- c("2019-07-30 23:00:00", "2019-07-31 00:00:00", "2019-07-31 01:00:00", "2019-07-31 02:00:00", "2019-07-31 03:00:00", "2019-07-31 04:00:00", "2019-07-31 05:00:00", "2019-07-31 06:00:00",
"2019-07-31 07:00:00", "2019-07-31 08:00:00", "2019-07-31 09:00:00", "2019-07-31 10:00:00", "2019-07-31 11:00:00", "2019-07-31 12:00:00",
"2019-07-31 13:00:00", "2019-07-31 14:00:00", "2019-07-31 15:00:00", "2019-07-31 16:00:00", "2019-07-31 17:00:00", "2019-07-31 18:00:00",
"2019-07-31 19:00:00", "2019-07-31 20:00:00", "2019-07-31 21:00:00", "2019-07-31 22:00:00", "2019-07-31 23:00:00", "2019-08-01 00:00:00","2019-08-01 01:00:00")
col <- c(110,100,105,100,105,100,110,100,110,100,110,100,110,100,110,100,
110,100,110,100,110,100,110,105,110,105,110)
hour <- c(23,24,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,1)
Timestamp <- as.POSIXct(Timestamp, tz = "GMT")
library(plotly)
library(lubridate)
library(scales)
library(ggplot2)
data <- data.frame(Timestamp,col,hour)
data$Timestamp <- as.POSIXct(data$Timestamp)
ggplot(data = data, aes(x = Timestamp))+
geom_line(aes(y = col))+
scale_x_datetime(date_breaks = "1 hours",
date_minor_breaks = "1 hour",
labels = date_format("%H"))
The desired output is a ggplot/ggplotly with the hours displayed as the x-axis (1-24 and not 0-23).
Future work would also a timeseries with 1 minute resolution to be included in the plot
edited: added working code. I want the x-axis to be 1-24 and start over at 1 after 24
Here i found the answer. It was an evolution and addition to the scale_x_datetime function
Timestamp <- c("2019-07-30 23:00:00", "2019-07-31 00:00:00", "2019-07-31 01:00:00", "2019-07-31 02:00:00", "2019-07-31 03:00:00", "2019-07-31 04:00:00", "2019-07-31 05:00:00", "2019-07-31 06:00:00",
"2019-07-31 07:00:00", "2019-07-31 08:00:00", "2019-07-31 09:00:00", "2019-07-31 10:00:00", "2019-07-31 11:00:00", "2019-07-31 12:00:00",
"2019-07-31 13:00:00", "2019-07-31 14:00:00", "2019-07-31 15:00:00", "2019-07-31 16:00:00", "2019-07-31 17:00:00", "2019-07-31 18:00:00",
"2019-07-31 19:00:00", "2019-07-31 20:00:00", "2019-07-31 21:00:00", "2019-07-31 22:00:00", "2019-07-31 23:00:00", "2019-08-01 00:00:00","2019-08-01 01:00:00")
col <- c(110,100,105,100,105,100,110,100,110,100,110,100,110,100,110,100,
110,100,110,100,110,100,110,105,110,105,110)
hour <- c(23,24,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,1)
Timestamp <- as.POSIXct(Timestamp, tz = "GMT")
library(plotly)
library(lubridate)
library(scales)
library(ggplot2)
data <- data.frame(Timestamp,col,hour)
data$Timestamp <- as.POSIXct(data$Timestamp)
ggplot(data = data, aes(x = Timestamp))+
geom_line(aes(y = col))+
scale_x_datetime(date_breaks = "1 hours",
date_minor_breaks = "1 hour",
labels = function(x) ifelse(hour(as.POSIXct(x, origin = '1970-01-01'))==0, 24, hour(as.POSIXct(x, origin = '1970-01-01'))))