I am trying to join two data tables using a rolling join. My time/date stamps are POSIXct specified as eastern standard time zone (EST). During the join, values using GMT are instead retrieved, i.e., the value from EST - 5 hours. Additionally, it's making corrections for daylight savings time, retrieving values from EST - 4 hours during daylight savings time.
I've tried using fuzzyjoin and that works correctly but suffers from being very slow even for small joins and runs out of resources for larger problems (I'm guessing because it's doing a full cartesian join then filtering) so won't run my real job. I've looked through stackoverlow and I haven't seen this problem described
In the example below, the flow data is stream flow measured at 15-minute intervals (in cubic meters per second) and the chem data are results from analyzing grab samples. What I want is to find the value of stream flow (Q.cms) that is closest in time to the time the grab sample was collected. In the result, Q.cms retrieved from the flow data should match flow.EST and not flow.GMT (flow.EST and flow.GMT were obtained using the vlookup function in Excel). You can look at the resulting table or use the ggplot to visualize the results. The black symbols should fall on the 1:1 line not the red symbols. Some of this follows the examples given here https://r-norberg.blogspot.com/2016/06/understanding-datatable-rolling-joins.html. In the roll statement, I've also tried "-Inf" and "T" - neither help.
library(tidyverse)
library(data.table)
NHchem <- tibble(SampleDateTime = list("2012-03-07 15:15:00",
"2012-03-07 07:57:00",
"2012-03-07 09:00:00",
"2012-03-07 08:31:00",
"2012-03-07 08:00:00"),
Parameter = rep("HgD", 5),
Value = c(11.83, 9.09, 8.98, 10.91, 10.91),
flow.EST = c(1.734292, 1.730299, 1.711665, 1.721448, 1.729299),
flow.GMT = c(1.729082, 1.855115, 1.818062, 1.838623, 1.855263)
)
NHchem$SampleDateTime <- lubridate::ymd_hms(NHchem$SampleDateTime, tz = "EST")
NHflow <- tibble(DateTime = list("2012-03-07 00:00:00", "2012-03-07 00:15:00", "2012-03-07 00:30:00",
"2012-03-07 00:45:00", "2012-03-07 01:00:00", "2012-03-07 01:15:00",
"2012-03-07 01:30:00", "2012-03-07 01:45:00", "2012-03-07 02:00:00",
"2012-03-07 02:15:00", "2012-03-07 02:30:00", "2012-03-07 02:45:00",
"2012-03-07 03:00:00", "2012-03-07 03:15:00", "2012-03-07 03:30:00",
"2012-03-07 03:45:00", "2012-03-07 04:00:00", "2012-03-07 04:15:00",
"2012-03-07 04:30:00", "2012-03-07 04:45:00", "2012-03-07 05:00:00",
"2012-03-07 05:15:00", "2012-03-07 05:30:00", "2012-03-07 05:45:00",
"2012-03-07 06:00:00", "2012-03-07 06:15:00", "2012-03-07 06:30:00",
"2012-03-07 06:45:00", "2012-03-07 07:00:00", "2012-03-07 07:15:00",
"2012-03-07 07:30:00", "2012-03-07 07:45:00", "2012-03-07 08:00:00",
"2012-03-07 08:15:00", "2012-03-07 08:30:00", "2012-03-07 08:45:00",
"2012-03-07 09:00:00", "2012-03-07 09:15:00", "2012-03-07 09:30:00",
"2012-03-07 09:45:00", "2012-03-07 10:00:00", "2012-03-07 10:15:00",
"2012-03-07 10:30:00", "2012-03-07 10:45:00", "2012-03-07 11:00:00",
"2012-03-07 11:15:00", "2012-03-07 11:30:00", "2012-03-07 11:45:00",
"2012-03-07 12:00:00", "2012-03-07 12:15:00", "2012-03-07 12:30:00",
"2012-03-07 12:45:00", "2012-03-07 13:00:00", "2012-03-07 13:15:00",
"2012-03-07 13:30:00", "2012-03-07 13:45:00", "2012-03-07 14:00:00",
"2012-03-07 14:15:00", "2012-03-07 14:30:00", "2012-03-07 14:45:00",
"2012-03-07 15:00:00", "2012-03-07 15:15:00", "2012-03-07 15:30:00",
"2012-03-07 15:45:00", "2012-03-07 16:00:00", "2012-03-07 16:15:00",
"2012-03-07 16:30:00", "2012-03-07 16:45:00", "2012-03-07 17:00:00",
"2012-03-07 17:15:00", "2012-03-07 17:30:00", "2012-03-07 17:45:00",
"2012-03-07 18:00:00", "2012-03-07 18:15:00", "2012-03-07 18:30:00",
"2012-03-07 18:45:00", "2012-03-07 19:00:00", "2012-03-07 19:15:00",
"2012-03-07 19:30:00", "2012-03-07 19:45:00", "2012-03-07 20:00:00",
"2012-03-07 20:15:00", "2012-03-07 20:30:00", "2012-03-07 20:45:00",
"2012-03-07 21:00:00", "2012-03-07 21:15:00", "2012-03-07 21:30:00",
"2012-03-07 21:45:00", "2012-03-07 22:00:00", "2012-03-07 22:15:00",
"2012-03-07 22:30:00", "2012-03-07 22:45:00", "2012-03-07 23:00:00",
"2012-03-07 23:15:00", "2012-03-07 23:30:00", "2012-03-07 23:45:00"
),
Q.cms = c(1.9183428976162, 1.90884554590853, 1.90659614969849, 1.89946370922413,
1.89667381849492, 1.88823268972675, 1.89009845845496, 1.88925133710857,
1.89175644143891, 1.88902902166044, 1.88199493586431, 1.87318875971247,
1.86917038183388, 1.86071830324102, 1.85932613686066, 1.86093401873381,
1.87101369024615, 1.87296981545807, 1.8762632040545, 1.86625922525504,
1.86445655629997, 1.86091440792692, 1.86261797589624, 1.85849714690774,
1.85809949483732, 1.85365423835426, 1.86769219811402, 1.87137568188775,
1.86957715377645, 1.86178854932717, 1.85842153505713, 1.85511511281169,
1.85526345829864, 1.84606992374992, 1.8386232796469, 1.8306698161104,
1.81806190831676, 1.80972662643993, 1.80671560966769, 1.79692071214778,
1.79246415779217, 1.78383758090983, 1.77911151556738, 1.77540849330392,
1.76977546791765, 1.76704647349273, 1.75506646810193, 1.75116656853259,
1.74116032441497, 1.73694852237565, 1.73539800459245, 1.73029929958069,
1.72929873358336, 1.72145096214292, 1.72144822481096, 1.7137286373663,
1.71166508485562, 1.72512015058183, 1.72801058727519, 1.71946126373971,
1.72908219540057, 1.73003063754833, 1.73433620637326, 1.74704929729279,
1.75374511203315, 1.75751451176466, 1.74550984385788, 1.7374425791972,
1.73028285052801, 1.73164553811319, 1.71538821542136, 1.71111583998635,
1.69850042893639, 1.7018112620745, 1.70068743638687, 1.70624871061042,
1.70513766128531, 1.70189855875322, 1.71388172504149, 1.70687667156756,
1.72965507980508, 1.73429231190258, 1.73062829040751, 1.74161628160761,
1.74502884750137, 1.74764319440737, 1.76775550519358, 1.74926846992512,
1.75406991794209, 1.75302950118107, 1.75219840078475, 1.75821138471585,
1.74749746409036, 1.75047602988317, 1.76192575356191, 1.76360735465132)
)
NHflow$DateTime <- lubridate::ymd_hms(NHflow$DateTime, tz = "EST")
NHflowDT <- setDT(NHflow)
NHchemDT <- setDT(NHchem)
NHflowDT[, join_time := DateTime]
NHchemDT[, join_time := SampleDateTime]
setkey(NHflowDT, join_time)
setkey(NHchemDT, join_time)
NHflow_chem <- NHflowDT[ NHchemDT, roll = 'nearest']
NHflow_chem
ggplot(data = NHflow_chem, mapping = aes(x = flow.EST, y = Q.cms)) +
geom_point(size = 4) +
geom_point(aes(x = flow.GMT, y = Q.cms), color = "red", size = 4) +
geom_abline(slope = 1, intercept = 0) +
labs(x = "flow.EST (black), flow.GMT (red)")
I'm not sure what your exact problem is, but the rolling update join below seems to work fine when I look at your sample data sets...
NHflowDT <- setDT(NHflow)
NHchemDT <- setDT(NHchem)
NHchemDT[, c("nearest_flow_time", "Q.cms") :=
NHflowDT[NHchemDT, .(x.DateTime, x.Q.cms),
on = .(DateTime = SampleDateTime),
roll = "nearest"] ][]
output
SampleDateTime Parameter Value flow.EST flow.GMT nearest_flow_time Q.cms
1: 2012-03-07 15:15:00 HgD 11.83 1.734292 1.729082 2012-03-07 15:15:00 1.730031
2: 2012-03-07 07:57:00 HgD 9.09 1.730299 1.855115 2012-03-07 08:00:00 1.855263
3: 2012-03-07 09:00:00 HgD 8.98 1.711665 1.818062 2012-03-07 09:00:00 1.818062
4: 2012-03-07 08:31:00 HgD 10.91 1.721448 1.838623 2012-03-07 08:30:00 1.838623
5: 2012-03-07 08:00:00 HgD 10.91 1.729299 1.855263 2012-03-07 08:00:00 1.855263
> attributes( NHchemDT$SampleDateTime)
$class
[1] "POSIXct" "POSIXt"
$tzone
[1] "EST"
> attributes( NHchemDT$nearest_flow_time)
$class
[1] "POSIXct" "POSIXt"
$tzone
[1] "EST"
In advance I have to say that I'm truly sorry for poor formatting! I just can not seem to get a hang of it...
I am also an R noob.
So,
I have a 80 files containing information on 80 individual fish's movements;
> head(PAVdata)
X meanlat meanlong meandepth mindepth maxdepth depthrange rec numhits datetime mon X.1 Y Z
1 23 58.39601 8.708952 6.493529 5.72 7.47 1.75 4 17 2012-05-11 11:15:00 1 482986.7 6472838 6.493529
2 24 58.39618 8.710993 4.035588 1.76 7.47 5.71 6 34 2012-05-11 11:45:00 1 483106.1 6472855 4.035588
3 25 58.39615 8.710919 6.037692 3.08 8.35 5.27 6 26 2012-05-11 12:15:00 1 483101.8 6472853 6.037692
4 26 58.39556 8.710654 3.987500 2.20 6.16 3.96 5 16 2012-05-11 12:45:00 1 483086.0 6472787 3.987500
5 27 58.39617 8.711363 2.944615 2.20 3.96 1.76 6 26 2012-05-11 13:15:00 1 483127.8 6472855 2.944615
6 28 58.39628 8.710904 2.860000 2.20 3.52 1.32 6 24 2012-05-11 13:45:00 1 483101.0 6472867 2.860000
But the columns in interest is datetime and Z, where Z is depth.
What I am aiming to do is;
Get a daily and monthly average of depth
Get a daily and monthly min & max (look at amplitude)
What I've tried so far is;
june<-c(PAVdata$datetime[grep("^2012-07",as.character(PAVdata$datetime))])
Which yield something thats completely farfetched to me
head(june)
[1] 2390 2391 2392 2393 2394 2395
I was reading about subset and aggregate and POSIXlt and tapply and zoo and wanted to try these, but I seemed to get the error that there was an length diff ..
Heres an attempt with the POSIXlt;
PAVdata$datetime <- strftime(as.POSIXlt(PAVdata$datetime,format="%Y-%m-%d %H:%M:%S"),format='%b')
june_Monthly_Avg <- ave(PAVdata$Z,PAVdata$datetime,FUN=mean)
head(june_Monthly_Avg)
[1] 9.341862 9.341862 9.341862 9.341862 9.341862 9.341862
And the whole thing consisted of the same numbers...
Think I'm most def using the commando very wrong.
But then I tried something simpler;
juni_Monthly_Avg <- ave(PAVdata$Z,PAVdata$datetime,FUN=mean)
head(juni_Monthly_Avg)
[1] 6.493529 4.035588 6.037692 3.987500 2.944615 2.860000
Confusing to me, seeing that for June I should only get ONE value.
So far I reckon grep looks most promising, but I can't seem to get my head around it.
Been trawling The R Book and the interwebs and of course SO for a couple days now and decided to cry out for help before I pull out all my hair (and before consulting my supervisors).
Thanks ^_^
> dput(head(PAVdata))
structure(list(X = 23:28, meanlat = c(58.3960117647059, 58.3961761764706,
58.3961507692308, 58.395558125, 58.3961726923077, 58.3962804166667
), meanlong = c(8.70895176470588, 8.71099294117647, 8.71091923076923,
8.71065375, 8.71136346153846, 8.71090416666667), meandepth = c(6.49352941176471,
4.03558823529412, 6.03769230769231, 3.9875, 2.94461538461538,
2.86), mindepth = c(5.72, 1.76, 3.08, 2.2, 2.2, 2.2), maxdepth = c(7.47,
7.47, 8.35, 6.16, 3.96, 3.52), depthrange = c(1.75, 5.71, 5.27,
3.96, 1.76, 1.32), rec = c(4L, 6L, 6L, 5L, 6L, 6L), numhits = c(17L,
34L, 26L, 16L, 26L, 24L), datetime = structure(1:6, .Label = c("2012-06-01 00:15:00", "2012-06-01 00:45:00",
"2012-06-01 01:15:00", "2012-06-01 01:45:00", "2012-06-01 02:15:00",
"2012-06-01 02:45:00", "2012-06-01 03:15:00", "2012-06-01 03:45:00",
"2012-06-01 04:15:00", "2012-06-01 04:45:00", "2012-06-01 05:45:00",
"2012-06-01 06:15:00", "2012-06-01 06:45:00", "2012-06-01 07:15:00",
"2012-06-01 07:45:00", "2012-06-01 08:15:00", "2012-06-01 08:45:00",
"2012-06-01 09:45:00", "2012-06-01 10:15:00", "2012-06-01 10:45:00",
"2012-06-01 11:15:00", "2012-06-01 11:45:00", "2012-06-01 12:15:00",
"2012-06-01 12:45:00", "2012-06-01 13:15:00", "2012-06-01 13:45:00",
"2012-06-01 14:45:00", "2012-06-01 15:15:00", "2012-06-01 15:45:00",
"2012-06-01 16:15:00", "2012-06-01 16:45:00", "2012-06-01 17:45:00",
"2012-06-01 18:15:00", "2012-06-01 18:45:00", "2012-06-01 19:15:00",
"2012-06-01 19:45:00", "2012-06-01 20:15:00", "2012-06-01 20:45:00",
"2012-06-01 21:15:00", "2012-06-01 22:15:00", "2012-06-01 22:45:00",
"2012-06-01 23:15:00", "2012-06-01 23:45:00", "2012-06-02 00:15:00",
"2012-06-02 00:45:00", "2012-06-02 01:15:00", "2012-06-02 01:45:00",
"2012-06-02 02:15:00", "2012-06-02 02:45:00", "2012-06-02 03:15:00",
"2012-06-02 03:45:00", "2012-06-02 04:15:00", "2012-06-02 04:45:00",
"2012-06-02 05:15:00", "2012-06-02 05:45:00", "2012-06-02 06:15:00",
"2012-06-02 06:45:00", "2012-06-02 07:15:00", "2012-06-02 07:45:00",
"2012-06-02 08:15:00", "2012-06-02 08:45:00", "2012-06-02 09:15:00",
"2012-06-02 09:45:00", "2012-06-02 10:15:00", "2012-06-02 10:45:00",
"2012-06-02 11:15:00", "2012-06-02 11:45:00", "2012-06-02 12:15:00",
"2012-06-02 13:15:00", "2012-06-02 13:45:00", "2012-06-02 14:15:00",
"2012-06-02 14:45:00", "2012-06-02 15:15:00", "2012-06-02 15:45:00",
"2012-06-02 16:15:00", "2012-06-02 16:45:00", "2012-06-02 17:15:00",
"2012-06-02 18:15:00", "2012-06-02 19:15:00", "2012-06-02 19:45:00",
"2012-06-02 20:15:00", "2012-06-02 20:45:00", "2012-06-02 21:15:00",
"2012-06-02 21:45:00", "2012-06-02 22:15:00", "2012-06-02 22:45:00",
"2012-06-02 23:15:00", "2012-06-02 23:45:00", "2012-06-03 00:15:00",
"2012-06-03 00:45:00", "2012-06-03 01:15:00", "2012-06-03 01:45:00",
"2012-06-03 02:15:00", "2012-06-03 02:45:00", "2012-06-03 03:15:00",
"2012-06-03 03:45:00", "2012-06-03 04:15:00", "2012-06-03 04:45:00",
"2012-06-03 05:15:00", "2012-06-03 05:45:00", "2012-06-03 06:15:00",
"2012-06-03 06:45:00", "2012-06-03 07:15:00", "2012-06-03 07:45:00",
"2012-06-03 08:15:00", "2012-06-03 08:45:00", "2012-06-03 09:15:00",
"2012-06-03 09:45:00", "2012-06-03 10:15:00", "2012-06-03 10:45:00",
"2012-06-03 11:15:00", "2012-06-03 11:45:00", "2012-06-03 12:15:00",
"2012-06-03 12:45:00", "2012-06-03 13:15:00", "2012-06-03 13:45:00",
"2012-06-03 14:15:00", "2012-06-03 15:45:00", "2012-06-03 16:15:00",
"2012-06-03 16:45:00", "2012-06-03 17:15:00", "2012-06-03 17:45:00",
"2012-06-03 18:15:00", "2012-06-03 18:45:00", "2012-06-03 19:15:00",
"2012-06-03 19:45:00", "2012-06-03 20:15:00", "2012-06-03 20:45:00",
"2012-06-03 21:15:00", "2012-06-03 21:45:00", "2012-06-03 22:15:00",
"2012-06-03 22:45:00", "2012-06-03 23:15:00", "2012-06-03 23:45:00",
"2012-06-04 00:15:00", "2012-06-04 00:45:00", "2012-06-04 01:15:00",
"2012-06-04 01:45:00", "2012-06-04 02:15:00", "2012-06-04 02:45:00",
"2012-06-04 03:15:00", "2012-06-04 03:45:00", "2012-06-04 04:45:00",
"2012-06-04 05:15:00", "2012-06-04 05:45:00", "2012-06-04 06:15:00",
"2012-06-04 06:45:00", "2012-06-04 07:15:00", "2012-06-04 07:45:00",
"2012-06-04 08:15:00", "2012-06-04 08:45:00", "2012-06-04 09:15:00",
"2012-06-04 09:45:00", "2012-06-04 10:15:00", "2012-06-04 10:45:00",
"2012-06-04 11:15:00", "2012-06-04 11:45:00", "2012-06-04 12:15:00",
"2012-06-04 12:45:00", "2012-06-04 13:15:00", "2012-06-04 13:45:00",
"2012-06-04 14:15:00", "2012-06-04 14:45:00", "2012-06-04 15:15:00",
"2012-06-04 15:45:00", "2012-06-04 16:15:00", "2012-06-04 16:45:00",
"2012-06-04 17:15:00", "2012-06-04 17:45:00", "2012-06-04 18:15:00",
"2012-06-04 18:45:00", "2012-06-04 19:15:00", "2012-06-04 19:45:00",
"2012-06-04 20:15:00", "2012-06-04 20:45:00", "2012-06-04 21:15:00",
"2012-06-04 21:45:00", "2012-06-04 22:15:00", "2012-06-04 22:45:00",
"2012-06-04 23:15:00", "2012-06-04 23:45:00", "2012-06-05 00:15:00",
"2012-06-05 00:45:00", "2012-06-05 01:15:00", "2012-06-05 01:45:00",
"2012-06-05 02:15:00", "2012-06-05 02:45:00", "2012-06-05 03:15:00",
"2012-06-05 03:45:00", "2012-06-05 04:15:00", "2012-06-05 04:45:00",
"2012-06-05 05:15:00", "2012-06-05 05:45:00", "2012-06-05 06:15:00",
"2012-06-05 06:45:00", "2012-06-05 07:15:00", "2012-06-05 07:45:00",
"2012-06-05 08:15:00", "2012-06-05 08:45:00", "2012-06-05 09:15:00",
"2012-06-05 09:45:00", "2012-06-05 10:15:00", "2012-06-05 10:45:00",
"2012-06-05 11:15:00", "2012-06-05 11:45:00", "2012-06-05 12:15:00",
"2012-06-05 12:45:00", "2012-06-05 13:15:00", "2012-06-05 13:45:00",
"2012-06-05 14:15:00", "2012-06-05 14:45:00", "2012-06-05 15:15:00",
"2012-06-05 15:45:00", "2012-06-05 16:15:00", "2012-06-05 16:45:00",
"2012-06-05 17:15:00", "2012-06-05 17:45:00", "2012-06-05 18:15:00",
"2012-06-05 18:45:00", "2012-06-05 19:15:00", "2012-06-05 19:45:00",
"2012-06-05 20:15:00", "2012-06-05 20:45:00", "2012-06-05 21:15:00",
"2012-06-05 21:45:00", "2012-06-05 22:15:00", "2012-06-05 22:45:00",
"2012-06-05 23:15:00", "2012-06-05 23:45:00", "2012-06-06 00:15:00",
"2012-06-06 00:45:00", "2012-06-06 01:15:00", "2012-06-06 01:45:00",
"2012-06-06 02:15:00", "2012-06-06 02:45:00", "2012-06-06 03:15:00",
"2012-06-06 03:45:00", "2012-06-06 04:15:00", "2012-06-06 04:45:00",
"2012-06-06 05:15:00", "2012-06-06 05:45:00", "2012-06-06 06:45:00",
"2012-06-06 07:15:00", "2012-06-06 07:45:00", "2012-06-06 08:15:00",
"2012-06-06 08:45:00", "2012-06-06 09:45:00", "2012-06-06 10:15:00",
"2012-06-06 10:45:00", "2012-06-06 11:15:00", "2012-06-06 11:45:00",
"2012-06-06 12:15:00", "2012-06-06 12:45:00", "2012-06-06 13:15:00",
"2012-06-06 13:45:00", "2012-06-06 14:15:00", "2012-06-06 14:45:00",
"2012-06-06 15:15:00", "2012-06-06 15:45:00", "2012-06-06 16:15:00",
"2012-06-06 16:45:00", "2012-06-06 17:15:00", "2012-06-06 17:45:00",
"2012-06-06 18:15:00", "2012-06-06 18:45:00", "2012-06-06 19:15:00",
"2012-06-06 19:45:00", "2012-06-06 20:15:00", "2012-06-06 20:45:00",
"2012-06-06 21:45:00", "2012-06-06 22:15:00", "2012-06-06 22:45:00",
"2012-06-06 23:15:00", "2012-06-06 23:45:00", "2012-06-07 00:15:00",
"2012-06-07 00:45:00", "2012-06-07 01:15:00", "2012-06-07 01:45:00",
"2012-06-07 02:15:00", "2012-06-07 02:45:00", "2012-06-07 03:15:00",
"2012-06-07 03:45:00", "2012-06-07 04:15:00", "2012-06-07 04:45:00",
"2012-06-07 05:15:00", "2012-06-07 05:45:00", "2012-06-07 06:15:00",
"2012-06-07 06:45:00", "2012-06-07 07:15:00", "2012-06-07 07:45:00",
"2012-06-07 08:15:00", "2012-06-07 08:45:00", "2012-06-07 09:15:00",
"2012-06-07 09:45:00", "2012-06-07 10:15:00", "2012-06-07 10:45:00",
"2012-06-07 11:15:00", "2012-06-07 11:45:00", "2012-06-07 12:15:00",
"2012-06-07 12:45:00", "2012-06-07 13:15:00", "2012-06-07 13:45:00",
"2012-06-07 14:15:00", "2012-06-07 14:45:00", "2012-06-07 15:15:00",
"2012-06-07 15:45:00", "2012-06-07 16:15:00", "2012-06-07 16:45:00",
"2012-06-07 17:15:00", "2012-06-07 17:45:00", "2012-06-07 18:15:00",
"2012-06-07 18:45:00", "2012-06-07 19:15:00", "2012-06-07 19:45:00",
"2012-06-07 20:15:00", "2012-06-07 20:45:00", "2012-06-07 21:15:00",
"2012-06-07 21:45:00", "2012-06-07 22:15:00", "2012-06-07 22:45:00",
"2012-06-07 23:15:00", "2012-06-07 23:45:00", "2012-06-08 00:15:00",
"2012-06-08 00:45:00", "2012-06-08 01:15:00", "2012-06-08 01:45:00",
"2012-06-08 02:15:00", "2012-06-08 02:45:00", "2012-06-08 03:15:00",
"2012-06-08 03:45:00", "2012-06-08 04:15:00", "2012-06-08 04:45:00",
"2012-06-08 05:15:00", "2012-06-08 05:45:00", "2012-06-08 06:15:00",
"2012-06-08 06:45:00", "2012-06-08 07:15:00", "2012-06-08 07:45:00",
"2012-06-08 08:15:00", "2012-06-08 08:45:00", "2012-06-08 09:15:00",
"2012-06-08 09:45:00", "2012-06-08 10:15:00", "2012-06-08 10:45:00",
"2012-06-08 11:15:00", "2012-06-08 11:45:00", "2012-06-08 12:15:00",
"2012-06-08 12:45:00", "2012-06-08 13:15:00", "2012-06-08 13:45:00",
"2012-06-08 14:15:00", "2012-06-08 14:45:00", "2012-06-08 15:15:00",
"2012-06-08 15:45:00", "2012-06-08 16:15:00", "2012-06-08 16:45:00",
"2012-06-08 17:15:00", "2012-06-08 17:45:00", "2012-06-08 18:15:00",
"2012-06-08 18:45:00", "2012-06-08 19:15:00", "2012-06-08 19:45:00",
"2012-06-08 20:15:00", "2012-06-08 20:45:00", "2012-06-08 21:15:00",
"2012-06-08 21:45:00", "2012-06-08 22:15:00", "2012-06-08 22:45:00",
"2012-06-08 23:15:00", "2012-06-08 23:45:00", "2012-06-09 00:15:00",
"2012-06-09 00:45:00", "2012-06-09 01:15:00", "2012-06-09 01:45:00",
"2012-06-09 02:15:00", "2012-06-09 02:45:00", "2012-06-09 03:15:00",
"2012-06-09 03:45:00", "2012-06-09 04:15:00", "2012-06-09 04:45:00",
"2012-06-09 05:15:00", "2012-06-09 05:45:00", "2012-06-09 06:45:00",
"2012-06-09 07:15:00", "2012-06-09 07:45:00", "2012-06-09 08:15:00",
"2012-06-09 08:45:00", "2012-06-09 09:15:00", "2012-06-09 09:45:00",
"2012-06-09 10:15:00", "2012-06-09 10:45:00", "2012-06-09 11:15:00",
"2012-06-09 11:45:00", "2012-06-09 12:15:00", "2012-06-09 12:45:00",
"2012-06-09 13:15:00", "2012-06-09 13:45:00", "2012-06-09 14:15:00",
"2012-06-09 14:45:00", "2012-06-09 15:15:00", "2012-06-09 15:45:00",
"2012-06-09 16:15:00", "2012-06-09 16:45:00", "2012-06-09 17:15:00",
"2012-06-09 17:45:00", "2012-06-09 18:15:00", "2012-06-09 18:45:00",
"2012-06-09 19:15:00", "2012-06-09 19:45:00", "2012-06-09 20:15:00",
"2012-06-09 20:45:00", "2012-06-09 21:15:00", "2012-06-09 21:45:00",
"2012-06-09 22:15:00", "2012-06-09 22:45:00", "2012-06-09 23:15:00",
"2012-06-09 23:45:00", "2012-06-10 00:15:00", "2012-06-10 00:45:00",
"2012-06-10 01:15:00", "2012-06-10 01:45:00", "2012-06-10 02:15:00",
"2012-06-10 02:45:00", "2012-06-10 03:15:00", "2012-06-10 03:45:00",
"2012-06-10 04:15:00", "2012-06-10 04:45:00", "2012-06-10 05:15:00",
"2012-06-10 05:45:00", "2012-06-10 06:15:00", "2012-06-10 06:45:00",
"2012-06-10 07:15:00", "2012-06-10 07:45:00", "2012-06-10 08:15:00",
"2012-06-10 08:45:00", "2012-06-10 09:15:00", "2012-06-10 09:45:00",
"2012-06-10 10:15:00", "2012-06-10 10:45:00", "2012-06-10 11:15:00",
"2012-06-10 11:45:00", "2012-06-10 12:15:00", "2012-06-10 12:45:00",
"2012-06-10 13:15:00", "2012-06-10 13:45:00", "2012-06-10 14:15:00",
"2012-06-10 14:45:00", "2012-06-10 15:15:00", "2012-06-10 15:45:00",
"2012-06-10 16:15:00", "2012-06-10 16:45:00", "2012-06-10 17:15:00",
"2012-06-10 17:45:00", "2012-06-10 18:15:00", "2012-06-10 18:45:00",
"2012-06-10 19:15:00", "2012-06-10 19:45:00", "2012-06-10 20:15:00",
"2012-06-10 20:45:00", "2012-06-10 21:15:00", "2012-06-10 21:45:00",
"2012-06-10 22:15:00", "2012-06-10 22:45:00", "2012-06-10 23:15:00",
"2012-06-10 23:45:00",
"2012-07-01 00:15:00", "2012-07-01 00:45:00", "2012-07-01 01:15:00",
"2012-07-01 01:45:00", "2012-07-01 02:15:00", "2012-07-01 02:45:00",
"2012-07-01 03:15:00", "2012-07-01 03:45:00", "2012-07-01 04:15:00",
"2012-07-01 04:45:00", "2012-07-01 05:15:00", "2012-07-01 05:45:00",
"2012-07-01 06:15:00", "2012-07-01 06:45:00", "2012-07-01 07:15:00",
"2012-07-01 07:45:00", "2012-07-01 08:15:00", "2012-07-01 08:45:00",
"2012-07-01 09:15:00", "2012-07-01 09:45:00", "2012-07-01 10:15:00",
"2012-07-01 10:45:00", "2012-07-01 11:15:00", "2012-07-01 11:45:00",
"2012-07-01 12:15:00", "2012-07-01 12:45:00", "2012-07-01 13:15:00",
"2012-07-01 13:45:00", "2012-07-01 14:15:00", "2012-07-01 14:45:00",
"2012-07-01 15:15:00", "2012-07-01 15:45:00", "2012-07-01 16:15:00",
"2012-07-01 16:45:00", "2012-07-01 17:15:00", "2012-07-01 17:45:00",
"2012-07-01 18:15:00", "2012-07-01 18:45:00", "2012-07-01 19:15:00",
"2012-07-01 19:45:00", "2012-07-01 20:15:00", "2012-07-01 20:45:00",
"2012-07-01 21:15:00", "2012-07-01 21:45:00", "2012-07-01 22:15:00",
"2012-07-01 22:45:00", "2012-07-01 23:15:00", "2012-07-01 23:45:00",
"2012-07-02 00:15:00", "2012-07-02 00:45:00", "2012-07-02 01:15:00",
"2012-07-02 01:45:00", "2012-07-02 02:15:00", "2012-07-02 02:45:00",
"2012-07-02 03:15:00", "2012-07-02 03:45:00", "2012-07-02 04:15:00",
"2012-07-02 04:45:00", "2012-07-02 05:15:00", "2012-07-02 05:45:00",
"2012-07-02 06:15:00", "2012-07-02 06:45:00", "2012-07-02 07:15:00",
"2012-07-02 07:45:00", "2012-07-02 08:15:00", "2012-07-02 08:45:00",
"2012-07-02 09:15:00", "2012-07-02 09:45:00", "2012-07-02 10:15:00",
"2012-07-02 10:45:00", "2012-07-02 11:15:00", "2012-07-02 11:45:00",
"2012-07-02 12:15:00", "2012-07-02 12:45:00", "2012-07-02 13:15:00",
"2012-07-02 13:45:00", "2012-07-02 14:15:00", "2012-07-02 14:45:00",
"2012-07-02 15:15:00", "2012-07-02 15:45:00", "2012-07-02 16:15:00",
"2012-07-02 16:45:00", "2012-07-02 17:15:00", "2012-07-02 17:45:00",
"2012-07-02 18:15:00", "2012-07-02 18:45:00", "2012-07-02 19:15:00",
"2012-07-02 19:45:00", "2012-07-02 20:15:00", "2012-07-02 20:45:00",
"2012-07-02 21:15:00", "2012-07-02 21:45:00", "2012-07-02 22:15:00",
"2012-07-02 22:45:00", "2012-07-02 23:15:00", "2012-07-02 23:45:00",
"2012-07-03 00:15:00", "2012-07-03 00:45:00", "2012-07-03 01:15:00",
"2012-07-03 01:45:00", "2012-07-03 02:15:00", "2012-07-03 02:45:00",
"2012-07-03 03:15:00", "2012-07-03 03:45:00", "2012-07-03 04:15:00",
"2012-07-03 04:45:00", "2012-07-03 05:15:00", "2012-07-03 05:45:00",
"2012-07-03 06:15:00", "2012-07-03 06:45:00", "2012-07-03 07:15:00",
"2012-07-03 07:45:00", "2012-07-03 08:15:00", "2012-07-03 08:45:00",
"2012-07-03 09:15:00", "2012-07-03 09:45:00", "2012-07-03 10:15:00",
"2012-07-03 10:45:00", "2012-07-03 11:15:00", "2012-07-03 11:45:00",
"2012-07-03 12:15:00", "2012-07-03 12:45:00", "2012-07-03 13:15:00",
"2012-07-03 13:45:00", "2012-07-03 14:15:00", "2012-07-03 15:15:00",
"2012-07-03 15:45:00", "2012-07-03 16:15:00", "2012-07-03 16:45:00",
"2012-07-03 17:15:00", "2012-07-03 17:45:00", "2012-07-03 18:15:00",
"2012-07-03 18:45:00", "2012-07-03 19:15:00", "2012-07-03 19:45:00",
"2012-07-03 20:15:00", "2012-07-03 20:45:00", "2012-07-03 21:15:00",
"2012-07-03 21:45:00", "2012-07-03 22:15:00", "2012-07-03 22:45:00",
"2012-07-03 23:15:00", "2012-07-03 23:45:00", "2012-07-04 00:15:00",
"2012-07-04 00:45:00", "2012-07-04 01:15:00", "2012-07-04 01:45:00",
"2012-07-04 02:15:00", "2012-07-04 02:45:00", "2012-07-04 03:15:00",
"2012-07-04 03:45:00", "2012-07-04 04:15:00", "2012-07-04 04:45:00",
"2012-07-04 05:15:00", "2012-07-04 05:45:00", "2012-07-04 06:15:00",
"2012-07-04 06:45:00", "2012-07-04 07:15:00", "2012-07-04 07:45:00",
"2012-07-04 08:45:00", "2012-07-04 09:15:00", "2012-07-04 09:45:00",
"2012-07-04 10:15:00", "2012-07-04 10:45:00", "2012-07-04 11:15:00",
"2012-07-04 11:45:00", "2012-07-04 12:15:00", "2012-07-04 12:45:00",
"2012-07-04 13:15:00", "2012-07-04 13:45:00", "2012-07-04 14:15:00",
"2012-07-04 14:45:00", "2012-07-04 15:15:00", "2012-07-04 15:45:00",
"2012-07-04 16:15:00", "2012-07-04 16:45:00", "2012-07-04 17:15:00",
"2012-07-04 17:45:00", "2012-07-04 18:15:00", "2012-07-04 18:45:00",
"2012-07-04 19:15:00", "2012-07-04 19:45:00", "2012-07-04 20:15:00",
"2012-07-04 20:45:00", "2012-07-04 21:15:00", "2012-07-04 21:45:00",
"2012-07-04 22:15:00", "2012-07-04 22:45:00", "2012-07-04 23:15:00",
"2012-07-04 23:45:00", "2012-07-05 00:15:00", "2012-07-05 00:45:00",
"2012-07-05 01:15:00", "2012-07-05 01:45:00", "2012-07-05 02:15:00",
"2012-07-05 02:45:00", "2012-07-05 03:15:00", "2012-07-05 03:45:00",
"2012-07-05 04:15:00", "2012-07-05 04:45:00", "2012-07-05 05:15:00",
"2012-07-05 05:45:00", "2012-07-05 06:15:00", "2012-07-05 06:45:00",
"2012-07-05 07:15:00", "2012-07-05 07:45:00", "2012-07-05 08:15:00",
"2012-07-05 08:45:00", "2012-07-05 09:15:00", "2012-07-05 09:45:00",
"2012-07-05 10:15:00", "2012-07-05 10:45:00", "2012-07-05 11:15:00",
"2012-07-05 11:45:00", "2012-07-05 12:15:00", "2012-07-05 12:45:00",
"2012-07-05 13:15:00", "2012-07-05 13:45:00", "2012-07-05 14:15:00",
"2012-07-05 14:45:00", "2012-07-05 15:15:00", "2012-07-05 15:45:00",
"2012-07-05 16:45:00", "2012-07-05 17:15:00", "2012-07-05 17:45:00",
"2012-07-05 18:15:00", "2012-07-05 18:45:00", "2012-07-05 19:15:00",
"2012-07-05 19:45:00", "2012-07-05 20:15:00", "2012-07-05 20:45:00",
"2012-07-05 21:15:00", "2012-07-05 21:45:00", "2012-07-05 22:15:00",
"2012-07-05 22:45:00", "2012-07-05 23:15:00", "2012-07-05 23:45:00",
"2012-07-06 00:15:00", "2012-07-06 00:45:00", "2012-07-06 01:15:00",
"2012-07-06 01:45:00", "2012-07-06 02:15:00", "2012-07-06 02:45:00",
"2012-07-06 03:15:00", "2012-07-06 03:45:00", "2012-07-06 04:15:00",
"2012-07-06 04:45:00", "2012-07-06 05:15:00", "2012-07-06 05:45:00",
"2012-07-06 06:15:00", "2012-07-06 06:45:00", "2012-07-06 07:15:00",
"2012-07-06 07:45:00", "2012-07-06 08:15:00", "2012-07-06 08:45:00",
"2012-07-06 09:15:00", "2012-07-06 09:45:00", "2012-07-06 10:15:00",
"2012-07-06 10:45:00", "2012-07-06 11:15:00", "2012-07-06 11:45:00",
"2012-07-06 12:15:00", "2012-07-06 12:45:00", "2012-07-06 13:15:00",
"2012-07-06 13:45:00", "2012-07-06 14:15:00", "2012-07-06 14:45:00",
"2012-07-06 15:15:00", "2012-07-06 15:45:00", "2012-07-06 16:15:00",
"2012-07-06 16:45:00", "2012-07-06 17:15:00", "2012-07-06 17:45:00",
"2012-07-06 18:15:00", "2012-07-06 18:45:00", "2012-07-06 19:15:00",
"2012-07-06 19:45:00", "2012-07-06 20:15:00", "2012-07-06 20:45:00",
"2012-07-06 21:15:00", "2012-07-06 21:45:00", "2012-07-06 22:15:00",
"2012-07-06 22:45:00", "2012-07-06 23:15:00", "2012-07-06 23:45:00",
"2012-07-07 00:15:00", "2012-07-07 00:45:00", "2012-07-07 01:15:00",
"2012-07-07 01:45:00", "2012-07-07 02:15:00", "2012-07-07 02:45:00",
"2012-07-07 03:15:00", "2012-07-07 03:45:00", "2012-07-07 04:15:00",
"2012-07-07 04:45:00", "2012-07-07 05:15:00", "2012-07-07 05:45:00",
"2012-07-07 06:15:00", "2012-07-07 06:45:00", "2012-07-07 07:15:00",
"2012-07-07 07:45:00", "2012-07-07 08:15:00", "2012-07-07 08:45:00",
"2012-07-07 09:15:00", "2012-07-07 09:45:00", "2012-07-07 10:15:00",
"2012-07-07 10:45:00", "2012-07-07 11:45:00", "2012-07-07 12:15:00",
"2012-07-07 12:45:00", "2012-07-07 13:15:00", "2012-07-07 13:45:00",
"2012-07-07 14:15:00", "2012-07-07 14:45:00", "2012-07-07 15:15:00",
"2012-07-07 15:45:00", "2012-07-07 16:15:00", "2012-07-07 16:45:00",
"2012-07-07 17:15:00", "2012-07-07 17:45:00", "2012-07-07 18:15:00",
"2012-07-07 18:45:00", "2012-07-07 19:15:00", "2012-07-07 19:45:00",
"2012-07-07 20:15:00", "2012-07-07 20:45:00", "2012-07-07 21:15:00",
"2012-07-07 21:45:00", "2012-07-07 22:15:00", "2012-07-07 22:45:00",
"2012-07-07 23:15:00", "2012-07-07 23:45:00", "2012-07-08 00:15:00",
"2012-07-08 00:45:00", "2012-07-08 01:15:00", "2012-07-08 01:45:00",
"2012-07-08 02:15:00", "2012-07-08 02:45:00", "2012-07-08 03:15:00",
"2012-07-08 03:45:00", "2012-07-08 04:15:00", "2012-07-08 04:45:00",
"2012-07-08 05:15:00", "2012-07-08 05:45:00", "2012-07-08 06:15:00",
"2012-07-08 06:45:00", "2012-07-08 07:45:00", "2012-07-08 08:15:00",
"2012-07-08 08:45:00", "2012-07-08 09:15:00", "2012-07-08 09:45:00",
"2012-07-08 10:15:00", "2012-07-08 10:45:00", "2012-07-08 11:15:00",
"2012-07-08 11:45:00", "2012-07-08 12:15:00", "2012-07-08 12:45:00",
"2012-07-08 13:15:00", "2012-07-08 13:45:00", "2012-07-08 14:15:00",
"2012-07-08 14:45:00", "2012-07-08 15:15:00", "2012-07-08 16:15:00",
"2012-07-08 16:45:00", "2012-07-08 17:15:00", "2012-07-08 17:45:00",
"2012-07-08 18:15:00", "2012-07-08 18:45:00", "2012-07-08 19:15:00",
"2012-07-08 19:45:00", "2012-07-08 20:15:00", "2012-07-08 20:45:00",
"2012-07-08 21:15:00", "2012-07-08 21:45:00", "2012-07-08 22:15:00",
"2012-07-08 22:45:00", "2012-07-08 23:15:00", "2012-07-08 23:45:00",
"2012-07-09 00:15:00", "2012-07-09 00:45:00", "2012-07-09 01:15:00",
"2012-07-09 01:45:00", "2012-07-09 02:15:00", "2012-07-09 02:45:00",
"2012-07-09 03:15:00", "2012-07-09 03:45:00", "2012-07-09 04:15:00",
"2012-07-09 04:45:00", "2012-07-09 05:15:00", "2012-07-09 05:45:00",
"2012-07-09 06:15:00", "2012-07-09 06:45:00", "2012-07-09 07:15:00",
"2012-07-09 07:45:00", "2012-07-09 08:15:00", "2012-07-09 08:45:00",
"2012-07-09 09:15:00", "2012-07-09 09:45:00", "2012-07-09 10:15:00",
"2012-07-09 10:45:00", "2012-07-09 11:15:00", "2012-07-09 11:45:00",
"2012-07-09 12:15:00", "2012-07-09 12:45:00", "2012-07-09 13:15:00",
"2012-07-09 13:45:00", "2012-07-09 14:15:00", "2012-07-09 14:45:00",
"2012-07-09 15:15:00", "2012-07-09 15:45:00", "2012-07-09 16:15:00",
"2012-07-09 16:45:00", "2012-07-09 17:15:00", "2012-07-09 17:45:00",
"2012-07-09 18:15:00", "2012-07-09 18:45:00", "2012-07-09 19:15:00",
"2012-07-09 19:45:00", "2012-07-09 20:15:00", "2012-07-09 21:15:00",
"2012-07-09 21:45:00", "2012-07-09 22:15:00", "2012-07-09 22:45:00",
"2012-07-09 23:15:00", "2012-07-09 23:45:00", "2012-07-10 00:15:00",
"2012-07-10 00:45:00", "2012-07-10 01:15:00", "2012-07-10 01:45:00",
"2012-07-10 02:15:00", "2012-07-10 02:45:00", "2012-07-10 03:15:00",
"2012-07-10 03:45:00", "2012-07-10 04:15:00", "2012-07-10 04:45:00",
"2012-07-10 05:15:00", "2012-07-10 05:45:00", "2012-07-10 06:15:00",
"2012-07-10 06:45:00", "2012-07-10 07:15:00", "2012-07-10 07:45:00",
"2012-07-10 08:15:00", "2012-07-10 08:45:00", "2012-07-10 09:15:00",
"2012-07-10 09:45:00", "2012-07-10 10:15:00", "2012-07-10 10:45:00",
"2012-07-10 11:15:00", "2012-07-10 11:45:00", "2012-07-10 12:15:00",
"2012-07-10 12:45:00", "2012-07-10 13:15:00", "2012-07-10 13:45:00",
"2012-07-10 14:15:00", "2012-07-10 14:45:00", "2012-07-10 15:15:00",
"2012-07-10 15:45:00", "2012-07-10 16:15:00", "2012-07-10 17:15:00",
"2012-07-10 17:45:00", "2012-07-10 18:15:00", "2012-07-10 18:45:00",
"2012-07-10 19:15:00", "2012-07-10 19:45:00", "2012-07-10 20:15:00",
"2012-07-10 20:45:00", "2012-07-10 21:15:00", "2012-07-10 21:45:00",
"2012-07-10 22:15:00", "2012-07-10 22:45:00", "2012-07-10 23:15:00",
"2012-07-10 23:45:00"), class = "factor"), mon = c(1L, 1L, 1L,
1L, 1L, 1L), X.1 = c(482986.747312444, 483106.142664631, 483101.821796104,
483086.019405256, 483127.799620954, 483101.003261731), Y = c(6472837.55691427,
6472855.34802117, 6472852.53770682, 6472786.61976395, 6472854.86710904,
6472866.97636095), Z = c(6.49352941176471, 4.03558823529412,
6.03769230769231, 3.9875, 2.94461538461538, 2.86)), .Names = c("X",
"meanlat", "meanlong", "meandepth", "mindepth", "maxdepth", "depthrange",
"rec", "numhits", "datetime", "mon", "X.1", "Y", "Z"), row.names = c(NA,
6L), class = "data.frame")
>
You look that You tray many things but you don't try one thing in a good way.
For example using POSIXlt, dat here is your PAVdata object
ll <- as.POSIXlt(dat$datetime)
dat$hours <- ll$hour ## I create a variable hour
dat$days <- ll$mday ## I create a variable day
dat$months <- ll$mon ## I create a variable month
Now i can use this variable to average by group.
here a solution using plyr package.
ddply(dat,.(months,days),summarise, m = mean(meandepth)) ## I group by months and days
months days m
1 4 11 4.393154
> ddply(dat,.(months,days,hours),summarise, m = mean(meandepth))
months days hours m
1 4 11 11 5.264558
2 4 11 12 5.012596
3 4 11 13 2.902308
Whatever you use, try to use one thing in a good way. Generally there are many method to do such things in R.