Convert HH:MM:SS column to ZT time - r

I am hoping somebody can help me with a logic question in R-studio. I have a rather large data set, with "Time" as one of my columns. This column has values from 00:00:00 to 23:59:00, and is in HH:MM:SS format.
Because of some trouble I have had with analysis of time in this format, I am trying to create a new column, called "ZT" where I convert this time column to ZT time. Lights turn on at 7am, so need the time 07:00:00 to correspond to ZT=0, with 07:01:00 to correspond to ZT=0.016... and so on and so forth.
Can anybody help me with this? It would be much appreciated!

Not sure if this is what you are going for or not but this seems to work at converting a character vector of times in the format HH:MM:SS to your ZT time in the format HH:MM:SS starting at 7am as 00:00:00.
I am unclear exactly what you mean when you state that 07:01:00 should correspond to ZT=0.016, but maybe this can be a start.
Fair warning this is a little slow (took about 1 minute on my machine) but maybe someone else can help vectorize it and speed it up:
#Make Some Fake Data
df<-data.frame(Time=format(seq(ISOdate(2020,1,1), ISOdate(2020,2,1), by = "min"), '%H:%M:00'), Variable1=runif(n=44641))
#We need the help of a an external package to handle time in HH:MM:SS format
library(lubridate)
time_store<- hms(df$Time) #Convert your times to HMS format
ZT_vec<-vector() #Create an empty vector that we will fill in
for (i in 1:length(time_store)){ #iterate over each observation
if (hour(time_store[i])<7){ #Make sure the conversions are going the right direction
ZT<-time_store[i]+hours(17)
ZT_vec<-c(ZT_vec,sprintf("%02d:%02d:%02d", hour(ZT), minute(ZT), second(ZT))) #format the times in HH:MM:SS
} else {
ZT<-time_store[i]-hours(7)
ZT_vec<-c(ZT_vec,sprintf("%02d:%02d:%02d", hour(ZT), minute(ZT), second(ZT)))
}
}
df<-cbind(df,ZT_vec) #Bind on our new column
head(df)
Time Variable1 ZT_vec
12:00:00 0.6560604 05:00:00
12:01:00 0.3485023 05:01:00
12:02:00 0.8396784 05:02:00
12:03:00 0.4773929 05:03:00
12:04:00 0.6969242 05:04:00
12:05:00 0.5371502 05:05:00
head(df[4020:4025,])
Time Variable1 ZT_vec
06:59:00 0.6758364 23:59:00
07:00:00 0.1255861 00:00:00
07:01:00 0.2789485 00:01:00
07:02:00 0.2175933 00:02:00
07:03:00 0.1855100 00:03:00
07:04:00 0.1632865 00:04:00

Related

Add hour if missing in timestamp using lubridate

I have a list of timestamps in the factor format the I want to convert using lurbridate.
However some of the timestamps lack time 00:00:00:
2013-12-24 23:00:00
2013-12-24
2013-12-24 01:00:00
How do I expand the df$timestamp <- ymd_hms(df$Timestamp_factor) to insert 00:00:00 if time i missing?
You can use the truncated term for lubridate to get those missing parameters. In this case, you are missing three parameters hour, minute and second
ymd_hms(c("2013-12-24 23:00:00", "2013-12-24", "2013-12-24 01:00:00"), truncated = 3)
That, however, will always return 00:00:00 as the time

Dealing with Eastern Standard Time (EST) and Eastern Daylight Savings (EDT) in R

I have a series of datasets from a water quality continuous monitoring probe with the Date and Time set in Eastern Standard Time (EST) so there is no correction of Daylight Savings Time (EDT). In R, the fields are recognized as factors when importing the data table from an MS access database however when converting using as.POSIXct() dates and times from 02:00 (24 clock) on 2016-03-13 become NAs. This is due to the transition from EST to EDT...therefore technically 2016-03-13 02:00 doesn't exist.
Some created data as an example
test<-data.frame(Date=rep(as.Date("2016-03-13"),120),Hour=rep(seq(0,23,1),5),Min=rep(seq(0,60,15),24))
Is there a way to convert the factor or character field to as POSIXct field while retaining the EST timezone designation? Alternatively is there a way to identify and convert the proper date and times into EST and EDT?
I have gone around and around and can't get anything to work. I have attempted to convert to GMT (or UTC) then convert back to EST (tz="America/New_York"). I realize that this is an ongoing issue and people who work with date and time data, especially in R would love to move away from EDT.
Any help is appreciated...I am at my wits end on this one.
The trouble with using POSIX tz = "America/New_York" is that daylight savings time is accounted for (UTC-4 or UTC-5), even if the underlying timestamps are stored in U.S. Eastern Standard Time (UTC-5).
You should be able to designate your tz as Etc/GMT+5. From there, it is easy to convert between EST, Eastern local time, and GMT. Note that in R, time zones west of UTC are denoted with a positive offset (see Time Zone Names documentation in ?timezone).
Here is some example data (daylight savings took effect at 2:00 AM Eastern Time on 3/16/16):
StartTime=as.numeric(as.POSIXct("2016-03-11 0:00:00",format="%Y-%m-%d %H:%M",origin="1970-01-01",tz="Etc/GMT+5"))
EndTime=as.numeric(as.POSIXct("2016-03-15 0:00:00",format="%Y-%m-%d %H:%M",origin="1970-01-01",tz="Etc/GMT+5"))
Interval=15*60 #15-min
data.EST=as.POSIXct(seq(from = StartTime,to = EndTime, by=Interval),origin="1970-01-01",tz="Etc/GMT+5") #generate date stamps
# convert Eastern Standard Time (in R: GMT+5) to local time (accounts for daylight savings):
data.EastCoast<- format(data.EST, tz="America/New_York")
# convert Eastern Standard Time (in R: GMT+5) to UTC/GMT:
data.UTC<- format(data.EST, tz="GMT")
compare.times<-data.frame(data.EST,data.EastCoast,data.UTC)
compare.times[(198:203),]
data.EST data.EastCoast data.UTC
198 2016-03-13 01:15:00 2016-03-13 01:15:00 2016-03-13 06:15:00
199 2016-03-13 01:30:00 2016-03-13 01:30:00 2016-03-13 06:30:00
200 2016-03-13 01:45:00 2016-03-13 01:45:00 2016-03-13 06:45:00
201 2016-03-13 02:00:00 2016-03-13 03:00:00 2016-03-13 07:00:00
202 2016-03-13 02:15:00 2016-03-13 03:15:00 2016-03-13 07:15:00
203 2016-03-13 02:30:00 2016-03-13 03:30:00 2016-03-13 07:30:00
Good luck!
During the conversion to POSIX you need to specify the time zone. See this example:
test<-data.frame(Date=rep(as.Date("2016-03-13"),96),Hour=rep(seq(0,23,1), each=4),Min=rep(seq(0,45,15)))
wrong<-as.POSIXct(paste(test$Date, test$Hour, test$Min), format="%Y-%m-%d %H %M")
ans<-as.POSIXct(paste(test$Date, test$Hour, test$Min), format="%Y-%m-%d %H %M", tz="EST")
compare<-cbind(test, wrong, ans)
In the vector "wrong", no timezone was specified thus the NA, but in the second case Eastern Standard was specified and the desired result is given.
I have run into a similar issue using water quality data that does not observe daylight savings time. The workaround I have found useful is to use America/Jamaica rather than America/New_York. Below are a list of GMT offsets and the tz to use.
-4 = America/Virgin
-5 = America/Jamaica
-6 = America/Regina
-8 = Pacific/Pitcairn
-9 = Pacific/Gambier
-10 = Pacific/Honolulu

convert hours since date to datetime format in r

I've downloaded some climate data from a website and am trying to analyse it in R.
The time format for the data is hours since 1800-01-01 00:00. For example:
ss <- seq(447042,455802, length.out = 1461)
which shows data at 6 hour intervals.
How can I convert this to a an actual time in R. This example should give data for 1851:
1851-01-01 00:00
1851-01-01 06:00
and so on...
How can this be done?
Any advice would be appreciated.
I think you have a typo as well as incorrect calculations. Let's assume you want time in the future of 1880 rather than the past. So it might be 1951 you wanted? Then to convert hours to seconds which are the basis for the POSIXt classed objects, just multiply by 3600 = 60*60:
> tail( as.POSIXct(ss*3600,origin='1880-01-01 00:00') )
[1] "1931-12-30 04:00:00 PST" "1931-12-30 10:00:00 PST" "1931-12-30 16:00:00 PST"
[4] "1931-12-30 22:00:00 PST" "1931-12-31 04:00:00 PST" "1931-12-31 10:00:00 PST"
As you can see it's nowhere the year 1951 either, but maybe you had two digits off and you wanted 1931? Conversions that span the range of years before the onset of daylight savings time and cross century boundaries where leap years and leap seconds were used may not "line up" with your expectations.

Binning time series in R?

I'm new to R. My data has 600k objects defined by three attributes: Id, Date and TimeOfCall.
TimeofCall has a 00:00:00 format and range from 00:00:00 to 23:59:59.
I want to bin the TimeOfCall attribute, into 24 bins, each one representing hourly slot (first bin 00:00:00 to 00:59:59 and so on).
Can someone talk me through how to do this? I tried using cut() but apparently my format is not numeric. Thanks in advance!
While you could convert to a formal time representation, in this case it might be easier to just use substr:
test <- c("00:00:01","02:07:01","22:30:15")
as.numeric(substr(test,1,2))
#[1] 0 2 22
Using a POSIXct time to deal with it would also work, and might be handy if you plan on further calculations (differences in time etc):
testtime <- as.POSIXct(test,format="%H:%M:%S")
#[1]"2013-12-09 00:00:01 EST" "2013-12-09 02:07:01 EST" "2013-12-09 22:30:15 EST"
as.numeric(format(testtime,"%H"))
#[1] 0 2 22
You can use cut.POsixlt function. But you should coerce your data to a valid time object. here I am using handy hms from lubridate. And strftime to get the time format.
library(lubridate)
x <- c("09:10:01", "08:10:02", "08:20:02","06:10:03 ", "Collided at 9:20:04 pm")
x.h <- strftime(cut(as.POSIXct(hms(x),origin=Sys.Date()),'hours'),
format='%H:%M:%S')
data.frame(x,x.h)
x x.h
1 09:10:01 10:00:00
2 08:10:02 09:00:00
3 08:20:02 09:00:00
4 06:10:03 07:00:00
5 Collided at 9:20:04 pm 22:00:00

How can I alter a time series (XTS or ZOO) in R?

I am new to stackoverflow and fairly new to R but have searched long and hard and cannot find an answer to the following question.
I have a number of data files that are temperature against a time series. I am importing the CSV as a ZOO object then converting to XTS. A correct file looks like this, with readings on the hour and the half hour:
>head(master1)
S_1
2010-03-03 00:00:00 2.8520
2010-03-03 00:30:00 2.6945
2010-03-03 01:00:00 2.5685
2010-03-03 01:30:00 2.3800
2010-03-03 02:00:00 2.2225
2010-03-03 02:30:00 2.0650
But the time value on some are slightly out - i.e. 23:59:00 not 00:00:00, or 00:29:00 instead of 00:30:00.
>head(master21)
S_21
2010-03-04 23:59:00 -0.593
2010-03-05 00:29:00 -0.908
2010-03-05 00:59:00 -1.034
2010-03-05 01:29:00 -1.223
2010-03-05 01:59:00 -1.349
2010-03-05 02:29:00 -1.538
I want to correct these time series, as the minute difference is not important for my analysis and I ultimately want to merge the files, so each timeseries needs to have the same timing.
I want a command that can just say "shift the time series forward by 1 minute, but don't alter the data column (e.g. S_21).
I have had some luck with gsub() on easier changes, and contemplated a complex regex to change the data before it is converted to ZOO or XTS. I have read about lag() and diff() but they seem to move the data values relative to the time series; please correct me if I am wrong.
Any help solving this issue would be much appreciated.
Try
index(master21) <- index(master21) + 60 # adds a minute
which will add a minute to the time index. You can then use merge() as the timestamps align.
More generally, the vignettes of the zoo package will be useful for you too.

Resources