I’m having difficulties with a date time problem. My data frame looks like this and I want to find the duration that each person watches TV.
Start.Time <- c(193221,201231,152324,182243,123432,192245)
End.Time <- c(202013,211232,154521,183422,133121,201513)
cbind(Start.Time,End.Time)
I have tried different methods to convert them in order to be able to make calculation but I didn’t produce any significant results.
as.POSIXct(Start.Time , origin="2015-11-01")
My results are completely wrong
[1] "2015-11-03 05:40:21 GMT" "2015-11-03 07:53:51 GMT"
[3] "2015-11-02 18:18:44 GMT" "2015-11-03 02:37:23 GMT"
[5] "2015-11-02 10:17:12 GMT" "2015-11-03 05:24:05 GMT"
For example I want 193221 to become 19:32:21 HH:MM:SS
Is there a package out there that easily does the conversion? and if its possible i don't want the date displayed, just the time.
You can convert your numbers to actual time stamps (in POSIXct format) like this:
Start.Time <- c(193221,201231,152324,182243,123432,192245)
Start.POSIX <- as.POSIXct(as.character(Start.Time), format = "%H%M%S")
Start.POSIX
## [1] "2015-12-19 19:32:21 CET" "2015-12-19 20:12:31 CET" "2015-12-19 15:23:24 CET"
## [4] "2015-12-19 18:22:43 CET" "2015-12-19 12:34:32 CET" "2015-12-19 19:22:45 CET"
As you can see, as.POSIXct assumes the times to belong to the current date. POSIXct alway denotes a specific moment in time and thus contains not only a time but also a date. You can now easily do calculations with these:
End.Time <- c(202013,211232,154521,183422,133121,201513)
End.POSIX <- as.POSIXct(as.character(End.Time), format = "%H%M%S")
End.POSIX - Start.POSIX
## Time differences in mins
## [1] 47.86667 60.01667 21.95000 11.65000 56.81667 52.46667
When you print the POSIXct objects (as I did above with Start.POSIX) they are acutally converted to characters and these are printed. You can see this, because there are " around the dates. You can control the format that is used when printing and thus, you could print the times only as follows:
format(Start.POSIX, "%H:%M:%S")
## [1] "19:32:21" "20:12:31" "15:23:24" "18:22:43" "12:34:32" "19:22:45"
Related
I am doing an experiment which produces automatically logged data. The software produces a timestamp that is of the format 41149.014850. I would like to convert this decimal timestamp to 28.08.2012 00:21:23. How can I do this in R most elegantly?
I tried using the function strsplit and also the function as.Date with a specified origin, and also the times function. But to no avail. I have problems in splitting the timestamp into two numbers that I can access with the functions as.Date and times.
Here is some demo code:
myDatetime <- c(41149.004641, # 28.08.2012 00:06:41
41149.009745, # 28.08.2012 00:14:02
41149.014850, # 28.08.2012 00:21:23
41149.019954) # 28.08.2012 00:28:44
## not working out for me
Dat.char <- as.character(myDatetime)
date.split <- strsplit(Dat.char, split = "\\.")
## how to proceed from here, if it is a good way at all
You dates are in an Excel-like date format (days after January 1, 1900), so you need to convert them to an R date format. Then you can convert it to a datetime format (POSIXct).
# first convert to R Date
datetime <- as.Date(myDatetime-1, origin="1899-12-31")
# now convert to POSIXct
(posixct <- .POSIXct(unclass(datetime)*86400, tz="GMT"))
# [1] "2012-08-28 00:06:40 GMT" "2012-08-28 00:14:01 GMT"
# [3] "2012-08-28 00:21:23 GMT" "2012-08-28 00:28:44 GMT"
# times are sometimes off by 1 second, add more digits to seconds to see why
options(digits.secs=6)
posixct
# [1] "2012-08-28 00:06:40.9823 GMT" "2012-08-28 00:14:01.9680 GMT"
# [3] "2012-08-28 00:21:23.0399 GMT" "2012-08-28 00:28:44.0256 GMT"
# round to nearest second
(posixct <- round(posixct, "sec"))
# [1] "2012-08-28 00:06:41 GMT" "2012-08-28 00:14:02 GMT"
# [3] "2012-08-28 00:21:23 GMT" "2012-08-28 00:28:44 GMT"
# now you can convert to your desired format
format(posixct, "%d.%m.%Y %H:%M:%S")
# [1] "28.08.2012 00:06:41" "28.08.2012 00:14:02"
# [3] "28.08.2012 00:21:23" "28.08.2012 00:28:44"
This gets close: The numbers indicate the number of seconds since a date close to 1/1/1900:
as.POSIXct(x*3600*24, origin=as.Date("1900-01-01")-2, tz="UTC")
[1] "2012-08-28 01:06:40 BST" "2012-08-28 01:14:01 BST" "2012-08-28 01:21:23 BST"
[4] "2012-08-28 01:28:44 BST"
There still is a timezone offset in there.
I'm currently playing around a lot with dates and times for a package I'm building.
Stumbling across this post reminded me again that it's generally not a bad idea to check out if something can be done with basic R features before turning to contrib packages.
Thus, is it possible to round a date of class POSIXct with base R functionality?
I checked
methods(round)
which "only" gave me
[1] round.Date round.timeDate*
Non-visible functions are asterisked
This is what I'd like to do (Pseudo Code)
x <- as.POSIXct(Sys.time())
[1] "2012-07-04 10:33:55 CEST"
round(x, atom="minute")
[1] "2012-07-04 10:34:00 CEST"
round(x, atom="hour")
[1] "2012-07-04 11:00:00 CEST"
round(x, atom="day")
[1] "2012-07-04 CEST"
I know this can be done with timeDate, lubridate etc., but I'd like to keep package dependencies down. So before going ahead and checking out the source code of the respective packages, I thought I'd ask if someone has already done something like this.
base has round.POSIXt to do this. Not sure why it doesn't come up with methods.
x <- as.POSIXct(Sys.time())
x
[1] "2012-07-04 10:01:08 BST"
round(x,"mins")
[1] "2012-07-04 10:01:00 BST"
round(x,"hours")
[1] "2012-07-04 10:00:00 BST"
round(x,"days")
[1] "2012-07-04"
On this theme with lubridate, also look into the ceiling_date() and floor_date() functions:
x <- as.POSIXct("2009-08-03 12:01:59.23")
ceiling_date(x, "second")
# "2009-08-03 12:02:00 CDT"
ceiling_date(x, "hour")
# "2009-08-03 13:00:00 CDT"
ceiling_date(x, "day")
# "2009-08-04 CDT"
ceiling_date(x, "week")
# "2009-08-09 CDT"
ceiling_date(x, "month")
# "2009-09-01 CDT"
If you don't want to call external libraries and want to keep POSIXct as I do this is one idea (inspired by this question): use strptime and paste a fake month and day. It should be possible to do it more straight forward, as said in this comment
"For strptime the input string need not specify the date completely:
it is assumed that unspecified seconds, minutes or hours are zero, and
an unspecified year, month or day is the current one."
Thus it seems that you have to use strftime to output a truncated string, paste the missing part and convert again in POSIXct.
This is how an update answer could look:
x <- as.POSIXct(Sys.time())
x
[1] "2018-12-27 10:58:51 CET"
round(x,"mins")
[1] "2018-12-27 10:59:00 CET"
round(x,"hours")
[1] "2018-12-27 11:00:00 CET"
round(x,"days")
[1] "2018-12-27 CET"
as.POSIXct(paste0(strftime(x,format="%Y-%m"),"-01")) #trunc by month
[1] "2018-12-01 CET"
as.POSIXct(paste0(strftime(x,format="%Y"),"-01-01")) #trunc by year
[1] "2018-01-01 CET"
How would I extract the time from a series of POSIXct objects discarding the date part?
For instance, I have:
times <- structure(c(1331086009.50098, 1331091427.42461, 1331252565.99979,
1331252675.81601, 1331262597.72474, 1331262641.11786, 1331269557.4059,
1331278779.26727, 1331448476.96126, 1331452596.13806), class = c("POSIXct",
"POSIXt"))
which corresponds to these dates:
"2012-03-07 03:06:49 CET" "2012-03-07 04:37:07 CET"
"2012-03-09 01:22:45 CET" "2012-03-09 01:24:35 CET"
"2012-03-09 04:09:57 CET" "2012-03-09 04:10:41 CET"
"2012-03-09 06:05:57 CET" "2012-03-09 08:39:39 CET"
"2012-03-11 07:47:56 CET" "2012-03-11 08:56:36 CET"
Now, I have some values for a parameter measured at those times:
val <- c(1.25343125e-05, 0.00022890575,
3.9269125e-05, 0.0002285681875,
4.26353125e-05, 5.982625e-05,
2.09575e-05, 0.0001516951251,
2.653125e-05, 0.0001021391875)
I would like to plot val vs time of the day, irrespectively of the specific day when val was measured.
Is there a specific function that would allow me to do that?
You can use strftime to convert datetimes to any character format:
> t <- strftime(times, format="%H:%M:%S")
> t
[1] "02:06:49" "03:37:07" "00:22:45" "00:24:35" "03:09:57" "03:10:41"
[7] "05:05:57" "07:39:39" "06:47:56" "07:56:36"
But that doesn't help very much, since you want to plot your data. One workaround is to strip the date element from your times, and then to add an identical date to all of your times:
> xx <- as.POSIXct(t, format="%H:%M:%S")
> xx
[1] "2012-03-23 02:06:49 GMT" "2012-03-23 03:37:07 GMT"
[3] "2012-03-23 00:22:45 GMT" "2012-03-23 00:24:35 GMT"
[5] "2012-03-23 03:09:57 GMT" "2012-03-23 03:10:41 GMT"
[7] "2012-03-23 05:05:57 GMT" "2012-03-23 07:39:39 GMT"
[9] "2012-03-23 06:47:56 GMT" "2012-03-23 07:56:36 GMT"
Now you can use these datetime objects in your plot:
plot(xx, rnorm(length(xx)), xlab="Time", ylab="Random value")
For more help, see ?DateTimeClasses
The data.table package has a function 'as.ITime', which can do this efficiently use below:
library(data.table)
x <- "2012-03-07 03:06:49 CET"
as.IDate(x) # Output is "2012-03-07"
as.ITime(x) # Output is "03:06:49"
There have been previous answers that showed the trick. In essence:
you must retain POSIXct types to take advantage of all the existing plotting functions
if you want to 'overlay' several days worth on a single plot, highlighting the intra-daily variation, the best trick is too ...
impose the same day (and month and even year if need be, which is not the case here)
which you can do by overriding the day-of-month and month components when in POSIXlt representation, or just by offsetting the 'delta' relative to 0:00:00 between the different days.
So with times and val as helpfully provided by you:
## impose month and day based on first obs
ntimes <- as.POSIXlt(times) # convert to 'POSIX list type'
ntimes$mday <- ntimes[1]$mday # and $mon if it differs too
ntimes <- as.POSIXct(ntimes) # convert back
par(mfrow=c(2,1))
plot(times,val) # old times
plot(ntimes,val) # new times
yields this contrasting the original and modified time scales:
Here's an update for those looking for a tidyverse method to extract hh:mm::ss.sssss from a POSIXct object. Note that time zone is not included in the output.
library(hms)
as_hms(times)
Many solutions have been provided, but I have not seen this one, which uses package chron:
hours = times(strftime(times, format="%T"))
plot(val~hours)
(sorry, I am not entitled to post an image, you'll have to plot it yourself)
I can't find anything that deals with clock times exactly, so I'd just use some functions from package:lubridate and work with seconds-since-midnight:
require(lubridate)
clockS = function(t){hour(t)*3600+minute(t)*60+second(t)}
plot(clockS(times),val)
You might then want to look at some of the axis code to figure out how to label axes nicely.
The time_t value for midnight GMT is always divisible by 86400 (24 * 3600). The value for seconds-since-midnight GMT is thus time %% 86400.
The hour in GMT is (time %% 86400) / 3600 and this can be used as the x-axis of the plot:
plot((as.numeric(times) %% 86400)/3600, val)
To adjust for a time zone, adjust the time before taking the modulus, by adding the number of seconds that your time zone is ahead of GMT. For example, US central daylight saving time (CDT) is 5 hours behind GMT. To plot against the time in CDT, the following expression is used:
plot(((as.numeric(times) - 5*3600) %% 86400)/3600, val)
My goal is to create a vector of POSIXct time stamps given a start, an end and a delta (15min, 1hour, 1day). I hoped I could use seq for this, but I have a problem converting between the numeric and POSIXct representation:
now <- Sys.time()
now
# [1] "2012-01-19 10:30:39 CET"
as.POSIXct(as.double(now), origin="1970-01-01", tz="CET")
# [1] "2012-01-19 09:30:39 CET"
as.POSIXct(as.double(now), origin=as.POSIXct("1970-01-01", tz="CET"), tz="CET")
# [1] "2012-01-19 09:30:39 CET"
One hour gets lost during this conversion. What am I doing wrong?
There is a seq() method for objects of class "POSIXt" which is the super class of the "POSIXlt" and "POSIXct" classes. As such you don't need to do any conversion.
> now <- Sys.time()
> tseq <- seq(from = now, length.out = 100, by = "mins")
> length(tseq)
[1] 100
> head(tseq)
[1] "2012-01-19 10:52:38 GMT" "2012-01-19 10:53:38 GMT"
[3] "2012-01-19 10:54:38 GMT" "2012-01-19 10:55:38 GMT"
[5] "2012-01-19 10:56:38 GMT" "2012-01-19 10:57:38 GMT"
You have to be aware that when converting from POSIXct to numeric, R takes the timezone into account but always starts counting from a GMT origin :
> xgmt <- as.POSIXct('2011-01-01 14:00:00',tz='GMT')
> xest <- as.POSIXct('2011-01-01 14:00:00',tz='EST')
> (as.numeric(xgmt) - as.numeric(xest)) / 3600
[1] -5
As you see, the time in EST is conceived to be five hours earlier than the time in GMT, which is the time difference between both timezones. It's that value that is saved internally.
The as.POSIXCT() function just adds an attribute containing the timezone. It doesn't alter the value, so you get the time presented in GMT time, but with an attribute telling it is EST. This also means that once you go from POSIXct to numeric, you should treat your data as if it's GMT time. (It's a whole lot more complex than that, but it's the general idea). So you have to calculate the offset as follows:
> nest <- as.numeric(xest)
> origin <- as.POSIXct('1970-01-01 00:00:00',tz='EST')
> offset <- as.numeric(origin)
> as.POSIXct(nest-offset,origin=origin)
[1] "2011-01-01 14:00:00 EST"
This works whatever the timezone is in your locale (in my case, that's actually CET). Also note that behaviour of timezone data can differ between systems.
These time zone issues are always fiddly, but I think the problem is that your origin is being calculated in the wrong time zone (since the string only specifies the date).
Try using origin <- now - as.numeric(now).
Alternatively, use lubridate::origin, which is the string "1970-01-01 UTC".
A full solution, again using lubridate.
start <- now()
seq(start, start + days(3), by = "15 min")
I do not have an answer to your problem, but I do have an alternative way of creating vectors of POSIXct objects. If, for example, you want to create a vector of 1000 timestamps from now with a delta_t of 15 minutes:
now = Sys.time()
dt = 15 * 60 # in seconds
timestamps = now + seq(0, 1000) * dt
> head(timestamps)
[1] "2012-01-19 11:17:46 CET" "2012-01-19 11:32:46 CET"
[3] "2012-01-19 11:47:46 CET" "2012-01-19 12:02:46 CET"
[5] "2012-01-19 12:17:46 CET" "2012-01-19 12:32:46 CET"
The trick is you can add a vector of seconds to a POSIXct object.
An alternative to using seq.POSIXt is xts::timeBasedSeq, which allows you to specify the sequence as a string:
library(xts)
now <- Sys.time()
timeBasedSeq(paste("2012-01-01/",format(now),"/H",sep="")) # Hourly steps
timeBasedSeq(paste("2012-01-01/",format(now),"/d",sep="")) # Daily steps
You need to use seq(from=start,to=end, by=step). Note that in step you can either use "days" or an integer defining how many seconds elapse from item to item.
I need to use as.Date on the index of a zoo object. Some of the dates are in BST and so when converting I lose a day on (only) these entries. I don't care about one hour's difference or even the time part of the date at all, I just want to make sure that the dates displayed stay the same. I'm guessing this is not very hard but I can't manage it. Can somebody help please?
class(xtsRet)
#[1] "xts" "zoo"
index(xtsRet)
#[1] "2007-07-31 BST" "2007-08-31 BST" "2007-09-30 BST" "2007-10-31 GMT"
class(index(xtsRet))
#[1] "POSIXt" "POSIXct"
index(xtsRet) <- as.Date(index(xtsRet))
index(xtsRet)
#[1] "2007-07-30" "2007-08-30" "2007-09-29" "2007-10-31"
Minimally reproducible example (not requiring zoo package):
my_date <- as.POSIXct("2007-04-01") # Users in non-UK timezone will need to
# do as.POSIXct("2007-04-01", "Europe/London")
my_date
#[1] "2017-04-01 BST"
as.Date(my_date)
#[1] "2017-03-31"
Suppose we have this sample data:
library(zoo)
x <- as.POSIXct("2000-01-01", tz = "GMT")
Then see if any of these are what you want:
# use current time zone
as.Date(as.character(x, tz = ""))
# use GMT
as.Date(as.character(x, tz = "GMT"))
# set entire session to GMT
Sys.setenv(TZ = "GMT")
as.Date(x)
Also try "BST" in place of "GMT" and note the article on dates and times in R News 4/1 .
You can offset the POSIX objects so its not based around midnight. 1 hour (3600 secs) should be sufficient:
d <- as.POSIXct(c("2007-07-31","2007-08-31","2007-09-30","2007-10-31"))
d
[1] "2007-07-31 BST" "2007-08-31 BST" "2007-09-30 BST" "2007-10-31 GMT"
as.Date(d)
[1] "2007-07-30" "2007-08-30" "2007-09-29" "2007-10-31"
as.Date(d+3600)
[1] "2007-07-31" "2007-08-31" "2007-09-30" "2007-10-31"
I would suggest using as.POSIXlt to convert to a date object, wrapped in as.Date:
d <- as.POSIXct(c("2007-07-31","2007-08-31","2007-09-30","2007-10-31"))
d
[1] "2007-07-31 BST" "2007-08-31 BST" "2007-09-30 BST" "2007-10-31 GMT"
as.Date(as.POSIXlt(d))
[1] "2007-07-31" "2007-08-31" "2007-09-30" "2007-10-31"
Achieves the same thing as the +3600 above, but slightly less of a hack