I have a time variable say x. The class of my vector is a character
x <- c("06:59:20","19:13:31", "06:08:02", "00:25:14", "19:25:01" ,"21:21:44"
,"19:04:25" , "19:43:31", "16:00:13", "05:15:37", "05:52:16", "21:47:51", "06:33:36"
,"06:59:29" ,"20:06:04", "05:08:15", "20:09:51", "21:52:25", "19:07:22", "19:37:26")
I convert it into a times object by using the below approach
x_posix <- as.POSIXct(x, format = "%H:%M:%S")
x<- strftime(x_posix, format="%H:%M:%S")
x <- times(x)
[1] 06:59:20 19:13:31 06:08:02 00:25:14 19:25:01 21:21:44 19:04:25 19:43:31 16:00:13 05:15:37 05:52:16 21:47:51 06:33:36 06:59:29 20:06:04
[16] 05:08:15 20:09:51 21:52:25 19:07:22 19:37:26
Now I want to find out how many values are lie between any time interval.
Suppose If I want to find out that how many values are between 6 to 7 then I want to include all those values that are in between 6:00:00(including 6:00:00) and up to all values that are less than or equal to 6:59:59.
Is there any way in R to accomplish this task ?
We can use
library(chron)
sum(x1 >= times("06:00:00") & x1 < times("07:00:00"))
#[1] 4
data
x1 <- times(x)
Related
I'm trying to set up a new variable that incorporates the difference (in number of days) between a known date and the end of a given year. Dummy data below:
> Date.event <- as.POSIXct(c("12/2/2000","8/2/2001"), format = "%d/%m/%Y", tz = "Europe/London")
> Year = c(2000,2001)
> Dates.test <- data.frame(Date.event,Year)
> Dates.test
Date.event Year
1 2000-02-12 2000
2 2001-02-08 2001
I've tried applying a function to achieve this, but it returns an error
> Time.dif.fun <- function(x) {
+ as.numeric(as.POSIXct(sprintf('31/12/%s', s= x['Year']),format = "%d/%m/%Y", tz = "Europe/London") - x['Date.event'])
+ }
> Dates.test$Time.dif <- apply(
+ Dates.test, 1, Time.dif.fun
+ )
Error in unclass(e1) - e2 : non-numeric argument to binary operator
It seems that apply() does not like as.POSIXct(), as testing a version of the function that only derives the end of year date, it is returned as a numeric in the form '978220800' (e.g. for end of year 2000). Is there any way around this? For the real data the function is a bit more complex, including conditional instances using different variables and sometimes referring to previous rows, which would be very hard to do without apply.
Here are some alternatives:
1) Your code works with these changes. We factored out s, not because it is necessary, but only because the following line gets very hard to read without that due to its length. Note that if x is a data frame then so is x["Year"] but x[["Year"]] is a vector as is x$Year. Since the operations are all vectorized we do not need apply.
Although we have not made this change, it would be a bit easier to define s as s <- paste0(x$Year, "-12-31") in which case we could omit the format argument in the following line owing to the use of the default format.
Time.dif.fun <- function(x) {
s <- sprintf('31/12/%s', x[['Year']])
as.numeric(as.POSIXct(s, format = "%d/%m/%Y", tz = "Europe/London") -x[['Date.event']])
}
Time.dif.fun(Dates.test)
## [1] 323 326
2) Convert to POSIXlt, set the year, month and day to the end of the year and subtract. Note that the year component uses years since 1900 and the mon component uses Jan = 0, Feb = 1, ..., Dec = 11. See ?as.POSIXlt for details on these and other components:
lt <- as.POSIXlt(Dates.test$Date.event)
lt$year <- Dates.test$Year - 1900
lt$mon <- 11
lt$mday <- 31
as.numeric(lt - Dates.test$Date.event)
## [1] 323 326
3) Another possibility is:
with(Dates.test, as.numeric(as.Date(paste0(Year, "-12-31")) - as.Date(Date.event)))
## [1] 323 326
You could use the difftime function:
Dates.test$diff_days <- difftime(as.POSIXct(paste0(Dates.test[,2],"-12-31"),format = "%Y-%m-%d", tz = "Europe/London"),Dates.test[,1],unit="days")
You can use ISOdate to build the end of year date, and the difftime(... units='days') to get the days til end of year.
From ?difftime:
Limited arithmetic is available on "difftime" objects: they can be
added or subtracted, and multiplied or divided by a numeric vector.
If you want to do more than the limited arithmetic, just coerce with as.numeric(), but you will have to stick with whatever units you specified.
By convention, you may wish to use the beginning of the next year (midnight on new year's eve) as your endpoint for that year. For example:
Dates.test <- data.frame(
Date.event = as.POSIXct(c("12/2/2000","8/2/2001"),
format = "%d/%m/%Y", tz = "Europe/London")
)
# use data.table::year() to get the year of a date
year <- function(x) as.POSIXlt(x)$year + 1900L
Dates.test$Date.end <- ISOdate(year(Dates.test$Date.event)+1,1,1)
# if you don't want class 'difftime', wrap it in as.numeric(), as in:
Dates.test$Date.diff <- as.numeric(
difftime(Dates.test$Date.end,
Dates.test$Date.event,
units='days')
)
Dates.test
# Date.event Date.end Date.diff
# 1 2000-02-12 2001-01-01 12:00:00 324.5
# 2 2001-02-08 2002-01-01 12:00:00 327.5
The apply() family are basically a clean way of doing for loops, and you should strive for more efficient, vectorized solutions.
I'm stuck on a problem calculating travel dates. I have a data frame of departure dates and return dates.
Departure Return
1 7/6/13 8/3/13
2 7/6/13 8/3/13
3 6/28/13 8/7/13
I want to create and pass a function that will take these dates and form a list of all the days away. I can do this individually by turning each column into dates.
## Turn the departure and return dates into a readable format
Dept <- as.Date(travelDates$Dept, format = "%m/%d/%y")
Retn <- as.Date(travelDates$Retn, format = "%m/%d/%y")
travel_dates <- na.omit(data.frame(dept_dates,retn_dates))
seq(from = travel_dates[1,1], to = travel_dates[1,2], by = 1)
This gives me [1] "2013-07-06" "2013-07-07"... and so on. I want to scale to cover the whole data frame, but my attempts have failed.
Here's one that I thought might work.
days_abroad <- data.frame()
get_days <- function(x,y){
all_days <- seq(from = x, to = y, by =1)
c(days_abroad, all_days)
return(days_abroad)
}
get_days(travel_dates$dept_dates, travel_dates$retn_dates)
I get this error:
Error in seq.Date(from = x, to = y, by = 1) : 'from' must be of length 1
There's probably a lot wrong with this, but what I would really like help on is how to run multiple dates through seq().
Sorry, if this is simple (I'm still learning to think in r) and sorry too for any breaches in etiquette. Thank you.
EDIT: updated as per OP comment.
How about this:
travel_dates[] <- lapply(travel_dates, as.Date, format="%m/%d/%y")
dts <- with(travel_dates, mapply(seq, Departure, Return, by="1 day"))
This produces a list with as many items as you had rows in your initial table. You can then summarize (this will be data.frame with the number of times a date showed up):
data.frame(count=sort(table(Reduce(append, dts)), decreasing=T))
# count
# 2013-07-06 3
# 2013-07-07 3
# 2013-07-08 3
# 2013-07-09 3
# ...
OLD CODE:
The following gets the #days of each trip, rather than a list with the dates.
transform(travel_dates, days_away=Return - Departure + 1)
Which produces:
# Departure Return days_away
# 1 2013-07-06 2013-08-03 29 days
# 2 2013-07-06 2013-08-03 29 days
# 3 2013-06-28 2013-08-07 41 days
If you want to put days_away in a separate list, that is trivial, though it seems more useful to have it as an additional column to your data frame.
I have a vector of dates of the form BW01.68, BW02.68, ... , BW26.10. BW stands for "bi-week", so for example, "BW01.68" represents the first bi-week of the year 1968, and "BW26.10" represents the 26th (and final) bi-week of the year 2010. Using R, how could I convert this vector into actual dates, say, of the form 01-01-1968, 01-15-1968, ... , 12-16-2010? Is there a way for R to know exactly which dates correspond to each bi-week? Thanks for any help!
An alternative solution.
biwks <- c("BW01.68", "BW02.68", "BW26.10")
bw <- substr(biwks,3,4)
yr <- substr(biwks,6,7)
yr <- paste0(ifelse(as.numeric(yr) > 15,"19","20"),yr)
# the %j in the date format is the number of days into the year
as.Date(paste(((as.numeric(bw)-1) * 14) + 1,yr,sep="-"),format="%j-%Y")
#[1] "1968-01-01" "1968-01-15" "2010-12-17"
Though I will note that a 'bi-week' seems a strange measure and I can't be sure that just using 14 day blocks is what is intended in your work.
You can make this code a lot shorter. I have spaced out each step to help understanding but you could finish it off in one (long) line of code.
bw <- c('BW01.68', 'BW02.68','BW26.10','BW22.13')
# the gsub will ensure that bw01.1 the same as bw01.01, bw1.01, or bw1.1
#isolating year no
yearno <- as.numeric(
gsub(
x = bw,
pattern = "BW.*\\.",
replacement = ""
)
)
#isolating and converting bw to no of days
dayno <- 14 * as.numeric(
gsub(
x = bw,
pattern = "BW|\\.[[:digit:]]{1,2}",
replacement = ""
)
)
#cutoff year chosen as 15
yearno <- yearno + 1900
yearno[yearno < 1915] <- yearno[yearno < 1915] + 100
# identifying dates
dates <- as.Date(paste0('01/01/',yearno),"%d/%m/%Y") + dayno
# specifically identifinyg mondays of that week no
mondaydates <- dates - as.numeric(strftime(dates,'%w')) + 1
Output -
> bw
[1] "BW01.68" "BW02.68" "BW26.10" "BW22.13"
> dates
[1] "1968-01-15" "1968-01-29" "2010-12-31" "2013-11-05"
> mondaydates
[1] "1968-01-15" "1968-01-29" "2010-12-27" "2013-11-04"
PS: Just be careful that you're aligned with how bw is measured in your data and whether you're translating it correctly. You should be able to manipulate this to get it to work, for instance you might encounter a bw 27.
I want to correct source activity based on the difference between reference and measurement date and source half life (measured in years). Say I have
ref_date <- as.Date('06/01/08',format='%d/%m/%y')
and a column in my data.frame with the same date format, e.g.,
today <- as.Date(Sys.Date(), format='%d/%m/%y')
I can find the number of years between these dates using the lubridate package
year(today)-year(ref_date)
[1] 5
Is there a function I can use to get a floating point answer today - ref_date = 5.2y, for example?
Yes, of course, use difftime() with an as numeric:
R> as.numeric(difftime(as.Date("2003-04-05"), as.Date("2001-01-01"),
+ unit="weeks"))/52.25
[1] 2.2529
R>
Note that we do have to switch to weeks scaled by 52.25 as there is a bit of ambiguity
there in terms of counting years---a February 29 comes around every 4 years but not every 100th etc.
So you have to define that. difftime() handles all time units up to weeks. Months cannot be done for the same reason of the non-constant 'numerator'.
The lubridate package contains a built-in function, time_length, which can help perform this task.
time_length(difftime(as.Date("2003-04-05"), as.Date("2001-01-01")), "years")
[1] 2.257534
time_length(difftime(as.Date("2017-03-01"), as.Date("2012-03-01")),"years")
[1] 5.00274
Documentation for the lubridate package can be found here.
Inspired by Bryan F, time_length() would work better if using interval object
time_length(interval(as.Date("2003-04-05"), as.Date("2001-01-01")), "years")
[1] -2.257534
time_length(difftime(as.Date("2017-03-01"), as.Date("2012-03-01")),"years")
[1] 5.00274
time_length(interval(as.Date("2017-03-01"), as.Date("2012-03-01")),"years")
[1] -5
You can see if you use interval() to get the time difference and then pass it to time_length(), time_length() would take into account the fact that not all months and years have the same number of days, e.g., the leap year.
Not an exact answer to your question, but the answer from Dirk Eddelbuettel in some situations can produce small errors.
Please, consider the following example:
as.numeric(difftime(as.Date("2012-03-01"), as.Date("2017-03-01"), unit="weeks"))/52.25
[1] -4.992481
The correct answer here should be at least 5 years.
The following function (using lubridate package) will calculate a number of full years between two dates:
# Function to calculate an exact full number of years between two dates
year.diff <- function(firstDate, secondDate) {
yearsdiff <- year(secondDate) - year(firstDate)
monthsdiff <- month(secondDate) - month(firstDate)
daysdiff <- day(secondDate) - day(firstDate)
if ((monthsdiff < 0) | (monthsdiff == 0 & daysdiff < 0)) {
yearsdiff <- yearsdiff - 1
}
yearsdiff
}
You can modify it to calculate a fractional part depending on how you define the number of days in the last (not finished) year.
You can use the function AnnivDates() of the package BondValuation:
R> library('BondValuation')
R> DateIndexes <- unlist(
+ suppressWarnings(
+ AnnivDates("2001-01-01", "2003-04-05", CpY=1)$DateVectors[2]
+ )
+ )
R> names(DateIndexes) <- NULL
R> DateIndexes[length(DateIndexes)] - DateIndexes[1]
[1] 2.257534
Click here for documentation of the package BondValuation.
To get the date difference in years (floating point) you can convert the dates to decimal numbers of Year and calculate then their difference.
#Example Dates
x <- as.Date(c("2001-01-01", "2003-04-05"))
#Convert Date to decimal year:
date2DYear <- function(x) {
as.numeric(format(x,"%Y")) + #Get Year an add
(as.numeric(format(x,"%j")) - 0.5) / #Day of the year divided by
as.numeric(format(as.Date(paste0(format(x,"%Y"), "-12-31")),"%j")) #days of the year
}
diff(date2DYear(x)) #Get the difference in years
#[1] 2.257534
I subtract 0.5 from the day of the year as it is not known if you are at the beginning or the end of the day and %j starts with 1.
I think the difference between 2012-03-01 and 2017-03-01 need not to be 5 Years, as 2012 has 366 days and 2017 365 and 2012-03-01 is on the 61 day of the year and 2017-03-01 on the 60.
x <- as.Date(c("2012-03-01", "2017-03-01"))
diff(date2DYear(x))
#[1] 4.997713
Note that using time_length and interval from lubridate need not come to the same result when you make a cumulative time difference.
library(lubridate)
x <- as.Date(c("2012-01-01", "2012-03-01", "2012-12-31"))
time_length(interval(x[1], x[3]), "years")
#[1] 0.9972678
time_length(interval(x[1], x[2]), "years") +
time_length(interval(x[2], x[3]), "years")
#[1] 0.9995509 #!
diff(date2DYear(x[c(1,3)]))
#[1] 0.9972678
diff(date2DYear(x[c(1,2)])) + diff(date2DYear(x[c(2,3)]))
#[1] 0.9972678
x <- as.Date(c("2013-01-01", "2013-03-01", "2013-12-31"))
time_length(interval(x[1], x[3]), "years")
#[1] 0.9972603
time_length(interval(x[1], x[2]), "years") +
time_length(interval(x[2], x[3]), "years")
#[1] 0.9972603
diff(date2DYear(x[c(1,3)]))
#[1] 0.9972603
diff(date2DYear(x[c(1,2)])) + diff(date2DYear(x[c(2,3)]))
#[1] 0.9972603
Since you are already using lubridate package, you can obtain number of years in floating point using a simple trick:
find number of seconds in one year:
seconds_in_a_year <- as.integer((seconds(ymd("2010-01-01")) - seconds(ymd("2009-01-01"))))
now obtain number of seconds between the 2 dates you desire
seconds_between_dates <- as.integer(seconds(date1) - seconds(date2))
your final answer for number of years in floating points will be
years_between_dates <- seconds_between_dates / seconds_in_a_year
This works for me in R:
# Setting up the first inner while-loop controller, the start of the next water year
NextH2OYear <- as.POSIXlt(firstDate)
NextH2OYear$year <- NextH2OYear$year + 1
NextH2OYear<-as.Date(NextH2OYear)
But this doesn't:
# Setting up the first inner while-loop controller, the start of the next water month
NextH2OMonth <- as.POSIXlt(firstDate)
NextH2OMonth$mon <- NextH2OMonth$mon + 1
NextH2OMonth <- as.Date(NextH2OMonth)
I get this error:
Error in as.Date.POSIXlt(NextH2OMonth) :
zero length component in non-empty POSIXlt structure
Any ideas why? I need to systematically add one year (for one loop) and one month (for another loop) and am comparing the resulting changed variables to values with a class of Date, which is why they are being converted back using as.Date().
Thanks,
Tom
Edit:
Below is the entire section of code. I am using RStudio (version 0.97.306). The code below represents a function that is passed an array of two columns (Date (CLass=Date) and Discharge Data (Class=Numeric) that are used to calculate the monthly averages. So, firstDate and lastDate are class Date and determined from the passed array. This code is adapted from successful code that calculates the yearly averages - there maybe one or two things I still need to change over, but I am prevented from error checking later parts due to the early errors I get in my use of POSIXlt. Here is the code:
MonthlyAvgDischarge<-function(values){
#determining the number of values - i.e. the number of rows
dataCount <- nrow(values)
# Determining first and last dates
firstDate <- (values[1,1])
lastDate <- (values[dataCount,1])
# Setting up vectors for results
WaterMonths <- numeric(0)
class(WaterMonths) <- "Date"
numDays <- numeric(0)
MonthlyAvg <- numeric(0)
# while loop variables
loopDate1 <- firstDate
loopDate2 <- firstDate
# Setting up the first inner while-loop controller, the start of the next water month
NextH2OMonth <- as.POSIXlt(firstDate)
NextH2OMonth$mon <- NextH2OMonth$mon + 1
NextH2OMonth <- as.Date(NextH2OMonth)
# Variables used in the loops
dayCounter <- 0
dischargeTotal <- 0
dischargeCounter <- 1
resultsCounter <- 1
loopCounter <- 0
skipcount <- 0
# Outer while-loop, controls the progression from one year to another
while(loopDate1 <= lastDate)
{
# Inner while-loop controls adding up the discharge for each water year
# and keeps track of day count
while(loopDate2 < NextH2OMonth)
{
if(is.na(values[resultsCounter,2]))
{
# Skip this date
loopDate2 <- loopDate2 + 1
# Skip this value
resultsCounter <- resultsCounter + 1
#Skipped counter
skipcount<-skipcount+1
} else{
# Adding up discharge
dischargeTotal <- dischargeTotal + values[resultsCounter,2]
}
# Adding a day
loopDate2 <- loopDate2 + 1
#Keeping track of days
dayCounter <- dayCounter + 1
# Keeping track of Dicharge position
resultsCounter <- resultsCounter + 1
}
# Adding the results/water years/number of days into the vectors
WaterMonths <- c(WaterMonths, as.Date(loopDate2, format="%mm/%Y"))
numDays <- c(numDays, dayCounter)
MonthlyAvg <- c(MonthlyAvg, round((dischargeTotal/dayCounter), digits=0))
# Resetting the left hand side variables of the while-loops
loopDate1 <- NextH2OMonth
loopDate2 <- NextH2OMonth
# Resetting the right hand side variable of the inner while-loop
# moving it one year forward in time to the next water year
NextH2OMonth <- as.POSIXlt(NextH2OMonth)
NextH2OMonth$year <- NextH2OMonth$Month + 1
NextH2OMonth<-as.Date(NextH2OMonth)
# Resettting vraiables that need to be reset
dayCounter <- 0
dischargeTotal <- 0
loopCounter <- loopCounter + 1
}
WaterMonths <- format(WaterMonthss, format="%mm/%Y")
# Uncomment the line below and return AvgAnnualDailyAvg if you want the water years also
# AvgAnnDailyAvg <- data.frame(WaterYears, numDays, YearlyDailyAvg)
return((MonthlyAvg))
}
Same error occurs in regular R. When doing it line by line, its not a problem, when running it as a script, it it.
Plain R
seq(Sys.Date(), length = 2, by = "month")[2]
seq(Sys.Date(), length = 2, by = "year")[2]
Note that this works with POSIXlt too, e.g.
seq(as.POSIXlt(Sys.Date()), length = 2, by = "month")[2]
mondate.
library(mondate)
now <- mondate(Sys.Date())
now + 1 # date in one month
now + 12 # date in 12 months
Mondate is bit smarter about things like mondate("2013-01-31")+ 1 which gives last day of February whereas seq(as.Date("2013-01-31"), length = 2, by = "month")[2] gives March 3rd.
yearmon If you don't really need the day part then yearmon may be preferable:
library(zoo)
now.ym <- yearmon(Sys.Date())
now.ym + 1/12 # add one month
now.ym + 1 # add one year
ADDED comment on POSIXlt and section on yearmon.
Here is you can add 1 month to a date in R, using package lubridate:
library(lubridate)
x <- as.POSIXlt("2010-01-31 01:00:00")
month(x) <- month(x) + 1
>x
[1] "2010-03-03 01:00:00 PST"
(note that it processed the addition correctly, as 31st of Feb doesn't exist).
Can you perhaps provide a reproducible example? What's in firstDate, and what version of R are you using? I do this kind of manipulation of POSIXlt dates quite often and it seems to work:
Sys.Date()
# [1] "2013-02-13"
date = as.POSIXlt(Sys.Date())
date$mon = date$mon + 1
as.Date(date)
# [1] "2013-03-13"