dropping the year from a date in R - r

I have data over 3 years that I would like to plot.
However I would like to plot each year side by side.
In order to do this, I'd like to make the date 03/17/2010 become 03/17, so that it lines up with 03/17/2011.
any ideas how to do that in R?
Here is an image of what I'd like it to look like:

R has its own Date representation, which you should use. Once you convert data to Date it is easy to manipulate their format using the format function.
http://www.statmethods.net/input/dates.html
as an example
> d <- as.Date( "2010-03-17" )
> d
[1] "2010-03-17"
> format( d, format="%m/%d")
[1] "03/17"
or with your data style
> format( as.Date("03/17/2010", "%m/%d/%Y"), format="%m/%d")
[1] "03/17"

You can use R's built in style for dates, using as.Date() and format to choose only month and day:
> dates <- c("02/27/92", "02/27/92", "01/14/92", "02/28/92", "02/01/92")
> format(as.Date(dates, "%m/%d/%y"), "%m/%d")
[1] "02/27" "02/27" "01/14" "02/28" "02/01"
For your example, just use your own dates.
I found this out using R's help where the previous was the example:
> ?as.Date
> ?format

Here's my solution:
It involves formatting the date to a string (without year) and then back to a date, which will default all of the dates to the same (current year).
The code and sample input file are below:
Code
# Clear all
rm(list = ls())
# Load the library that reads xls files
library(gdata)
# Get the data in
data = read.csv('Readings.csv')
# Extract each Column
readings = data[,"Reading"]
dates = as.Date(data[,"Reading.Date"])
# Order the data correctly
readings = readings[order(dates)]
dates = dates[order(dates)]
# Calculate the difference between each date (in days) and readings
diff.readings = diff(readings)
diff.dates = as.numeric(diff(dates)) # Convert from days to an integer
# Calculate the usage per reading period
usage.per.period = diff.readings/diff.dates
# Get Every single day between the very first reading and the very last
# seq will create a sequence: first argument is min, second is max, and 3rd is the step size (which in this case is 1 day)
days = seq(min(dates),max(dates), 1)
# This creates an empty vector to get data from the for loop below
usage.per.day = numeric()
# The length of the diff.dates is the number of periods that exist.
for (period in 1:(length(diff.dates))){
# to convert usage.per.period to usage.per.day, we need to replicate the
# value for the number of days in that period. the function rep will
# replicate a number: first argument is the number to replicate, and the
# second number is the number of times to replicate it. the function c will
# concatinate the current vector and the new period, sort of
# like value = value + 6, but with vectors.
usage.per.day = c(usage.per.day, rep(usage.per.period[period], diff.dates[period]))
}
# The for loop above misses out on the last day, so I add that single value manually
usage.per.day[length(usage.per.day)+1] = usage.per.period[period]
# Get the number of readings for each year
years = names(table(format(dates, "%Y")))
# Now break down the usages and the days by year
# list() creates an empty list
usage.per.day.grouped.by.year = list()
year.day = list()
# This defines some colors for plotting, rainbow(n) will give you
colors = rainbow(length(years))
for (year.index in 1:length(years)){
# This is a vector of trues and falses, to say whether a day is in a particular
# year or not
this.year = (days >= as.Date(paste(years[year.index],'/01/01',sep="")) &
days <= as.Date(paste(years[year.index],'/12/31',sep="")))
usage.per.day.grouped.by.year[[year.index]] = usage.per.day[this.year]
# We only care about the month and day, so drop the year
year.day[[year.index]] = as.Date(format(days[this.year], format="%m/%d"),"%m/%d")
# In the first year, we need to set up the whole plot
if (year.index == 1){
# create a png file with file name image.png
png('image.png')
plot(year.day[[year.index]], # x coords
usage.per.day.grouped.by.year[[year.index]], # y coords
"l", # as a line
col=colors[year.index], # with this color
ylim = c(min(usage.per.day),max(usage.per.day)), # this y max and y min
ylab='Usage', # with this lable for y axis
xlab='Date', # with this lable for x axis
main='Usage Over Time') # and this title
}
else {
# After the plot is set up, we just need to add each year
lines(year.day[[year.index]], # x coords
usage.per.day.grouped.by.year[[year.index]], # y coords
col=colors[year.index]) # color
}
}
# add a legend to the whole thing
legend("topright" , # where to put the legend
legend = years, # what the legend names are
lty=c(1,1), # what the symbol should look like
lwd=c(2.5,2.5), # what the symbol should look like
col=colors) # the colors to use for the symbols
dev.off() # save the png to file
Input file
Reading Date,Reading
1/1/10,10
2/1/10,20
3/6/10,30
4/1/10,40
5/7/10,50
6/1/10,60
7/1/10,70
8/1/10,75
9/22/10,80
10/1/10,85
11/1/10,90
12/1/10,95
1/1/11,100
2/1/11,112.9545455
3/1/11,120.1398601
4/1/11,127.3251748
5/1/11,134.5104895
6/1/11,141.6958042
7/1/11,148.8811189
8/1/11,156.0664336
9/17/11,190
10/1/11,223.9335664
11/1/11,257.8671329
12/1/11,291.8006993
1/1/12,325.7342657
2/1/12,359.6678322
3/5/12,375
4/1/12,380
5/1/12,385
6/1/12,390
7/1/12,400
8/1/12,410
9/1/12,420

seasonplot() does this very well!

Related

How to logical compare two data frames in R

library(data.table)
library(QuantTools)
date_from <- '2018-11-01'
date_to <- '2018-11-30'
ticker <- 'SPFB.RTS'
# get days
dataDaily <- get_finam_data(ticker, date_from, date_to, 'day')
# get hours
dataHourly <- get_finam_data(ticker, date_from, date_to, 'hour')
# percent change of the day
dataDaily$pc <- ((dataDaily$close - dataDaily$open)/dataDaily$open)*100
# mark days with > 2 percent change
dataDaily$isBigCh <- dataDaily$pc[dataDaily$pc > 2]
So, I have a code above which downloads a daily/hourly OHLC data of the futures.
Questions:
1) How can I move the marks from dataDaily$isBigCh to dataHourly? It seems not easy because these data frames have different time formats and different lengths of rows.
dataHourly$time # has a format like this 2018-11-09 23:00:00
dataDaily$date # has a format like this 2018-11-09
2) How can I select the first bar of the day in dataHourly$time?
Slightly modified code for readability
# percent change of the day
dataDaily[, price_change := ( close / open - 1 ) * 100 ]
# mark days with > 2 percent change
dataDaily[, isBigCh := price_change > 2 ]
Question 1
# add date column to hourly data
# note that 00:00 time corresponds to 23:00-00:00 candle
dataHourly[, date := as.Date( time - as.difftime( '01:00:00' ) ) ]
# copy dataDaily isBigCh to dataHourly isBigChDaily
dataHourly[ dataDaily, isBigChDaily := isBigCh, on = 'date' ]
Question 2
# select first bar of the day
dataHourly[, .SD[1], by = date ]
Optionally
# remove date column from hourly data
dataHourly[, date := NULL ]
Note
library(data.table) not necessary as QuantTools loads it automatically
please read data.table manual it will save you lots of time trying to figure out simple manipulations similar to what you asked

Data frame of departure and return dates, how do I get a list of all dates away?

I'm stuck on a problem calculating travel dates. I have a data frame of departure dates and return dates.
Departure Return
1 7/6/13 8/3/13
2 7/6/13 8/3/13
3 6/28/13 8/7/13
I want to create and pass a function that will take these dates and form a list of all the days away. I can do this individually by turning each column into dates.
## Turn the departure and return dates into a readable format
Dept <- as.Date(travelDates$Dept, format = "%m/%d/%y")
Retn <- as.Date(travelDates$Retn, format = "%m/%d/%y")
travel_dates <- na.omit(data.frame(dept_dates,retn_dates))
seq(from = travel_dates[1,1], to = travel_dates[1,2], by = 1)
This gives me [1] "2013-07-06" "2013-07-07"... and so on. I want to scale to cover the whole data frame, but my attempts have failed.
Here's one that I thought might work.
days_abroad <- data.frame()
get_days <- function(x,y){
all_days <- seq(from = x, to = y, by =1)
c(days_abroad, all_days)
return(days_abroad)
}
get_days(travel_dates$dept_dates, travel_dates$retn_dates)
I get this error:
Error in seq.Date(from = x, to = y, by = 1) : 'from' must be of length 1
There's probably a lot wrong with this, but what I would really like help on is how to run multiple dates through seq().
Sorry, if this is simple (I'm still learning to think in r) and sorry too for any breaches in etiquette. Thank you.
EDIT: updated as per OP comment.
How about this:
travel_dates[] <- lapply(travel_dates, as.Date, format="%m/%d/%y")
dts <- with(travel_dates, mapply(seq, Departure, Return, by="1 day"))
This produces a list with as many items as you had rows in your initial table. You can then summarize (this will be data.frame with the number of times a date showed up):
data.frame(count=sort(table(Reduce(append, dts)), decreasing=T))
# count
# 2013-07-06 3
# 2013-07-07 3
# 2013-07-08 3
# 2013-07-09 3
# ...
OLD CODE:
The following gets the #days of each trip, rather than a list with the dates.
transform(travel_dates, days_away=Return - Departure + 1)
Which produces:
# Departure Return days_away
# 1 2013-07-06 2013-08-03 29 days
# 2 2013-07-06 2013-08-03 29 days
# 3 2013-06-28 2013-08-07 41 days
If you want to put days_away in a separate list, that is trivial, though it seems more useful to have it as an additional column to your data frame.

Converting Vector into Dates in R

I have a vector of dates of the form BW01.68, BW02.68, ... , BW26.10. BW stands for "bi-week", so for example, "BW01.68" represents the first bi-week of the year 1968, and "BW26.10" represents the 26th (and final) bi-week of the year 2010. Using R, how could I convert this vector into actual dates, say, of the form 01-01-1968, 01-15-1968, ... , 12-16-2010? Is there a way for R to know exactly which dates correspond to each bi-week? Thanks for any help!
An alternative solution.
biwks <- c("BW01.68", "BW02.68", "BW26.10")
bw <- substr(biwks,3,4)
yr <- substr(biwks,6,7)
yr <- paste0(ifelse(as.numeric(yr) > 15,"19","20"),yr)
# the %j in the date format is the number of days into the year
as.Date(paste(((as.numeric(bw)-1) * 14) + 1,yr,sep="-"),format="%j-%Y")
#[1] "1968-01-01" "1968-01-15" "2010-12-17"
Though I will note that a 'bi-week' seems a strange measure and I can't be sure that just using 14 day blocks is what is intended in your work.
You can make this code a lot shorter. I have spaced out each step to help understanding but you could finish it off in one (long) line of code.
bw <- c('BW01.68', 'BW02.68','BW26.10','BW22.13')
# the gsub will ensure that bw01.1 the same as bw01.01, bw1.01, or bw1.1
#isolating year no
yearno <- as.numeric(
gsub(
x = bw,
pattern = "BW.*\\.",
replacement = ""
)
)
#isolating and converting bw to no of days
dayno <- 14 * as.numeric(
gsub(
x = bw,
pattern = "BW|\\.[[:digit:]]{1,2}",
replacement = ""
)
)
#cutoff year chosen as 15
yearno <- yearno + 1900
yearno[yearno < 1915] <- yearno[yearno < 1915] + 100
# identifying dates
dates <- as.Date(paste0('01/01/',yearno),"%d/%m/%Y") + dayno
# specifically identifinyg mondays of that week no
mondaydates <- dates - as.numeric(strftime(dates,'%w')) + 1
Output -
> bw
[1] "BW01.68" "BW02.68" "BW26.10" "BW22.13"
> dates
[1] "1968-01-15" "1968-01-29" "2010-12-31" "2013-11-05"
> mondaydates
[1] "1968-01-15" "1968-01-29" "2010-12-27" "2013-11-04"
PS: Just be careful that you're aligned with how bw is measured in your data and whether you're translating it correctly. You should be able to manipulate this to get it to work, for instance you might encounter a bw 27.

Problems adding a month to X using POSIXlt in R - need to reset value using as.Date(X)

This works for me in R:
# Setting up the first inner while-loop controller, the start of the next water year
NextH2OYear <- as.POSIXlt(firstDate)
NextH2OYear$year <- NextH2OYear$year + 1
NextH2OYear<-as.Date(NextH2OYear)
But this doesn't:
# Setting up the first inner while-loop controller, the start of the next water month
NextH2OMonth <- as.POSIXlt(firstDate)
NextH2OMonth$mon <- NextH2OMonth$mon + 1
NextH2OMonth <- as.Date(NextH2OMonth)
I get this error:
Error in as.Date.POSIXlt(NextH2OMonth) :
zero length component in non-empty POSIXlt structure
Any ideas why? I need to systematically add one year (for one loop) and one month (for another loop) and am comparing the resulting changed variables to values with a class of Date, which is why they are being converted back using as.Date().
Thanks,
Tom
Edit:
Below is the entire section of code. I am using RStudio (version 0.97.306). The code below represents a function that is passed an array of two columns (Date (CLass=Date) and Discharge Data (Class=Numeric) that are used to calculate the monthly averages. So, firstDate and lastDate are class Date and determined from the passed array. This code is adapted from successful code that calculates the yearly averages - there maybe one or two things I still need to change over, but I am prevented from error checking later parts due to the early errors I get in my use of POSIXlt. Here is the code:
MonthlyAvgDischarge<-function(values){
#determining the number of values - i.e. the number of rows
dataCount <- nrow(values)
# Determining first and last dates
firstDate <- (values[1,1])
lastDate <- (values[dataCount,1])
# Setting up vectors for results
WaterMonths <- numeric(0)
class(WaterMonths) <- "Date"
numDays <- numeric(0)
MonthlyAvg <- numeric(0)
# while loop variables
loopDate1 <- firstDate
loopDate2 <- firstDate
# Setting up the first inner while-loop controller, the start of the next water month
NextH2OMonth <- as.POSIXlt(firstDate)
NextH2OMonth$mon <- NextH2OMonth$mon + 1
NextH2OMonth <- as.Date(NextH2OMonth)
# Variables used in the loops
dayCounter <- 0
dischargeTotal <- 0
dischargeCounter <- 1
resultsCounter <- 1
loopCounter <- 0
skipcount <- 0
# Outer while-loop, controls the progression from one year to another
while(loopDate1 <= lastDate)
{
# Inner while-loop controls adding up the discharge for each water year
# and keeps track of day count
while(loopDate2 < NextH2OMonth)
{
if(is.na(values[resultsCounter,2]))
{
# Skip this date
loopDate2 <- loopDate2 + 1
# Skip this value
resultsCounter <- resultsCounter + 1
#Skipped counter
skipcount<-skipcount+1
} else{
# Adding up discharge
dischargeTotal <- dischargeTotal + values[resultsCounter,2]
}
# Adding a day
loopDate2 <- loopDate2 + 1
#Keeping track of days
dayCounter <- dayCounter + 1
# Keeping track of Dicharge position
resultsCounter <- resultsCounter + 1
}
# Adding the results/water years/number of days into the vectors
WaterMonths <- c(WaterMonths, as.Date(loopDate2, format="%mm/%Y"))
numDays <- c(numDays, dayCounter)
MonthlyAvg <- c(MonthlyAvg, round((dischargeTotal/dayCounter), digits=0))
# Resetting the left hand side variables of the while-loops
loopDate1 <- NextH2OMonth
loopDate2 <- NextH2OMonth
# Resetting the right hand side variable of the inner while-loop
# moving it one year forward in time to the next water year
NextH2OMonth <- as.POSIXlt(NextH2OMonth)
NextH2OMonth$year <- NextH2OMonth$Month + 1
NextH2OMonth<-as.Date(NextH2OMonth)
# Resettting vraiables that need to be reset
dayCounter <- 0
dischargeTotal <- 0
loopCounter <- loopCounter + 1
}
WaterMonths <- format(WaterMonthss, format="%mm/%Y")
# Uncomment the line below and return AvgAnnualDailyAvg if you want the water years also
# AvgAnnDailyAvg <- data.frame(WaterYears, numDays, YearlyDailyAvg)
return((MonthlyAvg))
}
Same error occurs in regular R. When doing it line by line, its not a problem, when running it as a script, it it.
Plain R
seq(Sys.Date(), length = 2, by = "month")[2]
seq(Sys.Date(), length = 2, by = "year")[2]
Note that this works with POSIXlt too, e.g.
seq(as.POSIXlt(Sys.Date()), length = 2, by = "month")[2]
mondate.
library(mondate)
now <- mondate(Sys.Date())
now + 1 # date in one month
now + 12 # date in 12 months
Mondate is bit smarter about things like mondate("2013-01-31")+ 1 which gives last day of February whereas seq(as.Date("2013-01-31"), length = 2, by = "month")[2] gives March 3rd.
yearmon If you don't really need the day part then yearmon may be preferable:
library(zoo)
now.ym <- yearmon(Sys.Date())
now.ym + 1/12 # add one month
now.ym + 1 # add one year
ADDED comment on POSIXlt and section on yearmon.
Here is you can add 1 month to a date in R, using package lubridate:
library(lubridate)
x <- as.POSIXlt("2010-01-31 01:00:00")
month(x) <- month(x) + 1
>x
[1] "2010-03-03 01:00:00 PST"
(note that it processed the addition correctly, as 31st of Feb doesn't exist).
Can you perhaps provide a reproducible example? What's in firstDate, and what version of R are you using? I do this kind of manipulation of POSIXlt dates quite often and it seems to work:
Sys.Date()
# [1] "2013-02-13"
date = as.POSIXlt(Sys.Date())
date$mon = date$mon + 1
as.Date(date)
# [1] "2013-03-13"

Extract Date in R

I struggle mightily with dates in R and could do this pretty easily in SPSS, but I would love to stay within R for my project.
I have a date column in my data frame and want to remove the year completely in order to leave the month and day. Here is a peak at my original data.
> head(ds$date)
[1] "2003-10-09" "2003-10-11" "2003-10-13" "2003-10-15" "2003-10-18" "2003-10-20"
> class((ds$date))
[1] "Date"
I "want" it to be.
> head(ds$date)
[1] "10-09" "10-11" "10-13" "10-15" "10-18" "10-20"
> class((ds$date))
[1] "Date"
If possible, I would love to set the first date to be October 1st instead of January 1st.
Any help you can provide will be greatly appreciated.
EDIT: I felt like I should add some context. I want to plot an NHL player's performance over the course of a season which starts in October and ends in April. To add to this, I would like to facet the plots by each season which is a separate column in my data frame. Because I want to compare cumulative performance over the course of the season, I believe that I need to remove the year portion, but maybe I don't; as I indicated, I struggle with dates in R. What I am looking to accomplish is a plot that compares cumulative performance over relative dates by season and have the x-axis start in October and end in April.
> d = as.Date("2003-10-09", format="%Y-%m-%d")
> format(d, "%m-%d")
[1] "10-09"
Is this what you are looking for?
library(ggplot2)
## make up data for two seasons a and b
a = as.Date("2010/10/1")
b = as.Date("2011/10/1")
a.date <- seq(a, by='1 week', length=28)
b.date <- seq(b, by='1 week', length=28)
## make up some score data
a.score <- abs(trunc(rnorm(28, mean = 10, sd = 5)))
b.score <- abs(trunc(rnorm(28, mean = 10, sd = 5)))
## create a data frame
df <- data.frame(a.date, b.date, a.score, b.score)
df
## Since I am using ggplot I better create a "long formated" data frame
df.molt <- melt(df, measure.vars = c("a.score", "b.score"))
levels(df.molt$variable) <- c("First season", "Second season")
df.molt
Then, I am using ggplot2 for plotting the data:
## plot it
ggplot(aes(y = value, x = a.date), data = df.molt) + geom_point() +
geom_line() + facet_wrap(~variable, ncol = 1) +
scale_x_date("Date", format = "%m-%d")
If you want to modify the x-axis (e.g., display format), then you'll probably be interested in scale_date.
You have to remember Date is a numeric format, representing the number of days passed since the "origin" of the internal date counting :
> str(Date)
Class 'Date' num [1:10] 14245 14360 14475 14590 14705 ...
This is the same as in EXCEL, if you want a reference. Hence the solution with format as perfectly valid.
Now if you want to set the first date of a year as October 1st, you can construct some year index like this :
redefine.year <- function(x,start="10-1"){
year <- as.numeric(strftime(x,"%Y"))
yearstart <- as.Date(paste(year,start,sep="-"))
year + (x >= yearstart) - min(year) + 1
}
Testing code :
Start <- as.Date("2009-1-1")
Stop <- as.Date("2011-11-1")
Date <- seq(Start,Stop,length.out=10)
data.frame( Date=as.character(Date),
year=redefine.year(Date))
gives
Date year
1 2009-01-01 1
2 2009-04-25 1
3 2009-08-18 1
4 2009-12-11 2
5 2010-04-05 2
6 2010-07-29 2
7 2010-11-21 3
8 2011-03-16 3
9 2011-07-09 3
10 2011-11-01 4

Resources