Let's assume I have a dataframe with one column - Date - that goes from 2000 to 2019. The problem is that I don't have perfect monthly frequence (in fact I should have 245 observations, instead I only have 215). My aim is to detect what are the missing months in the column.
Let's take this example. This is a sample dataframe:
df <- data.frame(Date = c("2015-01-22", "2015-03-05", "2015-04-15", "2015-06-03", "2015-07-16", "2015-09-03", "2015-10-22", "2015-12-03", "2016-01-21", "2016-03-10", "2016-04-21", "2016-06-02", "2016-07-21", "2016-09-08", "2016-10-20", "2016-12-08", "2017-01-19", "2017-03-09", "2017-04-27", "2017-06-08", "2017-07-20", "2017-09-07", "2017-10-26", "2017-12-14", "2018-01-25", "2018-03-08", "2018-04-26", "2018-06-14", "2018-07-26", "2018-09-13", "2018-10-25", "2018-12-13", "2019-01-24", "2019-03-07", "2019-04-10", "2019-06-06", "2019-07-25", "2019-09-12", "2019-10-24", "2019-12-12"))
df
I would like to find a code that gives me what are the missing months in my column vector of dates.
Can anyone help me?
Thanks a lot
Here are two types of results to see the missing months, with base R:
If you want to see the missing month regardless of years, you can try the following code
missingMonths <- month.name[setdiff(seq(12),as.numeric(format(as.Date(df$Date),"%m")))]
such that
> missingMonths
[1] "February" "May" "August" "November"
If you want to check the missing months by year, you can try the code below:
missingMonths <- lapply(split(df,format(as.Date(df$Date),"%Y")),
function(x) month.name[setdiff(seq(12),as.numeric(format(as.Date(x$Date),"%m")))])
such that
> missingMonths
$`2015`
[1] "February" "May" "August" "November"
$`2016`
[1] "February" "May" "August" "November"
$`2017`
[1] "February" "May" "August" "November"
$`2018`
[1] "February" "May" "August" "November"
$`2019`
[1] "February" "May" "August" "November"
Not as succinct as above, but still does the trick in a couple of steps:
month_date_strings <- unique(paste0(sub("-[^-]+$", "",
sapply(df$Date, as.character)), "-01"))
month_seq_strings <- unique(as.character(seq.Date(as.Date("2000-01-01", "%Y-%m-%d"),
as.Date("2019-12-31", "%Y-%m-%d"), by = "month")))
month_seq_strings[!(month_seq_strings %in% month_date_strings)]
Related
I have a df with a column which has dates stored in character format, for which I want to extract the months. For this I use the following:
mutate(
Date = as.Date(
str_remove(Timestamp, "_.*")
),
Month = month(
Date,
label = F)
)
However, the October, November and December are stored with an extra zero in front of the month. The lubridate library doesn't recognise it. How can I adjust the code above to fix this? This is my Timestamp column:
c("2021-010-01_00h39m", "2021-010-01_01h53m", "2021-010-01_02h36m",
"2021-010-01_10h32m", "2021-010-01_10h34m", "2021-010-01_14h27m"
)
First convert the values to date and use format to get months from it.
format(as.Date(x, '%Y-0%m-%d'), '%b')
#[1] "Oct" "Oct" "Oct" "Oct" "Oct" "Oct"
%b gives abbreviated month name, you may also use %B or %m depending on your choice.
format(as.Date(x, '%Y-0%m-%d'), '%B')
#[1] "October" "October" "October" "October" "October" "October"
format(as.Date(x, '%Y-0%m-%d'), '%m')
#[1] "10" "10" "10" "10" "10" "10"
One way would be use strsplit to extract the second element:
month.abb[readr::parse_number(sapply(strsplit(x, split = '-'), "[[", 2))]
which will return:
#"Oct" "Oct" "Oct" "Oct" "Oct" "Oct"
data:
c("2021-010-01_00h39m", "2021-010-01_01h53m", "2021-010-01_02h36m",
"2021-010-01_10h32m", "2021-010-01_10h34m", "2021-010-01_14h27m"
) -> x
I want to implement a Shiny app that has a dateInput, but I want to restrict the user to select a range of months (e.g. January - June) regardless the year. Is there any possible way?
Sure, here you go:
test_dt <- seq(as.Date('2015-01-01'), as.Date('2015-12-31'), by = "month")
as.character(test_dt, format = "%B")
[1] "January" "February" "March" "April" "May" "June" "July" "August" "September" "October" "November" "December"
Probably you may realize this function through validate... need construction, where you can check that months entered by user are within required range. You can read more here: http://shiny.rstudio.com/reference/shiny/latest/validate.html
In R, it's possible to format POSIXlt date-time objects as a month:
format(Sys.time(), format='%Y-%m')
Is there a way to do the same thing with seasons, or 3-month groups (DJF, MAM, JJA, SON)? These divisions are really common in climatological and ecological science, and it would be great to have a neat way to format them quickly like with months. Obviously DJF falls over 2 years, but for the purposes or this question, that doesn't really matter - just consistently shove them into either year, (or, ideally, it would be good to be able to specify which year they go into).
I'm using the output as a index for by(), so the output format doesn't matter much, just as long as each year/season is unique.
Edit: example data:
dates <- Sys.Date()+seq(1,380, by=35)
dates <- structure(c(16277, 16312, 16347, 16382, 16417, 16452, 16487,
16522, 16557, 16592, 16627), class = "Date")
dates
#[1] "2014-07-26" "2014-08-30" "2014-10-04" "2014-11-08" "2014-12-13"
# "2015-01-17" "2015-02-21" "2015-03-28" "2015-05-02" "2015-06-06" "2015-07-11"
should result in:
c("2014-JJA", "2014-JJA", "2014-SON", "2014-SON", "2015-DJF", "2015-DJF",
"2015-DJF", "2015-MAM", "2015-MAM", "2015-JJA", "2015-JJA")
But the "2015-DJF"s could also be "2014-DJF". Also, the form of the output doesn't matter - "2104q4 or 201404 would also be fine.
as.POSIXlt returns a named list (which makes it unsuitable for data.frame columns). The list columns can be individually accessed and include "year" (1900-based, unlike 1970 used for default) and "mon" (0-based). Best place to see this list in hte help system is ?DateTimeClasses:
First just a Seasons calculation, then a Year-Seasons calculation
c('DJF', 'MAM', 'JJA', 'SON')[ # select from character vector with numeric vector
1+((as.POSIXlt(dates)$mon+1) %/% 3)%%4]
[1] "JJA" "JJA" "SON" "SON" "DJF" "DJF" "DJF" "MAM" "MAM" "JJA"
[11] "JJA"
paste( 1900 + # this is the base year for POSIXlt year numbering
as.POSIXlt( dates )$year +
1*(as.POSIXlt( dates )$year==12) , # offset needed for December
c('DJF', 'MAM', 'JJA', 'SON')[ # indexing from 0-based-mon
1+((as.POSIXlt(dates)$mon+1) %/% 3)%%4]
, sep="-")
[1] "2014-JJA" "2014-JJA" "2014-SON" "2014-SON" "2014-DJF"
[6] "2015-DJF" "2015-DJF" "2015-MAM" "2015-MAM" "2015-JJA"
[11] "2015-JJA"
Shouldn't be that difficult to make a function that constructs the formatting you expect. This is just modulo arithmetic on the POSIXlt values for month and year.
I like using a lookup vector for these sorts of problems, e.g.:
x <- as.POSIXlt(
seq.Date(as.Date("2000-01-01"),as.Date("2002-01-01"),by="2 months")
)
E.g., if you want to specify southern hemisphere seasons, you could do:
src <- rep(c("su","au","wi","sp"),each=3)[c(2:12,1)]
paste(format(x,"%Y-%m"),src[x$mon+1])
# [1] "2000-01 su" "2000-03 au" "2000-05 au" "2000-07 wi" "2000-09 sp"
# [6] "2000-11 sp" "2001-01 su" "2001-03 au" "2001-05 au" "2001-07 wi"
#[11] "2001-09 sp" "2001-11 sp" "2002-01 su"
Change the src names as you see fit to relabel the categories.
Let Q1 be DJF; Q2, MAM; etc. then:
seasonal.quarters <- function(x) {
x <- as.POSIXlt(x)
x$mon <- (x$mon + 1) %% 12
quarters(x)
}
options(stringsAsFactors=FALSE)
nonleap.year <- seq(from=as.POSIXct('2013-1-1'), to=as.POSIXct('2014-1-1'), by='day')
d <- data.frame(ms=months(nonleap.year), qs=seasonal.quarters(nonleap.year))
by(d, INDICES=list(d$qs), FUN=function(x) unique(x$ms))
# : Q1
# [1] "January" "February" "December"
# -------------------------------------
# : Q2
# [1] "March" "April" "May"
# -------------------------------------
# : Q3
# [1] "June" "July" "August"
# -------------------------------------
# : Q4
# [1] "September" "October" "November"
leap.year <- seq(from=as.POSIXct('2016-1-1'), to=as.POSIXct('2017-1-1'), by='day')
d <- data.frame(ms=months(leap.year), qs=seasonal.quarters(leap.year))
by(d, INDICES=list(d$qs), FUN=function(x) unique(x$ms))
# : Q1
# [1] "January" "February" "December"
# -------------------------------------
# : Q2
# [1] "March" "April" "May"
# -------------------------------------
# : Q3
# [1] "June" "July" "August"
# -------------------------------------
# : Q4
# [1] "September" "October" "November"
This is an alternative to 42-'s answer above. Go see my comment there for the reason I posted it.
dates_orig <- as.POSIXlt(c("2013-01-01", "2013-02-01", "2013-03-01", "2013-04-01", "2013-05-01", "2013-06-01", "2013-07-01", "2013-08-01", "2013-09-01", "2013-10-01", "2013-11-01", "2013-12-01", "2014-01-01", "2014-02-01", "2014-03-01", "2014-04-01", "2014-05-01", "2014-06-01", "2014-07-01", "2014-08-01", "2014-09-01", "2014-10-01", "2014-11-01", "2014-12-01"))
format(dates_orig, format='%Y%b')
[1] "2013Jan" "2013Feb" "2013Mar" "2013Apr" "2013May" "2013Jun" "2013Jul" "2013Aug" "2013Sep" "2013Oct" "2013Nov" "2013Dec" "2014Jan" "2014Feb" "2014Mar"
[16] "2014Apr" "2014May" "2014Jun" "2014Jul" "2014Aug" "2014Sep" "2014Oct" "2014Nov" "2014Dec"
dates <- as.POSIXlt(dates_orig)
# shift Jan and Feb to the previous year
dates$year[dates$mon < 2] <- dates$year[dates$mon < 2] - 1
# convert months to seasons (named by first month of season)
dates$mon <- (((dates$mon - 2) %/% 3) %% 4) * 3 + 2
format(dates, format='%Y%b')
[1] "2012Dec" "2012Dec" "2013Mar" "2013Mar" "2013Mar" "2013Jun" "2013Jun" "2013Jun" "2013Sep" "2013Sep" "2013Sep" "2013Dec" "2013Dec" "2013Dec" "2014Mar"
[16] "2014Mar" "2014Mar" "2014Jun" "2014Jun" "2014Jun" "2014Sep" "2014Sep" "2014Sep" "2014Dec"
In R, it's possible to format POSIXlt date-time objects as a month:
format(Sys.time(), format='%Y-%m')
Is there a way to do the same thing with seasons, or 3-month groups (DJF, MAM, JJA, SON)? These divisions are really common in climatological and ecological science, and it would be great to have a neat way to format them quickly like with months. Obviously DJF falls over 2 years, but for the purposes or this question, that doesn't really matter - just consistently shove them into either year, (or, ideally, it would be good to be able to specify which year they go into).
I'm using the output as a index for by(), so the output format doesn't matter much, just as long as each year/season is unique.
Edit: example data:
dates <- Sys.Date()+seq(1,380, by=35)
dates <- structure(c(16277, 16312, 16347, 16382, 16417, 16452, 16487,
16522, 16557, 16592, 16627), class = "Date")
dates
#[1] "2014-07-26" "2014-08-30" "2014-10-04" "2014-11-08" "2014-12-13"
# "2015-01-17" "2015-02-21" "2015-03-28" "2015-05-02" "2015-06-06" "2015-07-11"
should result in:
c("2014-JJA", "2014-JJA", "2014-SON", "2014-SON", "2015-DJF", "2015-DJF",
"2015-DJF", "2015-MAM", "2015-MAM", "2015-JJA", "2015-JJA")
But the "2015-DJF"s could also be "2014-DJF". Also, the form of the output doesn't matter - "2104q4 or 201404 would also be fine.
as.POSIXlt returns a named list (which makes it unsuitable for data.frame columns). The list columns can be individually accessed and include "year" (1900-based, unlike 1970 used for default) and "mon" (0-based). Best place to see this list in hte help system is ?DateTimeClasses:
First just a Seasons calculation, then a Year-Seasons calculation
c('DJF', 'MAM', 'JJA', 'SON')[ # select from character vector with numeric vector
1+((as.POSIXlt(dates)$mon+1) %/% 3)%%4]
[1] "JJA" "JJA" "SON" "SON" "DJF" "DJF" "DJF" "MAM" "MAM" "JJA"
[11] "JJA"
paste( 1900 + # this is the base year for POSIXlt year numbering
as.POSIXlt( dates )$year +
1*(as.POSIXlt( dates )$year==12) , # offset needed for December
c('DJF', 'MAM', 'JJA', 'SON')[ # indexing from 0-based-mon
1+((as.POSIXlt(dates)$mon+1) %/% 3)%%4]
, sep="-")
[1] "2014-JJA" "2014-JJA" "2014-SON" "2014-SON" "2014-DJF"
[6] "2015-DJF" "2015-DJF" "2015-MAM" "2015-MAM" "2015-JJA"
[11] "2015-JJA"
Shouldn't be that difficult to make a function that constructs the formatting you expect. This is just modulo arithmetic on the POSIXlt values for month and year.
I like using a lookup vector for these sorts of problems, e.g.:
x <- as.POSIXlt(
seq.Date(as.Date("2000-01-01"),as.Date("2002-01-01"),by="2 months")
)
E.g., if you want to specify southern hemisphere seasons, you could do:
src <- rep(c("su","au","wi","sp"),each=3)[c(2:12,1)]
paste(format(x,"%Y-%m"),src[x$mon+1])
# [1] "2000-01 su" "2000-03 au" "2000-05 au" "2000-07 wi" "2000-09 sp"
# [6] "2000-11 sp" "2001-01 su" "2001-03 au" "2001-05 au" "2001-07 wi"
#[11] "2001-09 sp" "2001-11 sp" "2002-01 su"
Change the src names as you see fit to relabel the categories.
Let Q1 be DJF; Q2, MAM; etc. then:
seasonal.quarters <- function(x) {
x <- as.POSIXlt(x)
x$mon <- (x$mon + 1) %% 12
quarters(x)
}
options(stringsAsFactors=FALSE)
nonleap.year <- seq(from=as.POSIXct('2013-1-1'), to=as.POSIXct('2014-1-1'), by='day')
d <- data.frame(ms=months(nonleap.year), qs=seasonal.quarters(nonleap.year))
by(d, INDICES=list(d$qs), FUN=function(x) unique(x$ms))
# : Q1
# [1] "January" "February" "December"
# -------------------------------------
# : Q2
# [1] "March" "April" "May"
# -------------------------------------
# : Q3
# [1] "June" "July" "August"
# -------------------------------------
# : Q4
# [1] "September" "October" "November"
leap.year <- seq(from=as.POSIXct('2016-1-1'), to=as.POSIXct('2017-1-1'), by='day')
d <- data.frame(ms=months(leap.year), qs=seasonal.quarters(leap.year))
by(d, INDICES=list(d$qs), FUN=function(x) unique(x$ms))
# : Q1
# [1] "January" "February" "December"
# -------------------------------------
# : Q2
# [1] "March" "April" "May"
# -------------------------------------
# : Q3
# [1] "June" "July" "August"
# -------------------------------------
# : Q4
# [1] "September" "October" "November"
This is an alternative to 42-'s answer above. Go see my comment there for the reason I posted it.
dates_orig <- as.POSIXlt(c("2013-01-01", "2013-02-01", "2013-03-01", "2013-04-01", "2013-05-01", "2013-06-01", "2013-07-01", "2013-08-01", "2013-09-01", "2013-10-01", "2013-11-01", "2013-12-01", "2014-01-01", "2014-02-01", "2014-03-01", "2014-04-01", "2014-05-01", "2014-06-01", "2014-07-01", "2014-08-01", "2014-09-01", "2014-10-01", "2014-11-01", "2014-12-01"))
format(dates_orig, format='%Y%b')
[1] "2013Jan" "2013Feb" "2013Mar" "2013Apr" "2013May" "2013Jun" "2013Jul" "2013Aug" "2013Sep" "2013Oct" "2013Nov" "2013Dec" "2014Jan" "2014Feb" "2014Mar"
[16] "2014Apr" "2014May" "2014Jun" "2014Jul" "2014Aug" "2014Sep" "2014Oct" "2014Nov" "2014Dec"
dates <- as.POSIXlt(dates_orig)
# shift Jan and Feb to the previous year
dates$year[dates$mon < 2] <- dates$year[dates$mon < 2] - 1
# convert months to seasons (named by first month of season)
dates$mon <- (((dates$mon - 2) %/% 3) %% 4) * 3 + 2
format(dates, format='%Y%b')
[1] "2012Dec" "2012Dec" "2013Mar" "2013Mar" "2013Mar" "2013Jun" "2013Jun" "2013Jun" "2013Sep" "2013Sep" "2013Sep" "2013Dec" "2013Dec" "2013Dec" "2014Mar"
[16] "2014Mar" "2014Mar" "2014Jun" "2014Jun" "2014Jun" "2014Sep" "2014Sep" "2014Sep" "2014Dec"
In R, it's possible to format POSIXlt date-time objects as a month:
format(Sys.time(), format='%Y-%m')
Is there a way to do the same thing with seasons, or 3-month groups (DJF, MAM, JJA, SON)? These divisions are really common in climatological and ecological science, and it would be great to have a neat way to format them quickly like with months. Obviously DJF falls over 2 years, but for the purposes or this question, that doesn't really matter - just consistently shove them into either year, (or, ideally, it would be good to be able to specify which year they go into).
I'm using the output as a index for by(), so the output format doesn't matter much, just as long as each year/season is unique.
Edit: example data:
dates <- Sys.Date()+seq(1,380, by=35)
dates <- structure(c(16277, 16312, 16347, 16382, 16417, 16452, 16487,
16522, 16557, 16592, 16627), class = "Date")
dates
#[1] "2014-07-26" "2014-08-30" "2014-10-04" "2014-11-08" "2014-12-13"
# "2015-01-17" "2015-02-21" "2015-03-28" "2015-05-02" "2015-06-06" "2015-07-11"
should result in:
c("2014-JJA", "2014-JJA", "2014-SON", "2014-SON", "2015-DJF", "2015-DJF",
"2015-DJF", "2015-MAM", "2015-MAM", "2015-JJA", "2015-JJA")
But the "2015-DJF"s could also be "2014-DJF". Also, the form of the output doesn't matter - "2104q4 or 201404 would also be fine.
as.POSIXlt returns a named list (which makes it unsuitable for data.frame columns). The list columns can be individually accessed and include "year" (1900-based, unlike 1970 used for default) and "mon" (0-based). Best place to see this list in hte help system is ?DateTimeClasses:
First just a Seasons calculation, then a Year-Seasons calculation
c('DJF', 'MAM', 'JJA', 'SON')[ # select from character vector with numeric vector
1+((as.POSIXlt(dates)$mon+1) %/% 3)%%4]
[1] "JJA" "JJA" "SON" "SON" "DJF" "DJF" "DJF" "MAM" "MAM" "JJA"
[11] "JJA"
paste( 1900 + # this is the base year for POSIXlt year numbering
as.POSIXlt( dates )$year +
1*(as.POSIXlt( dates )$year==12) , # offset needed for December
c('DJF', 'MAM', 'JJA', 'SON')[ # indexing from 0-based-mon
1+((as.POSIXlt(dates)$mon+1) %/% 3)%%4]
, sep="-")
[1] "2014-JJA" "2014-JJA" "2014-SON" "2014-SON" "2014-DJF"
[6] "2015-DJF" "2015-DJF" "2015-MAM" "2015-MAM" "2015-JJA"
[11] "2015-JJA"
Shouldn't be that difficult to make a function that constructs the formatting you expect. This is just modulo arithmetic on the POSIXlt values for month and year.
I like using a lookup vector for these sorts of problems, e.g.:
x <- as.POSIXlt(
seq.Date(as.Date("2000-01-01"),as.Date("2002-01-01"),by="2 months")
)
E.g., if you want to specify southern hemisphere seasons, you could do:
src <- rep(c("su","au","wi","sp"),each=3)[c(2:12,1)]
paste(format(x,"%Y-%m"),src[x$mon+1])
# [1] "2000-01 su" "2000-03 au" "2000-05 au" "2000-07 wi" "2000-09 sp"
# [6] "2000-11 sp" "2001-01 su" "2001-03 au" "2001-05 au" "2001-07 wi"
#[11] "2001-09 sp" "2001-11 sp" "2002-01 su"
Change the src names as you see fit to relabel the categories.
Let Q1 be DJF; Q2, MAM; etc. then:
seasonal.quarters <- function(x) {
x <- as.POSIXlt(x)
x$mon <- (x$mon + 1) %% 12
quarters(x)
}
options(stringsAsFactors=FALSE)
nonleap.year <- seq(from=as.POSIXct('2013-1-1'), to=as.POSIXct('2014-1-1'), by='day')
d <- data.frame(ms=months(nonleap.year), qs=seasonal.quarters(nonleap.year))
by(d, INDICES=list(d$qs), FUN=function(x) unique(x$ms))
# : Q1
# [1] "January" "February" "December"
# -------------------------------------
# : Q2
# [1] "March" "April" "May"
# -------------------------------------
# : Q3
# [1] "June" "July" "August"
# -------------------------------------
# : Q4
# [1] "September" "October" "November"
leap.year <- seq(from=as.POSIXct('2016-1-1'), to=as.POSIXct('2017-1-1'), by='day')
d <- data.frame(ms=months(leap.year), qs=seasonal.quarters(leap.year))
by(d, INDICES=list(d$qs), FUN=function(x) unique(x$ms))
# : Q1
# [1] "January" "February" "December"
# -------------------------------------
# : Q2
# [1] "March" "April" "May"
# -------------------------------------
# : Q3
# [1] "June" "July" "August"
# -------------------------------------
# : Q4
# [1] "September" "October" "November"
This is an alternative to 42-'s answer above. Go see my comment there for the reason I posted it.
dates_orig <- as.POSIXlt(c("2013-01-01", "2013-02-01", "2013-03-01", "2013-04-01", "2013-05-01", "2013-06-01", "2013-07-01", "2013-08-01", "2013-09-01", "2013-10-01", "2013-11-01", "2013-12-01", "2014-01-01", "2014-02-01", "2014-03-01", "2014-04-01", "2014-05-01", "2014-06-01", "2014-07-01", "2014-08-01", "2014-09-01", "2014-10-01", "2014-11-01", "2014-12-01"))
format(dates_orig, format='%Y%b')
[1] "2013Jan" "2013Feb" "2013Mar" "2013Apr" "2013May" "2013Jun" "2013Jul" "2013Aug" "2013Sep" "2013Oct" "2013Nov" "2013Dec" "2014Jan" "2014Feb" "2014Mar"
[16] "2014Apr" "2014May" "2014Jun" "2014Jul" "2014Aug" "2014Sep" "2014Oct" "2014Nov" "2014Dec"
dates <- as.POSIXlt(dates_orig)
# shift Jan and Feb to the previous year
dates$year[dates$mon < 2] <- dates$year[dates$mon < 2] - 1
# convert months to seasons (named by first month of season)
dates$mon <- (((dates$mon - 2) %/% 3) %% 4) * 3 + 2
format(dates, format='%Y%b')
[1] "2012Dec" "2012Dec" "2013Mar" "2013Mar" "2013Mar" "2013Jun" "2013Jun" "2013Jun" "2013Sep" "2013Sep" "2013Sep" "2013Dec" "2013Dec" "2013Dec" "2014Mar"
[16] "2014Mar" "2014Mar" "2014Jun" "2014Jun" "2014Jun" "2014Sep" "2014Sep" "2014Sep" "2014Dec"