I have a data frame (dates) that looks like this:
year month start end
2000 06 01 10
2000 06 11 20
2000 06 21 30
I want to create a vector of character strings (one for each row in the data frame) so that each date follows this format:
year month start-end (first row would be 2000 06 01-10).
I've tried using a for loop with the paste function:
titles <- character()
for (i in 1:nrow(dates)){
titles[i] <- paste(dates[i, 1], dates[i,2], dates[i,3], dates[i,4])
}
> titles
[1] "2000 06 01 10" "2000 06 11 20" "2000 06 21 30"
but I can't figure out how to replace the last space with a dash. Is there a way to coerce the paste function into doing this or is there another function I can use?
Thanks for the help
Following your solution, if you just replace
paste(dates[i, 1], dates[i,2], dates[i,3], dates[i,4])
with
paste(dates[i, 1], dates[i,2], paste(dates[i,3], dates[i,4], sep = "-"))
that should work already. This just nests the "-" separating paste within the " " separating paste (default of paste is " ").
A more elegant one-liner would be to use apply:
apply(dates, 1, function(row)paste(row[1], row[2], paste(row[3], row[4], sep = "-")))
[1] "2000 06 01-10" "2000 06 11-20" "2000 06 21-30"
Instead of a loop, you may want to consider:
df$titles <- with(df, paste(year, month, start, end, sep = "-"))
df
# year month start end titles
# 1 2000 06 01 10 2000-06-01-10
# 2 2000 06 11 20 2000-06-11-20
# 3 2000 06 21 30 2000-06-21-30
We can use unite from tidyr:
library(tidyverse)
df %>%
unite("new_date", year:end, sep = " ") %>%
mutate(new_date = sub("\\s(\\d+)$", "-\\1", new_date))
or with two unite's:
df %>%
unite("temp_date", year:start, sep = " ") %>%
unite("new_date", temp_date, end, sep = "-")
Output:
new_date
1 2000 6 1-10
2 2000 6 11-20
3 2000 6 21-30
Related
The following file names were used in a camera trap study. The S number represents the site, P is the plot within a site, C is the camera number within the plot, the first string of numbers is the YearMonthDay and the second string of numbers is the HourMinuteSecond.
file.names <- c( 'S123.P2.C10_20120621_213422.jpg',
'S10.P1.C1_20120622_050148.jpg',
'S187.P2.C2_20120702_023501.jpg')
file.names
Use a combination of str_sub() and str_split() to produce a data frame with columns corresponding to the site, plot, camera, year, month, days, hour, minute, and second for these three file names. So we want to produce code that will create the data frame:
Site
Plot
Camera
Year
Month
Day
Hour
Minute
Second
S123
P2
C10
2012
06
21
21
34
22
S10
P1
C1
2012
06
22
05
01
48
S187
P2
C2
2012
07
02
02
35
01
My codes are below:
file.names %>%
str_sub(start = 1, end = -5) %>%
str_replace_all("_", ".") %>%
str_split(pattern = fixed("."), n = 5)
I have no idea how to split date and time
nms <- c("Site", "Plot", "Camera", "Year", "Month", "Day", "Hour", "Minute", "Second")
library(tidyverse)
data.frame(file.names) %>%
extract(file.names, nms,
'(\\w+)\\.(\\w+)\\.(\\w+)_(\\d{4})(\\d{2})(\\d{2})_(\\d{2})(\\d{2})(\\d{2})')
Site Plot Camera Year Month Day Hour Minute Second
1 S123 P2 C10 2012 06 21 21 34 22
2 S10 P1 C1 2012 06 22 05 01 48
3 S187 P2 C2 2012 07 02 02 35 01
in Base R:
type.convert(strcapture('(\\w+)\\.(\\w+)\\.(\\w+)_(\\d{4})(\\d{2})(\\d{2})_(\\d{2})(\\d{2})(\\d{2})',
file.names, as.list(setNames(character(length(nms)), nms))), as.is = TRUE)
Site Plot Camera Year Month Day Hour Minute Second
1 S123 P2 C10 2012 6 21 21 34 22
2 S10 P1 C1 2012 6 22 5 1 48
3 S187 P2 C2 2012 7 2 2 35 1
This is a specific case where your data is pretty neatly formatted with fields separated by either _ or ., and where the date and time fields have uniform character length. That means you can skip doing regex and instead just split by those delimeters, drop the substrings into a data frame, then separate the date components and the time components by their positions. As is often the case, as a tidyverse solution you're trading writing extra code for it being pretty easy to follow and scale.
library(magrittr)
strsplit(file.names, split = "[._]") %>%
purrr::map_dfr(setNames, c("site", "plot", "camera", "date", "time", "ext")) %>%
tidyr::separate(date, into = c("year", "month", "day"), sep = c(4, 6)) %>%
tidyr::separate(time, into = c("hour", "minute", "second"), sep = c(2, 4)) %>%
dplyr::select(-ext)
#> # A tibble: 3 × 9
#> site plot camera year month day hour minute second
#> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 S123 P2 C10 2012 06 21 21 34 22
#> 2 S10 P1 C1 2012 06 22 05 01 48
#> 3 S187 P2 C2 2012 07 02 02 35 01
The ext column was leftover from the initial string splitting, so you can drop it.
I don't know anything about str_sub or str_split other than the fact that they may be efforts to adapt the sub and strsplit functions to an alternate universe. I just learned base R and have not really seen the need to learn a new syntax. Here's a base solution:
as.POSIXct( sub( "([^_]+[_])(\\d{8})[_](\\d{6})", "\\2 \\4", file.names) , format="%Y%m%d %H%M%S")
[1] "2012-06-21" "2012-06-22" "2012-07-02"
You can real the sub pattern as
1) beginning with the start of the string collect all the non-underscore characters into the first capture group
2) Then get the next 8 digits (if they exist) in a second capture group
3) and everything that follows will be in a third capture group
The substitution is to just return the contents of the second capture group. The conversion to Date values is straightforward. I'm assuming that should be clear from the code, but if not then see ?as.Date.
Here's the rest;
as.POSIXct( sub( "([^_]+[_])(\\d{8})[_](\\d{6})(.+$)", "\\2 \\3", file.names) ,
format="%Y%m%d %H%M%S")
[1] "2012-06-21 21:34:22 PDT" "2012-06-22 05:01:48 PDT" "2012-07-02 02:35:01 PDT"
If you want the break out then convert to POSIXlt and extract the resulting list.
I have a dataframe with monthly data, one column containing the year and one column containing the month. I'd like to combine them into one column with Date format, going from this:
Year Month Data
2020 1 54
2020 2 58
2020 3 78
2020 4 59
To this:
Date Data
2020-01 54
2020-02 58
2020-03 78
2020-04 59
I think you can't represent a Date format in R without showing the day. If you want a character column, like in your example, you can do:
> x <- data.frame(Year = c(2020,2020,2020,2020), Month = c(1,2,3,4), Data = c(54,58,78,59))
> x$Month <- ifelse(nchar(x$Month == 1), paste0(0, x$Month), x$Month) # add 0 behind.
> x$Date <- paste(x$Year, x$Month, sep = '-')
> x
Year Month Data Date
1 2020 01 54 2020-01
2 2020 02 58 2020-02
3 2020 03 78 2020-03
4 2020 04 59 2020-04
> class(x$Date)
[1] "character"
If you want a Date type column you will have to add:
x$Date <- paste0(x$Date, '-01')
x$Date <- as.Date(x$Date, format = '%Y-%m-%d')
x
class(x$Date)
Maybe the simplest way would be to arbitrarily set a day (e.g. 01) to all your dates ? Therefore date intervals would be preserved.
data<-data.frame(Year=c(2020,2020,2020,2020), Month=c(1,2,3,4), Data=c(54,58,78,59))
data$Date<-gsub(" ","",paste(data$Year,"-",data$Month,"-","01"))
data$Date<-as.Date(data$Date,format="%Y-%m-%d")
You can use sprintf -
sprintf('%d-%02d', data$Year, data$Month)
#[1] "2020-01" "2020-02" "2020-03" "2020-04"
I have a dataset (8000 observations) that has a string date variable. I would like to split the variable into StartDt and EndDt of format "%B %d %Y". The variable also spans calendar years eg Dec 30 to Jan 5 2019. I have not had success trying to use the stringr package and manipulate accordingly - appreciate any insights!
Df<-data.frame(Date2=c("Dec 16 to 22 2018","Dec 23 to 29 2018", "Dec 30 to Jan 5 2019"))
Use str_match with regex and capture the values needed from the string. Pattern with ? means they are optional.
#extract the data in a dataframe based on pattern
dat <- as.data.frame(stringr::str_match(Df$Date2, '([A-Za-z]+)\\s(\\d+)\\sto\\s?([A-Za-z]+)?\\s(\\d+)\\s(\\d+)')[, -1])
#Change the columns to respective type
dat <- type.convert(dat, as.is = TRUE)
#Copy the year column
dat$V6 <- dat$V5
#Copy the month column if it is the same
dat$V3[is.na(dat$V3)] <- dat$V1[is.na(dat$V3)]
#Subtract 1 from the year only if the End month is earlier than Start month
dat <- transform(dat, V5 = V5 - as.integer(match(V1, month.abb) > match(V3, month.abb)))
#Create the final result dataframe pasting the values
result <- data.frame(Start = with(dat, paste(V1, V2, V5)),
End = with(dat, paste(V3, V4, V6)))
result
# Start End
#1 Dec 16 2018 Dec 22 2018
#2 Dec 23 2018 Dec 29 2018
#3 Dec 30 2018 Jan 5 2019
#4 Apr 15 2018 May 20 2018
data
Added an additional date ("Apr 15 to May 20 2018") in the input for testing purpose.
Df <- data.frame(Date2=c("Dec 16 to 22 2018","Dec 23 to 29 2018",
"Dec 30 to Jan 5 2019", "Apr 15 to May 20 2018"))
Bit long perhaps, but this will require a few steps, though it does only use vectorized functions:
library(glue)
library(stringr)
Df<-data.frame(Date2=c("Dec 16 to 22 2018","Dec 23 to 29 2018", "Dec 30 to Jan 5 2019"))
## a regular expression to match abbreivated month names:
mnthrx <- paste0( "(?:", paste( month.abb, collapse="|" ), ")" )
## the big regex we will use to match it all:
rx <- glue( "({mnthrx}) (\\d+) to (?:({mnthrx}) )?(\\d+) (\\d+)" )
m <- str_match( Df$Date2, rx )
## The end date:
day2 <- as.integer(m[,5])
month2 <- m[,4]
year2 <- as.integer( m[, ncol(m)])
## The start date:
day1 <- m[,3]
month1 <- m[,2]
year1 <- year2
## if month2 is missing, its because we're in month1 still
j <- is.na(month2)
month2[j] <- month1[j]
month.number1 <- match( month1, month.abb )
month.number2 <- match( month2, month.abb )
## if month2 is smaller than month1, we swapped years:
i.next.year <- month.number2 < month.number1
year1[i.next.year] <- year2[i.next.year]-1
data.frame(
StartDt = paste( month1,day1,year1, sep=" " ),
EndDt = paste( month2,day2,year2, sep=" " )
)
It produces this:
StartDt EndDt
1 Dec 16 2018 Dec 22 2018
2 Dec 23 2018 Dec 29 2018
3 Dec 30 2018 Jan 5 2019
I have a column with date formatted as MM-DD-YYYY, in the Date format.
I want to add 2 columns one which only contains YYYY and the other only contains MM.
How do I do this?
Once again base R gives you all you need, and you should not do this with sub-strings.
Here we first create a data.frame with a proper Date column. If your date is in text format, parse it first with as.Date() or my anytime::anydate() (which does not need formats).
Then given the date creating year and month is simple:
R> df <- data.frame(date=Sys.Date()+seq(1,by=30,len=10))
R> df[, "year"] <- format(df[,"date"], "%Y")
R> df[, "month"] <- format(df[,"date"], "%m")
R> df
date year month
1 2017-12-29 2017 12
2 2018-01-28 2018 01
3 2018-02-27 2018 02
4 2018-03-29 2018 03
5 2018-04-28 2018 04
6 2018-05-28 2018 05
7 2018-06-27 2018 06
8 2018-07-27 2018 07
9 2018-08-26 2018 08
10 2018-09-25 2018 09
R>
If you want year or month as integers, you can wrap as as.integer() around the format.
A base R option would be to remove the substring with sub and then read with read.table
df1[c('month', 'year')] <- read.table(text=sub("-\\d{2}-", ",", df1$date), sep=",")
Or using tidyverse
library(tidyverse)
separate(df1, date, into = c('month', 'day', 'year') %>%
select(-day)
Note: it may be better to convert to datetime class instead of using the string formatting.
df1 %>%
mutate(date =mdy(date), month = month(date), year = year(date))
data
df1 <- data.frame(date = c("05-21-2017", "06-25-2015"))
If I have a date, say "2014-05-13" and I want to calculate the month in decimal, I would do this:
5 + 13/31 = 5.419355
How would it be possible in R to take a vector of dates and turn in it into a "month decimal" vector?
For example:
dates = c("2010-01-24", "2013-04-08", "2014-03-05", "2013-03-08", "2014-02-14",
"2004-01-28", "2006-02-21", "2013-03-28", "2013-04-01", "2006-02-14",
"2006-01-28", "2014-01-19", "2012-03-12", "2014-01-30", "2005-04-17")
library(lubridate)
month(dates) + day(dates)/31
As you can see, it would be wrong to put "31" as the diviser since the number of days differ depending on the month, and sometimes year (leap years).
So what would be the best solution?
You can use monthDaysfunction from Hmisc package
> require(Hmisc)
> library(lubridate)
> month(dates) + day(dates)/monthDays(dates)
[1] 1.774194 4.266667 3.161290 3.258065 2.500000 1.903226 2.750000 3.903226 4.033333
[10] 2.500000 1.903226 1.612903 3.387097 1.967742 4.566667
With magrittr,
library(magrittr)
library(lubridate)
dates %>% ymd() %>% { month(.) + day(.) / days_in_month(.) }
## Jan Apr Mar Mar Feb Jan Feb Mar Apr Feb Jan
## 1.774194 4.266667 3.161290 3.258065 2.500000 1.903226 2.750000 3.903226 4.033333 2.500000 1.903226
## Jan Mar Jan Apr
## 1.612903 3.387097 1.967742 4.566667
For some reason the vector gets named, so add %>% unname() if you like.
Here is a base R hack that uses a trick I've seen on SO to get the first day of the next month and subtract 1 to return the last day of the month of interest.
# format dates to Date class
dates <- as.Date(dates)
# get the next month
nextMonths <- as.integer(substr(dates, 6, 7)) + 1L
# replace next month with 1 if it is equal to 13
nextMonths[nextMonths == 13] <- 1L
# extract the number of days using date formatting (%d), paste, and subtraction
dayCount <- as.integer(format(as.Date(paste(substr(dates, 1, 4),
nextMonths, "01", sep="-"))-1L, format="%d"))
dayCount
[1] 31 30 31 31 28 31 28 31 30 28 31 31 31 31 30
# get month with fraction using date formatting (%m)
as.integer(format(dates, format="%m")) + (as.integer(format(dates, format="%d")) / dayCount)
[1] 1.774194 4.266667 3.161290 3.258065 2.500000 1.903226 2.750000 3.903226 4.033333 2.500000
[11] 1.903226 1.612903 3.387097 1.967742 4.566667