I'm a PhD student (not that experienced in R), and I'm trying to recode a string variable, called RecordedDate into two separate variables: a Date variable and a Time variable. I am using RStudio.
An example of values are:
8/6/2018 18:56
7/26/2018 10:43
7/28/2018 8:36
I would like to you the first part of the value (example: 08/6/2018) to reformat this into a date variable, and the second part of the value (example: 18:56) into a time variable.
I'm thinking the first step would be to create code that can break this up into two variables, based on some rule. I’m thinking maybe I can separate separate everything before the "space" into the Date variable, and after the "space" in the Time variable. I am not able to figure this out.
Then, I'm looking for code that would change the Date from a "string" variable to a "date" type variable. I’m not sure if this is correct, but I’m thinking something like:
better_date <- as.Date(Date, "%m/%d/%Y")
Finally, then I would like to change theTime variable to a "time" type format (if this exists). Not sure how to do this part either, but something that indicates hours and minutes. This part is less important than getting the date variable.
Two immediate ways:
strsplit() on the white space
The proper ways: parse, and then format back out.
Only 2. will guarantee you do not end up with hour 27 or minute 83 ...
Examples:
R> data <- c("8/6/2018 18:56", "7/26/2018 10:43", "7/28/2018 8:36")
R> strsplit(data, " ")
[[1]]
[1] "8/6/2018" "18:56"
[[2]]
[1] "7/26/2018" "10:43"
[[3]]
[1] "7/28/2018" "8:36"
R>
And:
R> data <- c("8/6/2018 18:56", "7/26/2018 10:43", "7/28/2018 8:36")
R> df <- data.frame(data)
R> df$pt <- anytime::anytime(df$data) ## anytime package used
R> df$time <- format(df$pt, "%H:%M")
R> df$day <- format(df$pt, "%Y-%m-%d")
R> df
data pt time day
1 8/6/2018 18:56 2018-08-06 18:56:00 18:56 2018-08-06
2 7/26/2018 10:43 2018-07-26 10:43:00 10:43 2018-07-26
3 7/28/2018 8:36 2018-07-28 00:00:00 00:00 2018-07-28
R>
I often collect data in a data.frame (or data.table) and then add column by column.
Related
I currently have a data frame with a column for Start.Time (imported from a *.csv file), and the format is in 24 hour format (e.g., 20:00:00 equals 8pm). My goal is to capture observations with a start time in various intervals (e.g., between 9:00:00 and 10:00:00), which also meet other criteria. However, it seems that R sorts this 'character' variable in a way that does not align with how our day goes (e.g., 14:00:00 is considered a lower value than 9:00:00).
For example, below is a line of code that works as intended, where I am capturing observations on two different trail segments, which had a start time between 8:00:00 and 9:00:00.
RLLtoMist8.9<-sum((dataset1$Trail.Segment==52|dataset1$Trail.Segment==55) &
(dataset1$Start.Time>="8:00" & dataset1$Start.Time < "9:00"),
na.rm=TRUE)
RLLtoMist8.9
But, this code below does not work as intended, as R is 'valuing' 9:00:00 as greater than 10:00:00.
RLLtoMist9.10 <-
sum((dataset1$Trail.Segment==52|dataset1$Trail.Segment==55) &
(dataset1$Start.Time>="9:00:00 AM" & dataset1$Start.Time < "10:00:00 AM"),
na.rm=TRUE)
It's certainly true that character types are sorted so that "14:00" is less than "9:00". However R has a datetime class which would sort times correctly once a character representation has been parsed.
a <- as.POSIXct("14:00", format="%H:%M")
b <- as.POSIXct("8:00", format="%H:%M")
# test
> a < b
[1] FALSE
You would be able to convert an entire column with:
dataset1$Start.Time <- as.POSIXct(dataset1$Start.Time, format="%H:%M")
The dates of a and b were the system date at the time of conversion, so if you printed them you would see dates and times in the default format. There are packages, such as chron, that let you use just times, but POSIXt objects have dates and times necessarily. See ?DateTimeClasses. The lubridate package also has an 'interval' class and there exist a difftime function in base-R.
There's also seq.POSIXt and cut.POSIXt functions, either of which could be used to create multiple time or date boundaries for categorical transformations of datetimes.
Using the data.table library:
# convert to data table
dataset1<-data.table(dataset1)
# format to a date format rather that character
dataset1[, Start.Time := as.POSIXct(Start.Time, format="%H:%M:%S")]
#now do your filtering
dataset1[between(Start.Time, as.POSIXct("09:00:00", format="%H:%M:%S"), as.POSIXct("10:00:00", format="%H:%M:%S")) & (Trail.Segment==52 | Trail.Segment==55)]
I would like to extract the time from a table column sd_data$start in R with the following characteristics:
str(sd_data$start)
Factor w/ 122 levels "01/03/2017 08:00",..: 1 2 5 10 12 14 18 19 20 21 ...
I found similar questions on the forum but so far all the answers have only given me NAs or blank values (00:00:00) so I see no other option than raise the question again specifically for my dataset.
I have managed to extract the dates and move them to a new column in the table with little effort and I am very surprised how difficult it is (for me at least) to do the same for hours, minutes and seconds. I must be overlooking something.
sd_data$start_date <- as.Date(sd_data$start,format='%d/%m/%Y')
sd_data$start_time <-
Thanks in advance for helping me to find the right lines of code to complete this task.
Here an example of what I am trying to do and where I am failing to get the time out.
smpldata <- "01/03/2017 08:00"
smpltime <-as.Date(as.character(smpldata),format='%d/%m/%Y %M:%S')
smpltime
# [1] 08:00 = what I would like to see
# [1] "2017-03-01" = what I am seeing
Maybe using as.character() to convert to character before convert to date, because the factor type is not well transformed. And including the other string elements on the date format as suggested above by Sotos.
sd_data$start_date <-
as.Date(as.character(sd_data$start),
format='%d/%m/%Y %H:%M:%S')
Another tip is to take a look at lubridate package. It's very usefull for this kind of task.
library(lubridate)
smpldata <- as.factor("01/03/2017 08:00")
(smpltime <-dmy_hm(as.character(smpldata)))
[1] "2017-03-01 08:00:00 UTC"
Here you still see the date. You can handle just the time for plots and other needs using hour() and minute().
hour(smpltime)
[1] 8
minute(smpltime)
[1] 0
Or you can use the format() function to get exactly what you want.
format(smpltime, "%H:%M:%S")
[1] "08:00:00"
format(smpltime, "%H:%M")
[1] "08:00"
I am trying to do some analysis with a csv file that I have loaded into R. I was doing the following to access specific values via test[[3]][[1]] for example to get the specific value:
test <- read.csv(file = "test.csv")
test <- data.frame(lapply(test, as.character), stringsAsFactors=FALSE)
Otherwise I would have gotten something like this:
> chicago[[3]][[1]]
[1] 08/02/2002 11:00:00 AM
19747 Levels: 01/01/2001 03:49:00 AM 01/01/2001 06:17:00 PM 01/01/2001 12:00:00 AM ... 12/31/2015 11:46:00 AM
Since one column is saving dates I was converting it to POSIXlt.
test[[3]] <- strptime(test[[3]], format='%m/%d/%Y %I:%M:%S %p')
The values are now being changed as expected, for example:
01/28/2004 06:30:00 PM -> 2004-01-28 18:30:00
Trying to access the values now, I realised though that for example test[[3]][[1]] doesn't give the specific date - instead I get a list that contains every second of each row.
Testing a bit around, I found out that the POSIXit type is a bit "different"; meaning the value mentioned above seems to be some kind of list, being like this:
> unlist(unclass(value))
sec min hour mday mon year wday yday isdst zone gmtoff
"0" "0" "11" "2" "7" "102" "5" "213" "1" "CEST" NA
So my question is: is there a way to get values like "2004-01-28 18:30:00" instead of a list about the whole column?
You are making your life too difficult. You can parse to either Date or Datetime for an entire column. No need for lapply.
You (in general) do not want POSIXlt representation. Look into existing package such as my (relatively recent) anytime package (also on CRAN) which even converts from factor for you -- and does not require explicit format strings, origin values or other holdups.
But as your post does not contain a reproducible example I cannot help with more concrete steps.
I'm facing a very minor issue, but somehow can't resolve it.
When I'm importing a csv file that has date, the date is coming in "%Y-%m-%d" format. But I want it to be in "%d-%m-%Y" format. I tried "as.Date" to transform it. But it's not working.
The data structure look like this after importing:
Date Share_Val
21/01/2015 20
22/01/2015 19
23/01/2015 21
24/01/2015 23
25/01/2015 26
But when I'm importing the file by read.csv, the data look like the following:
Date Share_Val
01/21/2015 20
01/22/2015 19
01/23/2015 21
01/24/2015 23
01/25/2015 26
I tried lubridate. But it didn't help.
Sam's result comes exactly the way I wanted. But when I'm trying the following, it's not coming
data$date<-format(as.Date(data$date,"%m/%d/%Y"))
Can anybody please give me any suggestions?
See if this helps. Note the stringsAsFactors. If your Date field is a factor, you will need data$Date <- as.character(data$Date) first
data <- data.frame(Date = c("21/01/2015", "22/01/2015", "23/01/2015",
"24/01/2015", "25/01/2015"), Share_Val=c(20, 19, 21, 23, 26),
stringsAsFactors=F)
format(as.Date(data$Date, "%d/%m/%Y"), "%d-%m-%Y")
[1] "21-01-2015" "22-01-2015" "23-01-2015" "24-01-2015" "25-01-2015"
Too long for a comment.
I think you may be misunderstanding how Dates work in R. A variable (or column) of class Date is stored internally as the number of days since 1970-01-01. When you print a Date variable, it is displayed using the %Y-%m-%d format. The as.Date(...) function converts character to Date. The format=... argument controls how the character string is interpreted, not how the result is displayed, as in:
as.Date("02/05/2015", format="%m/%d/%Y")
# [1] "2015-02-05"
as.Date("02/05/2015", format="%d/%m/%Y")
# [1] "2015-05-02"
So in the first case the string is interpreted as 05 Feb, in the second 02 May. Note that in both cases the result is displayed (printed) in %Y-%m-%d format.
I have chemistry water data taken from a river. Normally, the sample dates were on a Wednesday every two weeks. The data record starts in 1987 and ends in 2013.
Now, I want to re-check if there are any inconsistencies within the data, that is if the samples are really taken every 14 days. For that task I want to use the r function difftime. But I have no idea on how to do that for multiple dates.
Here is some data:
Date Value
1987-04-16 12:00:00 1,5
1987-04-30 12:00:00 1,2
1987-06-25 12:00:00 1,7
1987-07-14 12:00:00 1,3
Can you tell me on how to use the function difftime properly in that case or any other function that does the job. The result should be the number of days between the samplings and/or a true and false for the 14 days.
Thanks to you guys in advance. Any google-fu was to no avail!
Assuming your data.frame is named dd, you'll want to verify that the Date column is being treated as a date. Most times R will read them as a character which gets converted to a factor in a data.frame. If class(df$Date) is "character" or "factor", run
dd$Date<-as.POSIXct(as.character(dd$Date), format="%Y-%m-%d %H:%M:%S")
Then you can so a simple diff() to get the time difference in days
diff(dd$Date)
# Time differences in days
# [1] 14 56 19
# attr(,"tzone")
# [1] ""
so you can check which ones are over 14 days.