I have read a lot of blogs, but I cannot find the answer to my question:
I have a date 2020-25-02 17:45:03 and I would like to convert it to two columns day and time.
hello <- strptime(as.character("2020-25-02 17:42:03"),"%Y-%m-%d %H:%M:%S")
df$day <- as.Date(hello, format = "%Y-%d-%m")
But I also would like df$time. Is it possible ?
dtimes = c("2002-06-09 12:45:40","2003-01-29 09:30:40",
+ "2002-09-04 16:45:40","2002-11-13 20:00:40",
+ "2002-07-07 17:30:40")
> dtparts = t(as.data.frame(strsplit(dtimes,' ')))
> row.names(dtparts) = NULL
> thetimes = chron(dates=dtparts[,1],times=dtparts[,2],
+ format=c('y-m-d','h:m:s'))
> thetimes
[1] (02-06-09 12:45:40) (03-01-29 09:30:40) (02-09-04 16:45:40)
[4] (02-11-13 20:00:40) (02-07-07 17:30:40)
Please see this link
Use function hms in package lubridate.
df <- data.frame(day = as.Date(hello, format = "%Y-%d-%m"))
df$time <- lubridate::hms(sub("^[^ ]*\\b(.*)$", "\\1", hello))
df
# day time
#1 2020-02-25 17H 42M 3S
str(df)
#'data.frame': 1 obs. of 2 variables:
# $ day : Date, format: "2020-02-25"
# $ time:Formal class 'Period' [package "lubridate"] with 6 slots
# .. ..# .Data : num 3
# .. ..# year : num 0
# .. ..# month : num 0
# .. ..# day : num 0
# .. ..# hour : num 17
# .. ..# minute: num 42
Related
This question already has answers here:
Why does unlist() kill dates in R?
(2 answers)
Closed 2 years ago.
I have a list of periods that I would like to convert it to behave like a vector (eventually to add as a column in a data frame).
library(lubridate)
x <- list(ms("09:10"), ms("09:02"), ms("1:10"))
# some_function(x)
# with output
ms(c("09:10", "09:02", "1:10"))
unlist and purrr::flatten don't work in this case since it loses it period properties.
d <- do.call("c", x)
class(d)
[1] "Period"
attr(,"package")
[1] "lubridate"
Or
d <- data.frame(date = do.call("c", x))
str(d)
'data.frame': 3 obs. of 1 variable:
$ date:Formal class 'Period' [package "lubridate"] with 6 slots
.. ..# .Data : num 10 2 10
.. ..# year : num 0 0 0
.. ..# month : num 0 0 0
.. ..# day : num 0 0 0
.. ..# hour : num 0 0 0
.. ..# minute: num 9 9 1
d
date
1 9M 10S
2 9M 2S
3 1M 10S
See here: why does unlist() kill dates in R
I have a dataframe of times looking like this:
library(lubridate)
times <- structure(list(exp1 = c("17:19:04 \r", "17:28:53 \r", "17:38:44 \r"),
exp2 = c("17:22:04 \r", "17:31:53 \r", "17:41:45 \r")),
row.names = c(NA, 3L), class = "data.frame")
I want to convert the times in more convenient date-time objects, which I will do with the hms() helper function from the lubridate package.
Running hms() on one column of my dataframe works like a charm:
hms(times[,1])
[1] "17H 19M 4S" "17H 28M 53S" "17H 38M 44S"
Great, surely I can just apply() on my whole dataframe then.
apply(times, 2, hms)
Which gives a weird dataframe with some integers, definitely not what I am expecting.
What is the right way to convert all of my times dataframe with the hms() function?
library(dplyr)
library(lubridate)
times %>% mutate_all(hms)
#OR
mutate_all(times, hms)
# exp1 exp2
#1 17H 19M 4S 17H 22M 4S
#2 17H 28M 53S 17H 31M 53S
#3 17H 38M 44S 17H 41M 45S
Instead of apply, can use lapply as apply converts to matrix and it wouln't hold the attributes created by hms
library(lubridate)
times[] <- lapply(times, hms)
str(times)
#'data.frame': 3 obs. of 2 variables:
# $ exp1:Formal class 'Period' [package "lubridate"] with 6 slots
# .. ..# .Data : num 4 53 44
# .. ..# year : num 0 0 0
# .. ..# month : num 0 0 0
# .. ..# day : num 0 0 0
# .. ..# hour : num 17 17 17
# .. ..# minute: num 19 28 38
# $ exp2:Formal class 'Period' [package "lubridate"] with 6 slots
# .. ..# .Data : num 4 53 45
# .. ..# year : num 0 0 0
# .. ..# month : num 0 0 0
# .. ..# day : num 0 0 0
# .. ..# hour : num 17 17 17
# .. ..# minute: num 22 31 41
With the devel version of dplyr, we can use mutate with across
library(dplyr)
times %>%
mutate(across(everything(), hms))
I have used lubridate to create a column of time format ####M ##S
df$delay <- minutes(df$finish-Delays$start)
How can I covert this column to just give the number #### in front of minutes?
Thanks for any help.
Take a look at the S4 slots...
library(lubridate)
# create data
df <- data.frame(finish = 20)
Delays <- data.frame(start = 10)
(df$delay <- minutes(df$finish-Delays$start))
[1] "10M 0S"
# take a look at the 'delay' object
str(df$delay)
Formal class 'Period' [package "lubridate"] with 6 slots
..# .Data : num 0
..# year : num 0
..# month : num 0
..# day : num 0
..# hour : num 0
..# minute: num 10
# access the 'minute' slot
df$delay#minute
[1] 10
I am trying to replace NULL values with NAs in a list pulled from an API, but the lengths are different and therefore can't be replaced.
I have tried using the nullToNA function in the toxboot package (found here), but it won't locate the function in R when I try to call it (I don't know if there have been changes to the package which I can't locate or whether it is because the list is not pulled from a MongoDB). I have also tried all the function call checks here . My code is below. Any help?
library(httr)
library(toxboot)
library(RJSONIO)
library(lubridate)
library(xlsx)
library(reshape2)
resUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3010CO3.M"
comUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3020CO3.M"
indUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3035CO3.M"
apiList <- list(resUrl, comUrl, indUrl)
results <- vector("list", length(apiList))
for(i in length(apiList)){
raw <- GET(url = as.character(apiList[i]))
char <- rawToChar(raw$content)
list <- fromJSON(char)
for (j in length(list$series[[1]]$data)){
if (is.null(list$series[[1]]$data[[j]][[2]])== TRUE)
##nullToNA(list$series[[1]]$data[[j]][[2]])
##list$series[1]$data[[j]][[2]] <- NA
else
next
}
##seriesData <- list$series[[1]]$data
unlistResult <- lapply(list, unlist)
##unlistResult <- lapply(seriesData, unlist)
##unlist2 <- lapply(unlistResult,unlist)
##results[[i]] <- unlistResult
results[[i]] <- unlistResult
}
My hashtags have some of the things that I have tried. But there are a few other methods I haven't tried.
I have seen lapply(list, function(x) ifelse (x == "NULL", NA, x)) but haven't had any luck with that eiter.
Try this:
library(httr)
resUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3010CO3.M"
x <- GET(resUrl)
y <- content(x)
str(head(y$series[[1]]$data))
# List of 6
# $ :List of 2
# ..$ : chr "201701"
# ..$ : NULL
# $ :List of 2
# ..$ : chr "201612"
# ..$ : num 6.48
# $ :List of 2
# ..$ : chr "201611"
# ..$ : num 7.42
# $ :List of 2
# ..$ : chr "201610"
# ..$ : num 9.75
# $ :List of 2
# ..$ : chr "201609"
# ..$ : num 12.1
# $ :List of 2
# ..$ : chr "201608"
# ..$ : num 14.3
In this first URL, only the first within $series[[1]]$data contained a NULL. BTW: be clear to distinguish between NULL (the literal) and "NULL" (a character string with 4 letters).
Here are some ways (with various data types) to check for NULLs:
is.null(NULL)
# [1] TRUE
length(NULL)
# [1] 0
Simple enough so far, let's try to list with NULLs:
l <- list(NULL, 1)
is.null(l)
# [1] FALSE
sapply(l, is.null)
# [1] TRUE FALSE
length(l)
# [1] 2
lengths(l)
# [1] 0 1
sapply(l, length)
# [1] 0 1
(The "0" lengths indicate NULLs.) I'll use lengths here:
y$series[[1]]$data <- lapply(y$series[[1]]$data, function(z) { z[ lengths(z) == 0 ] <- NA; z; })
str(head(y$series[[1]]$data))
# List of 6
# $ :List of 2
# ..$ : chr "201701"
# ..$ : logi NA
# $ :List of 2
# ..$ : chr "201612"
# ..$ : num 6.48
# $ :List of 2
# ..$ : chr "201611"
# ..$ : num 7.42
# $ :List of 2
# ..$ : chr "201610"
# ..$ : num 9.75
# $ :List of 2
# ..$ : chr "201609"
# ..$ : num 12.1
# $ :List of 2
# ..$ : chr "201608"
# ..$ : num 14.3
what about experience by parsing/converting strings like "now-1h", "today", "now-3d", "today+30m" in R?
how to recognize and convert string (for example as function's argument) to date_time?
I do not recommend using this code, which is brittle. This is just to demonstrate this is possible:
library(lubridate)
library(stringr)
library(dplyr)
data <- c("now-1h", "today", "now-3d", "today+30m", "-3d")
We're going to use lubridate functions now(), today(), days(), hours(), ... which resemble your input data:
str(now())
# POSIXct[1:1], format: "2017-03-24 12:26:18"
str(today())
# Date[1:1], format: "2017-03-24"
str(now() - days(3))
# POSIXct[1:1], format: "2017-03-21 12:27:11"
str(- days(3))
# Formal class 'Period' [package "lubridate"] with 6 slots
# ..# .Data : num 0
# ..# year : num 0
# ..# month : num 0
# ..# day : num -3
# ..# hour : num 0
# ..# minute: num 0
We're going to have to parse() them as strings, to be able to actually use them, like that:
eval(parse(text = "now() + days(3)"))
# [1] "2017-03-27 12:41:34 CEST"
Now let's parse the input strings with a regex, manipulate them a bit to match lubridate syntax, then eval()uate them:
res <-
str_match(data, "(today|now)?([+-])?(\\d+)?([dhms])?")[, - 1] %>%
apply(1, function(x) {
time_ <- if (is.na(x[1])) NULL else paste0(x[1], "()")
offset_ <- if (any(is.na(x[2:4]))) NULL else paste(x[2],
recode(x[4], "d" = "days(", "h" = "hours(", "m" = "minutes(", "s" = "seconds("),
x[3],
")")
parse(text = paste(time_, offset_))
}) %>%
lapply(eval)
Notice that you get a variety of classes as output (either POSIXct or Date or lubridate::Period):
invisible(lapply(res, function(x) { print(x) ; str(x) }))
# [1] "2017-03-24 11:57:52 CET"
# POSIXct[1:1], format: "2017-03-24 11:57:52"
# [1] "2017-03-24"
# Date[1:1], format: "2017-03-24"
# [1] "2017-03-21 12:57:52 CET"
# POSIXct[1:1], format: "2017-03-21 12:57:52"
# [1] "2017-03-24 00:30:00 UTC"
# POSIXlt[1:1], format: "2017-03-24 00:30:00"
# [1] "-3d 0H 0M 0S"
# Formal class 'Period' [package "lubridate"] with 6 slots
# ..# .Data : num 0
# ..# year : num 0
# ..# month : num 0
# ..# day : num -3
# ..# hour : num 0
# ..# minute: num 0
(What I recommend instead is to pre-process the data with the language that produced it and possesses the right tools for the job, which appears to be Perl).