I want to make a loop that saves the results of the loop to your computer every "x" seconds. For example, consider the following loop:
my_list <- list()
for (i in 1:10000000) {
a_i <- rnorm(1, 100, 100)
my_list[[i]] <- a_i
saveRDS(my_list, "my_list.RDS")
}
I want to make it so that the "saveRDS" command is executed every 33 seconds, thus overwriting the previous version of the file.
I know that the "Sys.time()" function can be used to record time in R and "Sys.sleep()" can be used to pause time - but I am not sure how it can be used to use these functions together to perform this task.
Can someone please show me how to do this?
Thank you!
# Jay.sf : I made this small adjustment based on your answer and everything seems to be working now! Thank you so much!
my_list <- list()
for (i in 1:100000000000) {
a_i = rnorm(1,100,100)
tm <- Sys.time()
my_list[[i]] <- a_i
saveRDS(my_list, "my_list.RDS")
Sys.sleep(5) ## actually 33 secs
}
If you want the loop to do stuff every x seconds, set Sys.sleep at the end, i.e. before the next iteration starts. The time might get longer, if the "stuff" to be done needs long time. Here my proposal:
my_list <- list()
for (i in 1:5) {
tm <- Sys.time()
## do stuff
my_list[[i]] <- tm
saveRDS(my_list, "my_list.RDS")
Sys.sleep(3) ## actually 33 secs
}
readRDS("my_list.RDS")
# [[1]]
# [1] "2022-09-25 19:37:26 CEST"
#
# [[2]]
# [1] "2022-09-25 19:37:29 CEST"
#
# [[3]]
# [1] "2022-09-25 19:37:32 CEST"
#
# [[4]]
# [1] "2022-09-25 19:37:35 CEST"
#
# [[5]]
# [1] "2022-09-25 19:37:38 CEST"
Related
There is a question that has a very similar title (cbind runs very slow) but it does not help me with my problem. I am retrieving 10+ JSON files with 100 variables each and I try to create one big data.frame/table with 1000 columns. In practice, I do not use the very same JSON-file as per the example but different ones. Ideally only the problematic line cx <- cbind(cx, bx) would speed up as the other lines (unlist, as.data.table) work well for me and I would not know what else to use. I know, "cbind is slow" but do I have any alternatives? Ideally with Base R.
library(jsonlite)
library(data.table)
starttime <- Sys.time()
for (i in 1:10) { # loop through all 10 json files
zz <- Sys.time() # measuring the time for each loop
urlx <- "http://mysafeinfo.com/api/data?list=englishmonarchs&format=json"
jsnx <- fromJSON(urlx)
if(i==1) {
ax <- unlist(jsnx)
bx <- as.data.table(ax)
cx <- bx
}
for (j in 1:100) { # loop through all 100 variables in each file
ax <- unlist(jsnx)
bx <- as.data.table(ax)
cx <- cbind(cx, bx) # <---- VERY SLOW ----
}
zz <- round(Sys.time()-zz,1)
print(sprintf("%1.1f", zz))
flush.console()
}
endtime <- Sys.time()
endtime-starttime
This gets slower and slower with more files, here my timings.
[1] "0.7"
[1] "1.3"
[1] "1.3"
[1] "1.6"
[1] "2.1"
[1] "2.2"
[1] "2.5"
[1] "3.2"
[1] "3.4"
[1] "3.5"
I would like to print all the hours: minutes in a day from 00:00 to 23:59.
This part goes beyond the question, but if you want to help me, this is the whole idea:
Once that is done, I would like to calculate all the "curious" times that can be interpreted as serendipities. Patterns like: 00:00, 22:22, 01:10, 12:34, 11:44, and the like.
Later on, I would like to count all the "serendipities", and divide them to the total number of hours to know the probabilities of find a "serendipity" each time a person look at the time on his smartphone.
To be honest, I am pretty lost. There is already some months without coding. For the first part of the problem, I guess that a loop can make the task.
For the second part, an if conditional can probably make it.
For the first part of the problem I have tried loops like this
for(i in x){
for(k in y){
cat(i,":",k, ",")
}
}
For the second, something like
Assuming the digits of the time are ab:cd
if(a==b & a==c & a==d){
print(ab:cd)
TRUE
}
if(a==b & c==d){
print(ab:cd)
TRUE
}
I would like to get the whole list of numbers first. Then, the list of "serendipities", and finally the count of both to make the percentage.
I find interesting how people find patterns in numbers when they look at the time, and I would like to know how probable is to get one of these patterns out of the 24*60 = 1440
I hope I have explained myself. (I used to be better with coding and maths, but after some months, I have forgotten almost everything).
Here's a way to generate the list of all possible times.
h <- seq(from=0, to=23)
m <- seq(from=0, to=59)
h <- sprintf('%02d', h)
m <- sprintf('%02d', m)
df <- data.frame(expand.grid(h, m))
df$times <- paste0(df$Var1, ':', df$Var2)
df <- df[order(df$times), ]
df$times
Partial output
df$times[1:25]
[1] "00:00" "00:01" "00:02" "00:03" "00:04" "00:05" "00:06" "00:07" "00:08"
[10] "00:09" "00:10" "00:11" "00:12" "00:13" "00:14" "00:15" "00:16" "00:17"
[19] "00:18" "00:19" "00:20" "00:21" "00:22" "00:23" "00:24"
Length of variable
dim(df)
[1] 1440 3
We can create a sequence of 1 minute interval starting from 00:00:00 to 23:59:00 and then use format to get output in desired format.
format(seq(as.POSIXct("00:00:00", format = "%T"),
as.POSIXct("23:59:00", format = "%T"), by = "1 min"), "%H:%M")
#[1] "00:00" "00:01" "00:02" "00:03" "00:04" "00:05" "00:06" "00:07" "00:08" "00:09"
# "00:10" "00:11" "00:12" "00:13" "00:14" "00:15" "00:16" "00:17" "00:18" "00:19" ...
Yet another way of doing it:
> result <- character(1440)
> for (i in 0:1439) result[i+1L] <- sprintf("%02d:%02d",
+ i %/% 60,
+ i %% 60
+ )
> head(result)
[1] "00:00" "00:01" "00:02" "00:03" "00:04" "00:05"
> tail(result)
[1] "23:54" "23:55" "23:56" "23:57" "23:58" "23:59"
I am trying to convert a vector of the following form:
data$Time[1:10]
[1] 0:00.00 0:00.01 0:00.02 0:00.03 0:00.04 0:00.05 0:00.06 0:00.07 0:00.08 0:00.09
573394 Levels: 0:00.00 0:00.01 0:00.02 0:00.03 0:00.04 0:00.05 0:00.06 0:00.07 0:00.08 0:00.09 0:00.10 0:00.11 0:00.12 0:00.13 0:00.14 ... 9:59.99
notice that this is a factor form
class(data$Time)
factor
I've tried the following
hms(data$Time[1:10])
[1] "0S" "1S" "2S" "3S" "4S" "5S" "6S" "7S" "8S" "9S"
it sees the 1/100 of a second as a second! same thing for
period_to_seconds(hms(data$Time[1:10]))
[1] 0 1 2 3 4 5 6 7 8 9
I need to be able to extract the time (with the require accuracy) to be able to subtract and calculate periods. Notice that these files will extend to few hours. So a solution that is good for HH:MM:SS.00 will be appreciated
another approach that only works if you have data that is either H M S or M S solely is the following:
Test <- c('03:5.05', '1:03.05.05')
tmp <- strptime(as.character(Test),"%H:%M:%OS")
tmp
[1] NA NA
tmp <- strptime(as.character(Test),"%M:%OS")
tmp
[1] "2016-04-30 00:03:05.05 CDT" "2016-04-30 00:01:03.05 CDT
(The hours had to be removed)
## set option to use digits for seconds
options(digits.secs = 2)
## convert your factor to a string and then to Posix format
tmp <- strptime(as.character(data$Time),'%H:%M:%OS')
## convert it to a numeric (unit seconds)
as.numeric(strftime(tmp,'%OS'))+60*as.numeric(strftime(tmp,'%M'))+60*60*as.numeric(strftime(tmp,'%H'))
There is a ms function in lubridate package to read only the minutes and seconds.
Test <- c('0:00.02', '9:59.99')
library(lubridate)
Test %>% ms() %>% period_to_seconds()
[1] 0.02 599.99
Based on Jorg's answer. I think I was able to solve my problem. The files I am working with extend for few hours (with each point representing 0.01 sec). So I split the vector (data$Time) and applied the MS script for the first 360000 points and the HMS script for what following:
options(digits.secs = 2)
tmp1 <- strptime(as.character(data$Time[1:360000]),"%M:%OS")
tmp2 <- strptime(as.character(data$Time[-(1:360000)]),"%H:%M:%OS")
tmp1_numeric <-as.numeric(strftime(tmp1,'%OS'))+60*as.numeric(strftime(tmp1,'%M'))+60*60*as.numeric(strftime(tmp1,'%H'))
tmp2_numeric <-as.numeric(strftime(tmp2,'%OS'))+60*as.numeric(strftime(tmp2,'%M'))+60*60*as.numeric(strftime(tmp2,'%H'))
tmp_numeric <- c(tmp1_numeric, tmp2_numeric)
I need to write some code in R that builds a string by looping over dates and I cant' seem to find an example of this in my books or by Googling. Basically:
for theDate = 1Jan14 to 31Dec14{
"http://website.com/api/" + theDate
}
I thought about creating an input file that held the dates, but that seems inelegant.Does anybody know of a better solution?
This doesn't consume that much memory and doesn't need the julian function:
start <- as.Date("01-08-14",format="%d-%m-%y")
end <- as.Date("08-09-14",format="%d-%m-%y")
theDate <- start
while (theDate <= end)
{
print(paste0("http://website.com/api/",format(theDate,"%d%b%y")))
theDate <- theDate + 1
}
.
[1] "http://website.com/api/01Aug14"
[1] "http://website.com/api/02Aug14"
[1] "http://website.com/api/03Aug14"
[1] "http://website.com/api/04Aug14"
[1] "http://website.com/api/05Aug14"
[1] "http://website.com/api/06Aug14"
[1] "http://website.com/api/07Aug14"
[1] "http://website.com/api/08Aug14"
[1] "http://website.com/api/09Aug14"
[1] "http://website.com/api/10Aug14"
[1] "http://website.com/api/11Aug14"
[1] "http://website.com/api/12Aug14"
[1] "http://website.com/api/13Aug14"
[1] "http://website.com/api/14Aug14"
[1] "http://website.com/api/15Aug14"
[1] "http://website.com/api/16Aug14"
[1] "http://website.com/api/17Aug14"
[1] "http://website.com/api/18Aug14"
[1] "http://website.com/api/19Aug14"
[1] "http://website.com/api/20Aug14"
[1] "http://website.com/api/21Aug14"
[1] "http://website.com/api/22Aug14"
[1] "http://website.com/api/23Aug14"
[1] "http://website.com/api/24Aug14"
[1] "http://website.com/api/25Aug14"
[1] "http://website.com/api/26Aug14"
[1] "http://website.com/api/27Aug14"
[1] "http://website.com/api/28Aug14"
[1] "http://website.com/api/29Aug14"
[1] "http://website.com/api/30Aug14"
[1] "http://website.com/api/31Aug14"
[1] "http://website.com/api/01Sep14"
[1] "http://website.com/api/02Sep14"
[1] "http://website.com/api/03Sep14"
[1] "http://website.com/api/04Sep14"
[1] "http://website.com/api/05Sep14"
[1] "http://website.com/api/06Sep14"
[1] "http://website.com/api/07Sep14"
[1] "http://website.com/api/08Sep14"
>
You can use
> dates <- seq(as.Date("2014-01-01"), as.Date("2014-12-31"), by=1)
to generate a vector of consecutive days. What you want to do with this is not entirely clear from your pseudo-code, but you can iterate directly over the vector (which is generally not what you want in R)
> for (d in dates) {
# Code goes here.
}
The comment-solution by #Roland will give you a vector of the form:
> paste0("http://website.com/api/", dates)
[1] "http://website.com/api/2014-01-01" "http://website.com/api/2014-01-02"
[3] "http://website.com/api/2014-01-03" "http://website.com/api/2014-01-04"
[5] "http://website.com/api/2014-01-05" "http://website.com/api/2014-01-06"
...
Of course after I ask the question I happen to find this.
days <- seq(from=as.Date('2011-02-01'), to=as.Date("2011-03-02"),by='days' )
for ( i in seq_along(days) )
{
print(paste(days[i],"T12:00:00", sep=""))
}
You could translate your date into julian days and then write a loop based on the julian days.
To convert to julian days you can use the code described here
And then you could write code using the the julian days like:
tmp <- as.POSIXlt("1Jan14", format = "%d%b%y")
strdate <- julian(tmp)
tmp <- as.POSIXlt("31Dec14", format = "%d%b%y")
enddate <- julian(tmp)
for (theDate in strdate:enddate){
paste ("http://website.com/api/", toString(theDate), sep = "")
}
you have to figure out how to convert back. I am not to sure about the julian function. maybe you should also have a look into "yday" of lubridate package.
I'm trying to get a list where each element has a name, by applying a function to each row of a data frame, but can't get the right output.
Assuming this is the function that I want to apply to each row:
format_setup_name <- function(m, v, s) {
a <- list()
a[[paste(m, "machines and", v, s, "GB volumes")]] <- paste(num_machines,num_volumes,vol_size,sep="-")
a
}
If this is the input data frame:
df <- data.frame(m=c(1,2,3), v=c(3,3,3), s=c(15,20,30))
I can't get a list that looks like:
$`1-3-15`
[1] "1 machines and 3 15 GB volumes"
$`2-3-20`
[1] "2 machines and 3 20 GB volumes"
$`3-3-30`
[1] "3 machines and 3 30 GB volumes"
Can someone give me hints how to do it?
Why do I need this? Well, I want to populate selectizeInput in shiny using values coming from the database. Since I'm combining several columns, I need a way to match the selected input with the values.
This is a good use case for setNames which can add the names() attribute to an object, in place. Also, if you use as.list, you can do this in just one line without any looping:
setNames(as.list(paste(df$m, ifelse(df$m == 1, "machine", "machines"), "and", df$v, df$s, "GB volumes")), paste(df$m,df$v,df$s,sep="-"))
# $`1-3-15`
# [1] "1 machine and 3 15 GB volumes"
#
# $`2-3-20`
# [1] "2 machines and 3 20 GB volumes"
#
# $`3-3-30`
# [1] "3 machines and 3 30 GB volumes"
Thomas has already found a pretty neat solution to your problem (and in one line, too!). But I'll just show you how you could have succeeded with the approach you first tried:
# We'll use the same data, this time called "dat" (I avoid calling
# objects `df` because `df` is also a function's name)
dat <- data.frame(m = c(1,2,3), v = c(3,3,3), s = c(15,20,30))
format_setup_name <- function(m, v, s) {
a <- list() # initialize the list, all is well up to here
# But here we'll need a loop to assign in turn each element to the list
for(i in seq_along(m)) {
a[[paste(m[i], v[i], s[i], sep="-")]] <-
paste(m[i], "machines and", v[i], s[i], "GB volumes")
}
return(a)
}
Note that what goes inside the brackets is the name of the element, while what's at the right side of the <- is the content to be assigned, not the inverse as your code was suggesting.
So let's try it:
my.setup <- format_setup_name(dat$m, dat$v, dat$s)
my.setup
# $`1-3-15`
# [1] "1 machines and 3 15 GB volumes"
#
# $`2-3-20`
# [1] "2 machines and 3 20 GB volumes"
#
# $`3-3-30`
# [1] "3 machines and 3 30 GB volumes"
Everything seems nice. Just one thing to note: with the $ operator, you'll need to use single or double quotes to access individual items by their names:
my.setup$"1-3-15" # my.setup$1-3-15 won't work
# [1] "1 machines and 3 15 GB volumes"
my.setup[['1-3-15']] # equivalent
# [1] "1 machines and 3 15 GB volumes"
Edit: lapply version
Since loops have really fallen out of favor, here's a version with lapply:
format_setup_name <- function(m, v, s) {
a <- lapply(seq_along(m), function(i) paste(m[i], "machines and", v[i], s[i], "GB volumes"))
names(a) <- paste(m, v, s, sep="-")
return(a)
}