Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Right now I wrote some script that could analyze the daily dumped file from Hadoop. What I want is to let my R script runs daily at 4AM after new data becomes available. Is there any script from R side or OS side could make this happen automatically?
What I can think of is to leave have another R script idling and keep checking system time to decide to call my script to run, but is this too much? I prefer to have R closed unless necessarily.
OK, I see the answer. Does anyone have experience commenting on the stability between R and Python, in terms of running large scale data processing task.
http://www.thegeekstuff.com/2009/06/15-practical-crontab-examples/
-or better yet-
http://tgmstat.wordpress.com/2013/09/11/schedule-rscript-with-cron/
Those websites should be all you need to get it going. Assuming you are using linux.
you can use this code
Sys.time()
for(period in 1:365){
{
your code here
}
newdate=as.POSIXct("2014-11-14 04:00:00 GMT")+24*60*60*period
Sys.sleep( newdate - Sys.time() )
}
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I'm writing an R code, which calls C++, and C++ functions use a lot of parallel computing based on openMP. This is my first code using openMP and what I saw is that even setting the same C++ random seed, the code never gives the same results.
I read a lot of posts here, where it seems that this is an issue with openMP, but they are all old (between12 to 5 years ago)
I want to know if there are solutions now and if there are published article which explain this problem or/and possible solutions.
Thanks
You need to read up on parallel random number generation. This is not an OpenMP problem, but one that will afflict any use of random numbers in a parallel code.
Start with
Parallel Random Numbers: As Easy as 1, 2, 3 - The Salmons
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
Does anyone know which method of saving data is faster fwrite from data.table or saveWorkbook in openxlsx?
Not quite an answer, but too long for a comment.
The easy comment is: Just try to benchmark your code with bench::mark
library(bench)
...
mark(
data.table::fwrite(data, tempfile()),
openxlsx::saveWorkbook(data, tempfile()),
check = FALSE
)
The slightly longer comment is: Do you just want to have the fastest read/write? Then you might want to look into fst and or qs.
I presented a lightning talk at our last R User Group where I benchmarked different read/write speeds, memory usages, file sizes etc. You find the slides here.
Hope that helps
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am having a larger data set with more than 1 million entries. If I am running scripts it sometimes takes up a while till I get an output. Sometimes it seems that there is no output what so ever, even if I let it run for hours. Is there a way to track the progress of the computation (or maybe just see if it is not stuck)?
1. Start small
Write your analysis script and then test it using trivially small amounts of data. Gradually scale up and see how the runtime increases. The microbenchmark package is great at this. In the example below, I compare the amount of time it takes to run the same function with three different sized chunks of data.
library(microbenchmark)
long_running_function <- function(x) {
for(i in 1:nrow(x)) {
Sys.sleep(0.01)
}
}
microbenchmark(long_running_function(mtcars[1:5,]),
long_running_function(mtcars[1:10,]),
long_running_function(mtcars[1:15,]))
2. Look for functions that provide progress bars
I'm not sure what kind of analysis you're performing, but some packages already have this functionality. For example, ranger gives you more updates than the equivalent RandomForest functions.
3. Write your own progress updates
I regularly add print() or cat() statements to large code blocks to tell me when R has finished running a particular part of my analysis. Functions like txtProgressBar() let you add your own progress bars to functions as well.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 years ago.
Improve this question
I'm new to R and I'm wondering if R has something similar to SAS program where it can store the codes? I need to run the analysis on my data set (update monthly) every month. In SAS, I can just run the program and it will give me the results. I'd like to know if R has something similar to that? Thank you so much!
[Update] Thanks for the answers. R script is what I'm looking for!
Are you just talking about running an R script?? If you have a text file called codefile.R containing R code, then from within an interactive R session source("codefile.R") will run it. Or you can use R CMD BATCH codefile.R from a command line/shell/terminal.
update: Dirk Eddelbuettel points out that Rscript or the littler package are recommended over R CMD BATCH ...
Yes. Just like you can create SAS programs in the enhanced editor, you can create R scripts in R. This process is even easier, in my opinion, when you write your R scripts using the tools that come with R Studio.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I'm new in R
can you give me some example code I need use to read and write tiff image in R
or just list the step
Thanks
Google is going to be your friend when learning R. It's old enough that everything is out there. :)
Tiff Package
To install this, you might want to try :
install.packages("tiff")
But if you want to use it put this at the top of each script
library("tiff")
To read and write I suggest :
writeTIFF(yourdatahere, getwd())
You don't assign a name to it because you are just outputting data at this point. Here is getting a TIFF.
TiffObject <- readTIFF(yourfile)
Make sure your file is in your working directory. You can set your working directory by doing :
setwd(path)
If you need anything else, just comment and I shall help.