R / Rstudio : is there a watch function to continuously monitor some values? - r

I have a lot of different operations running on quite a big dataframe. It starts to be a pain for maintenance, especially with some data being improperly formatted, and I'm looking at some options to make my life easier.
The problem is that at one point in the flow of operations NAs are introduced in several lines, including the id (certainly due to some bad subsetting). Now I cannot find the culprit easily because I have each time to str() it, or to view() it in Rstudio... This takes time and I already did it once without finding the bad operation...
So I'm curious if there is some package answering to this problem or a way to program something "daemon-like", to pop up a warning message when a specific value appears.
A while loop doesn't help, because it evaluates all the statements, and of course at one point the condition is not true and it doesn't print when it stops ...
while(nrow(df[is.na(df$id),]) > 0){
statements OK
breaking statement
other OK statements
}
I'll look for other options but I wanted to ask before...
EDIT : thanks for the useful comments, I'll definitely will look more into those functions. However I tried also to build myself a watch function (see my answer).

Ok, I guess I have finally built something quite like it :
This is a function to source a file line per line until a given condition is met :
watchIt <- function(file,watchexpression,startwatchline){
line <- 1
sourceList <- scan(file = "source_test.R", what="character", sep="\n", blank.lines.skip = FALSE)
maxLines <- length(sourceList)
while(startwatchline > line && maxLines >= line){
cat("l")
eval(parse(text=sourceList[line]))
line <- line+1
cat(line)
cat(" ")
}
while(eval(parse(text=watchexpression)) == FALSE && maxLines >= line){
cat(" L")
eval(parse(text=sourceList[line]))
line <- line+1
cat(line)
cat(" ")
}
if(maxLines <= line) {
cat("End of file reached without condition getting TRUE")
}
else{
cat("Condition evaluated to TRUE on line :")
cat(line)
cat("\n")
cat(sourceList[line])
}
}
So this is how I use it :
watchIt("source_test.R","nrow(df[is.na(df$id),]) > 0",10)
This puts "source_test.R" in a list, each line a new list item, and, starting from line 10, I test if the resultant dataframe as NAs in the id field. The execution stops either when the condition evaluates TRUE or when the end of the list items is reached.
Still I'm waiting for some other/better answers... Also, this is kind of my fourth function I managed to create in R, so I guess there might be ameliorations to be made to it...

Related

How to deal with C stack usage error in R [duplicate]

I'm attempting to run some fairly deep recursive code in R and it keeps giving me this error:
Error: C stack usage is too close to the limit
My output from CStack_info() is:
Cstack_info()
size current direction eval_depth
67108864 8120 1 2
I have plenty of memory on my machine, I'm just trying to figure out how I can increase the CStack for R.
EDIT: Someone asked for a reproducible example. Here's some basic sample code that causes the problem. Running f(1,1) a few times you'll get the error. Note that I've already set --max-ppsize = 500000 and options(expressions=500000) so if you don't set those you might get an error about one of those two things instead. As you can see, the recursion can go pretty deep here and I've got no idea how to get it to work consistently. Thanks.
f <- function(root=1,lambda=1) {
x <- c(0,1);
prob <- c(1/(lambda+1),lambda/(lambda+1));
repeat {
if(root == 0) {
break;
}
else {
child <- sample(x,2,replace=TRUE,prob);
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1) {
child[1] <- f(root=child[1],lambda);
}
if(child[2] == 1 && child[1] == 0) {
child[2] <- f(root=child[2],lambda);
}
}
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1 || child[2] == 1) {
root <- sample(x,1,replace=TRUE,prob);
}
}
return(root)
}
The stack size is an operating system parameter, adjustable per-process (see setrlimit(2)). You can't adjust it from within R as far as I can tell, but you can adjust it from the shell before starting R, with the ulimit command. It works like this:
$ ulimit -s # print default
8192
$ R --slave -e 'Cstack_info()["size"]'
size
8388608
8388608 = 1024 * 8192; R is printing the same value as ulimit -s, but in bytes instead of kilobytes.
$ ulimit -s 16384 # enlarge stack limit to 16 megs
$ R --slave -e 'Cstack_info()["size"]'
size
16777216
To make a permanent adjustment to this setting, add the ulimit command to your shell startup file, so it's executed every time you log in. I can't give more specific directions than that, because it depends on exactly which shell you have and stuff. I also don't know how to do it for logging into a graphical environment (which will be relevant if you're not running R inside a terminal window).
I suspect that, regardless of stack limit, you'll end up with recursions that are too deep. For instance, with lambda = Inf, f(1) leads to an immediate recursion, indefinitely. The depth of the recursion seems to be a random walk, with some probability r of going deeper, 1 - r of finishing the current recursion. By the time you've hit the stack limit, you've made a large number of steps 'deeper'. This implies that r > 1 / 2, and the very large majority of time you'll just continue to recurse.
Also, it seems like it is almost possible to derive an analytic or at least numerical solution even in the face of infinite recursion. One can define p as the probability that f(1) == 1, write implicit expressions for the 'child' states after a single iteration, and equate these with p, and solve. p can then be used as the chance of success in a single draw from a binomial distribution.
This error is not due to memory it is due to recursion. A function is calling itself. This isn't always obvious from examining the definition of only one function. To illustrate the point, here is a minimal example of 2 functions that call each other:
change_to_factor <- function(x){
x <- change_to_character(x)
as.factor(x)
}
change_to_character <- function(x){
x <- change_to_factor(x)
as.character(x)
}
change_to_character("1")
Error: C stack usage 7971600 is too close to the limit
The functions will continue to call each other recursively and will theoretically never complete, even if you increase the limit it will still be exceeded. It is only checks within your system that prevent this from occurring indefinitely and consuming all of the compute resources of your machine. You need to alter the functions to ensure that they won't indefinitely call itself (or each other) recursively.
This happened to me for a completely different reason. I accidentally created a superlong string while combining two columns:
output_table_subset = mutate(big_data_frame,
combined_table = paste0(first_part, second_part, col = "_"))
instead of
output_table_subset = mutate(big_data_frame,
combined_table = paste0(first_part, second_part, sep = "_"))
Took me for ever to figure it out as I never expected the paste to have caused the problem.
I encountered the same problem of receiving the "C stack usage is too close to the limit" error (albeit for another application than the one stated by user2045093 above). I tried zwol's proposal but it didn't work out.
To my own surprise, I could solve the problem by installing the newest version of R for OS X (currently: version 3.2.3) as well as the newest version of R Studio for OS X (currently: 0.99.840), since I am working with R Studio.
Hopefully, this may be of some help to you as well.
One issue here can be that you're calling f inside itself
plop <- function(a = 2){
pouet <- sample(a)
plop(pouet)
}
plop()
Erreur : évaluations trop profondément imbriquées : récursion infinie / options(expressions=) ?
Erreur pendant l'emballage (wrapup) : évaluations trop profondément imbriquées : récursion infinie / options(expressions=) ?
Mine is perhaps a more unique case, but may help the few who have this exact problem:
My case has absolutely nothing to do with space usage, still R gave the:
C stack usage is too close to the limit
I had a defined function which is an upgrade of the base function:
saveRDS()
But,
Accidentally, this defined function was called saveRDS() instead of safe_saveRDS().
Thus, past that definition, when the code got to the line wihch actually uses saveRDS(...) (which calls the original base version, not the upgraded one), it gave the above error and crushed.
So, if you're getting that error when calling some saving function, see if you didn't accidentally run over it.
On Linux, I have permanently increased the size of the stack and memlock memories by doing so :
sudo vi /etc/security/limits.conf
Then, add the following lines at the end of the file.
* soft memlock unlimited
* hard memlock unlimited
* soft stack unlimited
* hard stack unlimited
For everyone's information, I am suddenly running into this with R 3.6.1 on Windows 7 (64-bit). It was not a problem before, and now stack limits seem to be popping up everywhere, when I try to "save(.)" data or even do a "save.image(.)". It's like the serialization is blowing these stacks away.
I am seriously considering dropping back to 3.6.0. Didn't happen there.
I often include a commented-out source("path/to/file/thefile.R") line at the top of an R script, e.g. thefile.R, so I can easily copy-paste this into the terminal to run it. I get this error if I forget to comment out the line, since running the file runs the file, which runs the file, which runs the file, ...
If that is the cause, the solution is simple: comment out the line.
Not sure if we re listing issues here but it happened to me with leaflet().
I was trying to map a dataframe in which a date column was of class POSIXlt.
Changing back to POSIXct solved the issue.
As Martin Morgan wrote... The problem is that you get too deep inside of recursion. If the recursion does not converge at all, you need to break it by your own. I hope this code is going to work, because It is not tested. However at least point should be clear here.
f <- function(root=1,lambda=1,depth=1) {
if(depth > 256){
return(NA)
}
x <- c(0,1);
prob <- c(1/(lambda+1),lambda/(lambda+1));
repeat {
if(root == 0) {
break;
} else {
child <- sample(x,2,replace=TRUE,prob);
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1) {
child[1] <- f(root=child[1],lambda,depth+1);
}
if(child[2] == 1 && child[1] == 0) {
child[2] <- f(root=child[2],lambda,depth+1);
}
}
if(child[1] == NA | child[2] == NA){
return NA;
}
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1 || child[2] == 1) {
root <- sample(x,1,replace=TRUE,prob);
}
}
return(root)
}
If you're using plot_ly check which columns you are passing. It seems that for POSIXdt/ct columns, you have to use as.character() before passing to plotly or you get this exception!
Here is how I encountered this error message. I met this error message when I tried to print a data.table in the console. It turned out it was because I mistakenly made a super super long string (by using collapse in paste() when I shouldn't) in a column.
The package caret has a function called createDataPartition that always results in error when the dataset to be partitioned has more than 1m rows.
Just for your info.
I faced the same issue. This problem won't be solved by reinstalling R or Rstudio or by increasing the stack size. Here is a solution that solved this problem -
If you are sourcing a.R inside b.R and at the same time sourcing b.R inside a.R, then the stack will fill up very fast.
Problem
This is the first file a.R in which b.R is sourced
#---- a.R File -----
source("/b.R")
...
...
#--------------------
This is the second file b.R, in which a.R is sourced
#---- b.R File -----
source("/a.R")
...
...
#--------------------
Solution
Source only one file to avoid the recursive calling of files within each other
#---- a.R File -----
source("/b.R")
...
...
#--------------------
#---- b.R File -----
...
...
#--------------------
OR
#---- a.R File -----
...
...
...
#--------------------
#---- b.R File -----
source("/a.R")
...
...
#--------------------
Another way to cause the same problem:
library(debug)
mtrace(lapply)
The recursive call isn't as obvious here.

How to force a For loop() or lapply() to run with error message in R

On this code when I use for loop or the function lapply I get the following error
"Error in get_entrypoint (debug_port):
Cannot connect R to Chrome. Please retry. "
library(rvest)
library(xml2) #pull html data
library(selectr) #for xpath element
url_stackoverflow_rmarkdown <-
'https://stackoverflow.com/questions/tagged/r-markdown?tab=votes&pagesize=50'
web_page <- read_html(url_stackoverflow_rmarkdown)
questions_per_page <- html_text(html_nodes(web_page, ".page-numbers.current"))[1]
link_questions <- html_attr(html_nodes(web_page, ".question-hyperlink")[1:questions_per_page],
"href")
setwd("~/WebScraping_chrome_print_to_pdf")
for (i in 1:length(link_questions)) {
question_to_pdf <- paste0("https://stackoverflow.com",
link_questions[i])
pagedown::chrome_print(question_to_pdf)
}
Is it possible to build a for loop() or use lapply to repeat the code from where it break? That is, from the last i value without breaking the code?
Many thanks
I edited #Rui Barradas idea of tryCatch().
You can try to do something like below.
The IsValues will get either the link value or bad is.
IsValues <- list()
for (i in 1:length(link_questions)) {
question_to_pdf <- paste0("https://stackoverflow.com",
link_questions[i])
IsValues[[i]] <- tryCatch(
{
message(paste("Converting", i))
pagedown::chrome_print(question_to_pdf)
},
error=function(cond) {
message(paste("Cannot convert", i))
# Choose a return value in case of error
return(i)
})
}
Than, you can rbind your values and extract the bad is:
do.call(rbind, IsValues)[!grepl("\\.pdf$", do.call(rbind, IsValues))]
[1] "3" "5" "19" "31"
You can read more about tryCatch() in this answer.
Based on your example, it looks like you have two errors to contend with. The first error is the one you mention in your question. It is also the most frequent error:
Error in get_entrypoint (debug_port): Cannot connect R to Chrome. Please retry.
The second error arises when there are links in the HTML that return 404:
Failed to generate output. Reason: Failed to open https://lh3.googleusercontent.com/-bwcos_zylKg/AAAAAAAAAAI/AAAAAAAAAAA/AAnnY7o18NuEdWnDEck_qPpn-lu21VTdfw/mo/photo.jpg?sz=32 (HTTP status code: 404)
The key phrase in the first error is "Please retry". As far as I can tell, chrome_print sometimes has issues connecting to Chrome. It seems to be fairly random, i.e. failed connections in one run will be fine in the next, and vice versa. The easiest way to get around this issue is to just keep trying until it connects.
I can't come up with any fix for the second error. However, it doesn't seem to come up very often, so it might make sense to just record it and skip to the next URL.
Using the following code I'm able to print 48 of 50 pages. The only two I can't get to work have the 404 issue I describe above. Note that I use purrr::safely to catch errors. Base R's tryCatch will also work fine, but I find safely to be a little more convient. That said, in the end it's really just a matter of preference.
Also note that I've dealt with the connection error by utilizing repeat within the for loop. R will keep trying to connect to Chrome and print until it is either successful, or some other error pops up. I didn't need it, but you might want to include a counter to set an upper threshold for the number of connection attempts:
quest_urls <- paste0("https://stackoverflow.com", link_questions)
errors <- NULL
safe_print <- purrr::safely(pagedown::chrome_print)
for (qurl in quest_urls){
repeat {
output <- safe_print(qurl)
if (is.null(output$error)) break
else if (grepl("retry", output$error$message)) next
else {errors <- c(errors, `names<-`(output$error$message, qurl)); break}
}
}

How to exit from the script in R

I have a function like this
Squ <- function(x, expected.result){
result <- x*x
if(result != expected.result){
stop("We have some error /n/n")
return(NULL)
}
return(result)
}
I am running below three lines.
Squ(2, 4)
Squ(3, 7)
Squ(4, 16)
Ideally i expected this script will stop in second line i.e.Squ(3, 7) as if condition is true in the function but it didn't stop & runs the script completely. How to write the script to get stop if conditions met true.
Thanks,
Mani
Normally, running R code interactively means you'll keep running new lines of code even when there is an error. If you want the code to stop as soon as there's any kind of error, you should wrap the calls to Squ in a function. You'll have to edit the function if you want to save the values of each test, but it looks like you already know what each value should be and you're trying to find the first error. Using a function will give you the first error message and then halt execution.
test_Squ <- function() {
Squ(2, 4)
Squ(3, 7)
Squ(4, 16)
print("All code ran without throwing an error")
}
test_Squ()
# Error in Squ(3, 7) : We have some error /n/n
Let me know if you have a more specific use case in mind and I'll edit my answer.

Running R script_Readline and Scan does not pause for user input

I have looked at other posts that appeared similar to this question but they have not helped me. This may be just my ignorance of R. Thus I decided to sign up and make my first post on stack-overflow.
I am running an R-script and would like the user to decide either to use one of the two following loops. The code to decide user input looks similar to the one below:
#Define the function
method.choice<-function() {
Method.to.use<-readline("Please enter 'New' for new method and'Old' for old method: ")
while(Method.to.use!="New" && Method.to.use!="Old"){ #Make sure selection is one of two inputs
cat("You have not entered a valid input, try again", "\n")
Method.to.use<-readline("Please enter 'New' for new method and 'Old' for old method: ")
cat("You have selected", Method.to.use, "\n")
}
return(Method.to.use)
}
#Run the function
method.choice()
Then below this I have the two possible choices:
if(Method.to.use=="New") {
for(i in 1:nrow(linelist)){...}
}
if(Method.to.use=="Old"){
for(i in 1:nrow(linelist)){...}
}
My issue is, and what I have read from other posts, is that whether I use "readline", "scan" or "ask", R does not wait for my input. Instead R will use the following lines as the input.
The only way I found that R would pause for input is if the code is all on the same line or if it is run line by line (instead of selecting all the code at once). See example from gtools using "ask":
silly <- function()
{
age <- ask("How old are you? ")
age <- as.numeric(age)
cat("In 10 years you will be", age+10, "years old!\n")
}
This runs with a pause:
silly(); paste("this is quite silly")
This does not wait for input:
silly()
paste("this is quite silly")
Any guidance would be appreciated to ensure I can still run my entire script and have it pause at readline without continuing. I am using R-studio and I have checked that interactive==TRUE.
The only other work-around I found is wrapping my entire script into one main function, which is not ideal for me. This may require me to use <<- to write to my environment.
Thank you in advance.

Error: C stack usage is too close to the limit

I'm attempting to run some fairly deep recursive code in R and it keeps giving me this error:
Error: C stack usage is too close to the limit
My output from CStack_info() is:
Cstack_info()
size current direction eval_depth
67108864 8120 1 2
I have plenty of memory on my machine, I'm just trying to figure out how I can increase the CStack for R.
EDIT: Someone asked for a reproducible example. Here's some basic sample code that causes the problem. Running f(1,1) a few times you'll get the error. Note that I've already set --max-ppsize = 500000 and options(expressions=500000) so if you don't set those you might get an error about one of those two things instead. As you can see, the recursion can go pretty deep here and I've got no idea how to get it to work consistently. Thanks.
f <- function(root=1,lambda=1) {
x <- c(0,1);
prob <- c(1/(lambda+1),lambda/(lambda+1));
repeat {
if(root == 0) {
break;
}
else {
child <- sample(x,2,replace=TRUE,prob);
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1) {
child[1] <- f(root=child[1],lambda);
}
if(child[2] == 1 && child[1] == 0) {
child[2] <- f(root=child[2],lambda);
}
}
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1 || child[2] == 1) {
root <- sample(x,1,replace=TRUE,prob);
}
}
return(root)
}
The stack size is an operating system parameter, adjustable per-process (see setrlimit(2)). You can't adjust it from within R as far as I can tell, but you can adjust it from the shell before starting R, with the ulimit command. It works like this:
$ ulimit -s # print default
8192
$ R --slave -e 'Cstack_info()["size"]'
size
8388608
8388608 = 1024 * 8192; R is printing the same value as ulimit -s, but in bytes instead of kilobytes.
$ ulimit -s 16384 # enlarge stack limit to 16 megs
$ R --slave -e 'Cstack_info()["size"]'
size
16777216
To make a permanent adjustment to this setting, add the ulimit command to your shell startup file, so it's executed every time you log in. I can't give more specific directions than that, because it depends on exactly which shell you have and stuff. I also don't know how to do it for logging into a graphical environment (which will be relevant if you're not running R inside a terminal window).
I suspect that, regardless of stack limit, you'll end up with recursions that are too deep. For instance, with lambda = Inf, f(1) leads to an immediate recursion, indefinitely. The depth of the recursion seems to be a random walk, with some probability r of going deeper, 1 - r of finishing the current recursion. By the time you've hit the stack limit, you've made a large number of steps 'deeper'. This implies that r > 1 / 2, and the very large majority of time you'll just continue to recurse.
Also, it seems like it is almost possible to derive an analytic or at least numerical solution even in the face of infinite recursion. One can define p as the probability that f(1) == 1, write implicit expressions for the 'child' states after a single iteration, and equate these with p, and solve. p can then be used as the chance of success in a single draw from a binomial distribution.
This error is not due to memory it is due to recursion. A function is calling itself. This isn't always obvious from examining the definition of only one function. To illustrate the point, here is a minimal example of 2 functions that call each other:
change_to_factor <- function(x){
x <- change_to_character(x)
as.factor(x)
}
change_to_character <- function(x){
x <- change_to_factor(x)
as.character(x)
}
change_to_character("1")
Error: C stack usage 7971600 is too close to the limit
The functions will continue to call each other recursively and will theoretically never complete, even if you increase the limit it will still be exceeded. It is only checks within your system that prevent this from occurring indefinitely and consuming all of the compute resources of your machine. You need to alter the functions to ensure that they won't indefinitely call itself (or each other) recursively.
This happened to me for a completely different reason. I accidentally created a superlong string while combining two columns:
output_table_subset = mutate(big_data_frame,
combined_table = paste0(first_part, second_part, col = "_"))
instead of
output_table_subset = mutate(big_data_frame,
combined_table = paste0(first_part, second_part, sep = "_"))
Took me for ever to figure it out as I never expected the paste to have caused the problem.
I encountered the same problem of receiving the "C stack usage is too close to the limit" error (albeit for another application than the one stated by user2045093 above). I tried zwol's proposal but it didn't work out.
To my own surprise, I could solve the problem by installing the newest version of R for OS X (currently: version 3.2.3) as well as the newest version of R Studio for OS X (currently: 0.99.840), since I am working with R Studio.
Hopefully, this may be of some help to you as well.
One issue here can be that you're calling f inside itself
plop <- function(a = 2){
pouet <- sample(a)
plop(pouet)
}
plop()
Erreur : évaluations trop profondément imbriquées : récursion infinie / options(expressions=) ?
Erreur pendant l'emballage (wrapup) : évaluations trop profondément imbriquées : récursion infinie / options(expressions=) ?
Mine is perhaps a more unique case, but may help the few who have this exact problem:
My case has absolutely nothing to do with space usage, still R gave the:
C stack usage is too close to the limit
I had a defined function which is an upgrade of the base function:
saveRDS()
But,
Accidentally, this defined function was called saveRDS() instead of safe_saveRDS().
Thus, past that definition, when the code got to the line wihch actually uses saveRDS(...) (which calls the original base version, not the upgraded one), it gave the above error and crushed.
So, if you're getting that error when calling some saving function, see if you didn't accidentally run over it.
On Linux, I have permanently increased the size of the stack and memlock memories by doing so :
sudo vi /etc/security/limits.conf
Then, add the following lines at the end of the file.
* soft memlock unlimited
* hard memlock unlimited
* soft stack unlimited
* hard stack unlimited
For everyone's information, I am suddenly running into this with R 3.6.1 on Windows 7 (64-bit). It was not a problem before, and now stack limits seem to be popping up everywhere, when I try to "save(.)" data or even do a "save.image(.)". It's like the serialization is blowing these stacks away.
I am seriously considering dropping back to 3.6.0. Didn't happen there.
I often include a commented-out source("path/to/file/thefile.R") line at the top of an R script, e.g. thefile.R, so I can easily copy-paste this into the terminal to run it. I get this error if I forget to comment out the line, since running the file runs the file, which runs the file, which runs the file, ...
If that is the cause, the solution is simple: comment out the line.
Not sure if we re listing issues here but it happened to me with leaflet().
I was trying to map a dataframe in which a date column was of class POSIXlt.
Changing back to POSIXct solved the issue.
As Martin Morgan wrote... The problem is that you get too deep inside of recursion. If the recursion does not converge at all, you need to break it by your own. I hope this code is going to work, because It is not tested. However at least point should be clear here.
f <- function(root=1,lambda=1,depth=1) {
if(depth > 256){
return(NA)
}
x <- c(0,1);
prob <- c(1/(lambda+1),lambda/(lambda+1));
repeat {
if(root == 0) {
break;
} else {
child <- sample(x,2,replace=TRUE,prob);
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1) {
child[1] <- f(root=child[1],lambda,depth+1);
}
if(child[2] == 1 && child[1] == 0) {
child[2] <- f(root=child[2],lambda,depth+1);
}
}
if(child[1] == NA | child[2] == NA){
return NA;
}
if(child[1] == 0 && child[2] == 0) {
break;
}
if(child[1] == 1 || child[2] == 1) {
root <- sample(x,1,replace=TRUE,prob);
}
}
return(root)
}
If you're using plot_ly check which columns you are passing. It seems that for POSIXdt/ct columns, you have to use as.character() before passing to plotly or you get this exception!
Here is how I encountered this error message. I met this error message when I tried to print a data.table in the console. It turned out it was because I mistakenly made a super super long string (by using collapse in paste() when I shouldn't) in a column.
The package caret has a function called createDataPartition that always results in error when the dataset to be partitioned has more than 1m rows.
Just for your info.
I faced the same issue. This problem won't be solved by reinstalling R or Rstudio or by increasing the stack size. Here is a solution that solved this problem -
If you are sourcing a.R inside b.R and at the same time sourcing b.R inside a.R, then the stack will fill up very fast.
Problem
This is the first file a.R in which b.R is sourced
#---- a.R File -----
source("/b.R")
...
...
#--------------------
This is the second file b.R, in which a.R is sourced
#---- b.R File -----
source("/a.R")
...
...
#--------------------
Solution
Source only one file to avoid the recursive calling of files within each other
#---- a.R File -----
source("/b.R")
...
...
#--------------------
#---- b.R File -----
...
...
#--------------------
OR
#---- a.R File -----
...
...
...
#--------------------
#---- b.R File -----
source("/a.R")
...
...
#--------------------
Another way to cause the same problem:
library(debug)
mtrace(lapply)
The recursive call isn't as obvious here.

Resources