Intermittent errors with ggplot2, segfault: memory not mapped - r

Question

I've been having an issue in a Shiny R application where I am getting intermittent errors when the application is supposed to render a ggplot2 plot.
The errors I've been seeing include:
Error in eval(expr, envir, enclos) :
arguments imply differing number of rows: 136, 129
and
Error in eval(expr, envir, enclos) :
Results must be all atomic, or all data frames
I'm seeing the same errors in my Shiny app.
After lots of searching, I found a recent post here that seems to reproduce the same set of errors:
http://cowboyjob.com/post/6523856/Reupping-my-question-from-a-few-hours-ago-I-have-word-from-another-channel-that
I also found this question, which may be related: Tracing root cause for R segfault
The code in that first link is as follows:
library(ggplot2)
set.seed(12345678)
sessionInfo()
littledata = data.frame(x=1:128, y=rlnorm(128))
bigdata = data.frame(x=1:129, y=rlnorm(129))
# plots as expected
lp = ggplot(littledata, aes(x, y))+geom_histogram(stat="identity", binwidth=1)
for (i in 1:20){
print(i)
try(print(lp+ggtitle(paste("128 points", i))))
}
# always warns "position_stack requires constant width",
# intermittently throws error, hangs, or segfaults.
# See below for details.
bp = ggplot(bigdata, aes(x, y))+geom_histogram(stat="identity", binwidth=1)
for(i in 1:20){
print(i)
try(print(bp+ggtitle(paste("129 points", i))))
}
## End demo code
For the machine I'm working on, here is the output from that code block:
> source('~/R/testing/segfault_test.R')
[1] 1
[1] 2
[1] 3
[1] 4
[1] 5
[1] 6
[1] 7
[1] 8
[1] 9
[1] 10
[1] 11
[1] 12
[1] 13
[1] 14
[1] 15
[1] 16
[1] 17
[1] 18
[1] 19
[1] 20
[1] 1
Error in eval(expr, envir, enclos) : replacement has length zero
In addition: Warning messages:
1: package ‘ggplot2’ was built under R version 3.1.2
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 2
[1] 3
[1] 4
[1] 5
Error in eval(expr, envir, enclos) :
Results must be all atomic, or all data frames
In addition: Warning messages:
1: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
3: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
4: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 6
[1] 7
Error in eval(expr, envir, enclos) :
arguments imply differing number of rows: 136, 129
In addition: Warning messages:
1: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 8
Error in eval(expr, envir, enclos) :
Results must be all atomic, or all data frames
In addition: Warning message:
In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 9
[1] 10
[1] 11
Error in eval(expr, envir, enclos) :
arguments imply differing number of rows: 136, 129
In addition: Warning messages:
1: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
3: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 12
[1] 13
Error in eval(expr, envir, enclos) :
Results must be all atomic, or all data frames
In addition: Warning messages:
1: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 14
[1] 15
Error in eval(expr, envir, enclos) :
arguments imply differing number of rows: 136, 129
In addition: Warning messages:
1: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 16
[1] 17
[1] 18
Error in eval(expr, envir, enclos) :
arguments imply differing number of rows: 136, 129
In addition: Warning messages:
1: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
3: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
[1] 19
[1] 20
Warning messages:
1: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
2: In loop_apply(n, do.ply) :
position_stack requires constant width: output may be incorrect
>
From this output, we can see that the first loop runs without any issues; however, the second loop with 129 points gives errors on some of the plots.
When I embed several ggplot plots into my shiny app, I'm seeing these errors show-up, often resulting in a segmentation fault.
Here's what I see in the javascript console when running the app on the shiny-server:
Error in eval(substitute(expr), envir, enclos) :
Results must be all atomic, or all data frames
*** caught segfault ***
address 0x2, cause 'memory not mapped'
Traceback:
1: dim(x)
2: FUN(X[[211L]], ...)
3: lapply(res, NROW)
4: unlist(lapply(res, NROW))
5: list_to_dataframe(res, attr(.data, "split_labels"), .id, id_as_factor)
6: ldply(.data = pieces, .fun = .fun, ..., .progress = .progress, .inform = .inform, .parallel = .parallel, .paropts = .paropts)
7: ddply(munched, .(group), function(df) { data.frame(solid = identical(unique(df$linetype), 1), constant = nrow(unique(df[, c("alpha", "colour", "size", "linetype")])) == 1)})
8: get(x, envir = this, inherits = inh)(this, ...)
9: GeomPath$draw(data, scales, coordinates, arrow, ...)
10: get(x, envir = this, inherits = inh)(this, ...)
11: .$draw(...)
12: get(x, envir = this, inherits = inh)(this, ...)
13: (function (...) get(x, envir = this, inherits = inh)(this, ...))(data = data, scales = scales, coordinates = cs)
14: do.call(.$geom$draw_groups, c(data = list(as.name("data")), scales = list(as.name("scales")), coordinates = list(as.name("cs")), .$geom_params))
15: get(x, envir = this, inherits = inh)(this, ...)
16: layer$make_grob(df, scales = panel$ranges[[panel_i]], cs = plot$coord)
17: .fun(piece, ...)
18: (function (i) { piece <- pieces[[i]] if (.inform) { res <- try(.fun(piece, ...)) if (inherits(res, "try-error")) { piece <- paste(capture.output(print(piece)), collapse = "\n") stop("with piece ", i, ": \n", piece, call. = FALSE) } } else { res <- .fun(piece, ...) } progress$step() res})(1L)
19: eval(substitute(expr), envir, enclos)
20: evalq((function (i) { piece <- pieces[[i]] if (.inform) { res <- try(.fun(piece, ...)) if (inherits(res, "try-error")) { piece <- paste(capture.output(print(piece)), collapse = "\n") stop("with piece ", i, ": \n", piece, call. = FALSE) } } else { res <- .fun(piece, ...) } progress$step() res})(1L), <environment>)
21: doTryCatch(return(expr), name, parentenv, handler)
22: tryCatchOne(expr, names, parentenv, handlers[[1L]])
23: tryCatchList(expr, classes, parentenv, handlers)
24: tryCatch(evalq((function (i) { piece <- pieces[[i]] if (.inform) { res <- try(.fun(piece, ...)) if (inherits(res, "try-error")) { piece <- paste(capture.output(print(piece)), collapse = "\n") stop("with piece ", i, ": \n", piece, call. = FALSE) } } else { res <- .fun(piece, ...) } progress$step() res})(1L), <environment>), error = .rcpp_error_recorder)
25: withCallingHandlers(tryCatch(evalq((function (i) { piece <- pieces[[i]] if (.inform) { res <- try(.fun(piece, ...)) if (inherits(res, "try-error")) { piece <- paste(capture.output(print(piece)), collapse = "\n") stop("with piece ", i, ": \n", piece, call. = FALSE) } } else { res <- .fun(piece, ...) } progress$step() res})(1L), <environment>), error = .rcpp_error_recorder), warning = .rcpp_warning_recorder)
aborting ...
Segmentation fault (core dumped)
The output of my sessionInfo() command is as follows:
R version 3.1.1 (2014-07-10)
Platform: x86_64-redhat-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] ggplot2_1.0.1
loaded via a namespace (and not attached):
[1] colorspace_1.2-6 digest_0.6.8 grid_3.1.2 gtable_0.1.2 labeling_0.3 MASS_7.3-35
[7] munsell_0.4.2 plyr_1.8.2 proto_0.3-10 Rcpp_0.11.6 reshape2_1.4.1 scales_0.2.4
[13] stringr_0.6.2 tools_3.1.2
Has anyone else run into this? It's been driving me nuts (intermittently!) over the last few days. I've tried this with different versions of R (3.1.1 - 3.2.0), Rcpp (0.11.6, 0.11.5), plyr (1.8.1 - 1.8.2) as well as two different (Red Hat) Linux machines, and can reproduce the error on both.
Any help or suggestions are much appreciated.
EDIT:
I've done clean-installs of R and all packages on the machines I'm working on (based on an older suggestion found in issue #1006 of hadley/ggplot2)
I'm using version tracking and have the identical code base for the Shiny application on an OSX laptop as well as two different RedHat Linux machines.
I have had no problems running on OSX; so I used packrat to replicate the package library to the two RedHat machines and all three are running R-3.1.2, but this does not resolve the problem.
The Shiny application still crashes with the following output in the JS console:
Warning in run(timeoutMs) :
Removed 1 rows containing missing values (geom_path).
Error: Results must be all atomic, or all data frames
Execution halted
Here are results from the Javascript Console:
67: try(handler(binary, message))
68: (function (handle, binary, message) { for (handler in .wsconns[[as.character(handle)]]$.messageCallbacks) { result <- try(handler(binary, message)) if (inherits(result, "try-error")) { .wsconns[[as.character(handle)]]$close() return() } }})("266303168", FALSE, "{\"method\":\"update\",\"data\":{\".clientdata_output_out035b13d236a68453_width\":386,\".clientdata_output_out035b13d236a68453_height\":400,\".clientdata_output_out45a459f761c2bcff_width\":386,\".clientdata_output_out45a459f761c2bcff_height\":400,\".clientdata_output_out0b680f21f3d73958_width\":386,\".clientdata_output_out0b680f21f3d73958_height\":400,\".clientdata_output_out785a5b0a4a8d2872_width\":386,\".clientdata_output_out785a5b0a4a8d2872_height\":400,\".clientdata_output_out4d4c261305d448a2_width\":386,\".clientdata_output_out4d4c261305d448a2_height\":400,\".clientdata_output_outae8b99c9ab2044d8_width\":386,\".clientdata_output_outae8b99c9ab2044d8_height\":400,\".clientdata_output_oute11f4c69b81158cc_width\":386,\".clientdata_output_oute11f4c69b81158cc_height\":400,\".clientdata_output_out1beb34c46b1bdebd_width\":386,\".clientdata_output_out1beb34c46b1bdebd_height\":400,\".clientdata_output_out9b9abdc2e1b58daa_width\":386,\".clientdata_output_out9b9abdc2e1b58daa_height\":400,\".clientdata_output_out035b13d236a68453_hidden\":false,\".clientdata_output_out45a459f761c2bcff_hidden\":false,\".clientdata_output_out0b680f21f3d73958_hidden\":false,\".clientdata_output_out785a5b0a4a8d2872_hidden\":false,\".clientdata_output_out4d4c261305d448a2_hidden\":false,\".clientdata_output_outae8b99c9ab2044d8_hidden\":false,\".clientdata_output_oute11f4c69b81158cc_hidden\":false,\".clientdata_output_out1beb34c46b1bdebd_hidden\":false,\".clientdata_output_out9b9abdc2e1b58daa_hidden\":false}}")
69: eval(substitute(expr), envir, enclos)
70: evalq((function (handle, binary, message) { for (handler in .wsconns[[as.character(handle)]]$.messageCallbacks) { result <- try(handler(binary, message)) if (inherits(result, "try-error")) { .wsconns[[as.character(handle)]]$close() return() } }})("266303168", FALSE, "{\"method\":\"update\",\"data\":{\".clientdata_output_out035b13d236a68453_width\":386,\".clientdata_output_out035b13d236a68453_height\":400,\".clientdata_output_out45a459f761c2bcff_width\":386,\".clientdata_output_out45a459f761c2bcff_height\":400,\".clientdata_output_out0b680f21f3d73958_width\":386,\".clientdata_output_out0b680f21f3d73958_height\":400,\".clientdata_output_out785a5b0a4a8d2872_width\":386,\".clientdata_output_out785a5b0a4a8d2872_height\":400,\".clientdata_output_out4d4c261305d448a2_width\":386,\".clientdata_output_out4d4c261305d448a2_height\":400,\".clientdata_output_outae8b99c9ab2044d8_width\":386,\".clientdata_output_outae8b99c9ab2044d8_height\":400,\".clientdata_output_oute11f4c69b81158cc_width\":386,\".clientdata_output_oute11f4c69b81158cc_height\":400,\".clientdata_output_out1beb34c46b1bdebd_width\":386,\".clientdata_output_out1beb34c46b1bdebd_height\":400,\".clientdata_output_out9b9abdc2e1b58daa_width\":386,\".clientdata_output_out9b9abdc2e1b58daa_height\":400,\".clientdata_output_out035b13d236a68453_hidden\":false,\".clientdata_output_out45a459f761c2bcff_hidden\":false,\".clientdata_output_out0b680f21f3d73958_hidden\":false,\".clientdata_output_out785a5b0a4a8d2872_hidden\":false,\".clientdata_output_out4d4c261305d448a2_hidden\":false,\".clientdata_output_outae8b99c9ab2044d8_hidden\":false,\".clientdata_output_oute11f4c69b81158cc_hidden\":false,\".clientdata_output_out1beb34c46b1bdebd_hidden\":false,\".clientdata_output_out9b9abdc2e1b58daa_hidden\":false}}"), <environment>)
71: doTryCatch(return(expr), name, parentenv, handler)
72: tryCatchOne(expr, names, parentenv, handlers[[1L]])
73: tryCatchList(expr, classes, parentenv, handlers)
74: tryCatch(evalq((function (handle, binary, message) { for (handler in .wsconns[[as.character(handle)]]$.messageCallbacks) { result <- try(handler(binary, message)) if (inherits(result, "try-error")) { .wsconns[[as.character(handle)]]$close() return() } }})("266303168", FALSE, "{\"method\":\"update\",\"data\":{\".clientdata_output_out035b13d236a68453_width\":386,\".clientdata_output_out035b13d236a68453_height\":400,\".clientdata_output_out45a459f761c2bcff_width\":386,\".clientdata_output_out45a459f761c2bcff_height\":400,\".clientdata_output_out0b680f21f3d73958_width\":386,\".clientdata_output_out0b680f21f3d73958_height\":400,\".clientdata_output_out785a5b0a4a8d2872_width\":386,\".clientdata_output_out785a5b0a4a8d2872_height\":400,\".clientdata_output_out4d4c261305d448a2_width\":386,\".clientdata_output_out4d4c261305d448a2_height\":400,\".clientdata_output_outae8b99c9ab2044d8_width\":386,\".clientdata_output_outae8b99c9ab2044d8_height\":400,\".clientdata_output_oute11f4c69b81158cc_width\":386,\".clientdata_output_oute11f4c69b81158cc_height\":400,\".clientdata_output_out1beb34c46b1bdebd_width\":386,\".clientdata_output_out1beb34c46b1bdebd_height\":400,\".clientdata_output_out9b9abdc2e1b58daa_width\":386,\".clientdata_output_out9b9abdc2e1b58daa_height\":400,\".clientdata_output_out035b13d236a68453_hidden\":false,\".clientdata_output_out45a459f761c2bcff_hidden\":false,\".clientdata_output_out0b680f21f3d73958_hidden\":false,\".clientdata_output_out785a5b0a4a8d2872_hidden\":false,\".clientdata_output_out4d4c261305d448a2_hidden\":false,\".clientdata_output_outae8b99c9ab2044d8_hidden\":false,\".clientdata_output_oute11f4c69b81158cc_hidden\":false,\".clientdata_output_out1beb34c46b1bdebd_hidden\":false,\".clientdata_output_out9b9abdc2e1b58daa_hidden\":false}}"), <environment>), error = .rcpp_error_recorder)
75: withCallingHandlers(tryCatch(evalq((function (handle, binary, message) { for (handler in .wsconns[[as.character(handle)]]$.messageCallbacks) { result <- try(handler(binary, message)) if (inherits(result, "try-error")) { .wsconns[[as.character(handle)]]$close() return() } }})("266303168", FALSE, "{\"method\":\"update\",\"data\":{\".clientdata_output_out035b13d236a68453_width\":386,\".clientdata_output_out035b13d236a68453_height\":400,\".clientdata_output_out45a459f761c2bcff_width\":386,\".clientdata_output_out45a459f761c2bcff_height\":400,\".clientdata_output_out0b680f21f3d73958_width\":386,\".clientdata_output_out0b680f21f3d73958_height\":400,\".clientdata_output_out785a5b0a4a8d2872_width\":386,\".clientdata_output_out785a5b0a4a8d2872_height\":400,\".clientdata_output_out4d4c261305d448a2_width\":386,\".clientdata_output_out4d4c261305d448a2_height\":400,\".clientdata_output_outae8b99c9ab2044d8_width\":386,\".clientdata_output_outae8b99c9ab2044d8_height\":400,\".clientdata_output_oute11f4c69b81158cc_width\":386,\".clientdata_output_oute11f4c69b81158cc_height\":400,\".clientdata_output_out1beb34c46b1bdebd_width\":386,\".clientdata_output_out1beb34c46b1bdebd_height\":400,\".clientdata_output_out9b9abdc2e1b58daa_width\":386,\".clientdata_output_out9b9abdc2e1b58daa_height\":400,\".clientdata_output_out035b13d236a68453_hidden\":false,\".clientdata_output_out45a459f761c2bcff_hidden\":false,\".clientdata_output_out0b680f21f3d73958_hidden\":false,\".clientdata_output_out785a5b0a4a8d2872_hidden\":false,\".clientdata_output_out4d4c261305d448a2_hidden\":false,\".clientdata_output_outae8b99c9ab2044d8_hidden\":false,\".clientdata_output_oute11f4c69b81158cc_hidden\":false,\".clientdata_output_out1beb34c46b1bdebd_hidden\":false,\".clientdata_output_out9b9abdc2e1b58daa_hidden\":false}}"), <environment>), error = .rcpp_error_recorder), warning = .rcpp_warning_recorder)
aborting ...
One additional item I've noticed; the shiny app I'm running is like a monte-carlo simulation - and the more runs I pass through, the greater the chance that the application crashes. Fewer runs tend to be successful all the way through; it appears as though it may be related to memory allocation.
EDIT 2:
Reverting to older versions of Rcpp, plyr, and ggplot seems to resolve it on all three machines (OSX & Red Hat); I haven't tested to see if it's possible to upgrade one of the three from this point.
Rcpp 0.11.3
plyr 1.8.1
ggplot2 0.9.3
I can't post the code, because SO won't let me post more than two links, but I've installed using the following for all 3 packages:
install.packages(<<url to source tarball>>,repos = NULL,type="source")`

Answer 1

This error was caused by a bug, which was affecting the plyr package used by ggplot2. Hadley has fixed this in the latest development version by reverting to an older version of loop_apply.
The current fix is to install the development version of plyr:
devtools::install_github("hadley/plyr")
Fixed in commit: https://github.com/hadley/plyr/commit/3256bb524dbe04982ad54300057dd1e1bf852906