Issue with doparallel and foreach on centos cluster - r

Question

I am trying to use foreach and doParallel to optimize the computation of some image processing. I have ParamIter which is the parameter to iterate over. ImageProcessing is my custom function to process the images and save the data that has been processed.
library(methods)
library(fftwtools)
library(EBImage)
library(tidyverse)
library(foreach)
library(doParallel)
registerDoParallel(20)
foreach(ParamIter = unique(AllImages$ParamIter)) %dopar% {
AllImagesTemp = AllImages[AllImages$ParamIter == ParamIter,]
ImageProcessing(Image = AllImagesTemp,
Plate = unique(AllImagesTemp$Plate),
TimePoint = unique(AllImagesTemp$TimePoint),
Marker = unique(AllImagesTemp$Marker),
Replicate = unique(AllImagesTemp$Replicate),
Well = unique(AllImagesTemp$ID),
Position = unique(AllImagesTemp$Position),
SaveDir = WellDir,
SaveParam = ParamIter,
ThrEmptyImage = .04)
}
Everything works perfectly locally on a machine with Ubuntu 18.04 and a session as per below:
> sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=de_DE.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=de_DE.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=de_DE.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] bindrcpp_0.2.2 fftwtools_0.9-8 doParallel_1.0.14 iterators_1.0.10 foreach_1.4.4 forcats_0.3.0 stringr_1.4.0 dplyr_0.7.7
[9] purrr_0.3.1 readr_1.1.1 tidyr_0.8.2 tibble_2.0.1 ggplot2_3.1.0 tidyverse_1.2.1 EBImage_4.24.0
loaded via a namespace (and not attached):
[1] tidyselect_0.2.5 locfit_1.5-9.1 haven_1.1.2 lattice_0.20-38 colorspace_1.4-0 htmltools_0.3.6 yaml_2.2.0
[8] rlang_0.3.1 pillar_1.3.1 withr_2.1.2 glue_1.3.0 BiocGenerics_0.28.0 modelr_0.1.2 readxl_1.1.0
[15] jpeg_0.1-8 bindr_0.1.1 plyr_1.8.4 munsell_0.5.0 gtable_0.2.0 cellranger_1.1.0 rvest_0.3.2
[22] htmlwidgets_1.3 codetools_0.2-16 broom_0.5.0 Rcpp_1.0.0 backports_1.1.3 scales_1.0.0 jsonlite_1.6
[29] abind_1.4-5 hms_0.4.2 png_0.1-7 digest_0.6.18 stringi_1.3.1 tiff_0.1-5 grid_3.5.3
[36] cli_1.0.1 tools_3.5.3 bitops_1.0-6 magrittr_1.5 RCurl_1.95-4.11 lazyeval_0.2.1 crayon_1.3.4
[43] pkgconfig_2.0.2 xml2_1.2.0 lubridate_1.7.4 assertthat_0.2.0 httr_1.4.0 rstudioapi_0.9.0 R6_2.3.0
[50] nlme_3.1-137 compiler_3.5.3
However if I am trying to do this on my local cluster it doesn't work and provides me with this error:
Error in { :
task 1 failed - "a call to callNextMethod() appears in a call to ‘.Method’, but the call does not seem to come from either a generic function or another 'callNextMethod'"
Calls: %dopar% -> <Anonymous>
Execution halted
My session on the centos cluster looks like:
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)
Matrix products: default
BLAS: /cm/shared/apps/R/64/3.4.1/lib64/R/lib/libRblas.so
LAPACK: /cm/shared/apps/R/64/3.4.1/lib64/R/lib/libRlapack.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] doParallel_1.0.14 iterators_1.0.10 foreach_1.4.4 forcats_0.3.0
[5] stringr_1.3.1 dplyr_0.7.8 purrr_0.3.0 readr_1.3.1
[9] tidyr_0.8.2 tibble_2.0.1 ggplot2_3.1.0 tidyverse_1.2.1
[13] EBImage_4.20.1 fftwtools_0.9-8
loaded via a namespace (and not attached):
[1] tidyselect_0.2.5 locfit_1.5-9.1 haven_2.0.0
[4] lattice_0.20-35 colorspace_1.4-0 generics_0.0.2
[7] htmltools_0.3.6 rlang_0.3.1 pillar_1.3.1
[10] withr_2.1.2 glue_1.3.0 BiocGenerics_0.24.0
[13] modelr_0.1.2 readxl_1.2.0 bindrcpp_0.2.2
[16] jpeg_0.1-8 bindr_0.1.1 plyr_1.8.4
[19] munsell_0.5.0 gtable_0.2.0 cellranger_1.1.0
[22] rvest_0.3.2 htmlwidgets_1.3 codetools_0.2-15
[25] broom_0.5.1 Rcpp_1.0.0 scales_1.0.0
[28] backports_1.1.3 jsonlite_1.6 abind_1.4-5
[31] hms_0.4.2 png_0.1-7 digest_0.6.18
[34] stringi_1.2.4 tiff_0.1-5 grid_3.4.1
[37] cli_1.0.1 bitops_1.0-6 tools_3.4.1
[40] magrittr_1.5 RCurl_1.95-4.10 lazyeval_0.2.1
[43] crayon_1.3.4 pkgconfig_2.0.2 xml2_1.2.0
[46] lubridate_1.7.4 assertthat_0.2.0 httr_1.4.0
[49] rstudioapi_0.9.0 R6_2.3.0 nlme_3.1-131
[52] compiler_3.4.1
I do not understand the error and what it is pointing me towards. Does someone have an idea of how I could solve this problem?
Upgrading to a newer version of R is not a possibility on my local cluster.
Cheers,
Mathieu