How to run an lapply in parallel in R? - r

Question

So I have tried a few different ways of doing this but each returns an a different error which is making me question if i'm even doing it correctly.
So without any parallel components, we have the following:
all_necks <- lapply(b_list, b_fun)
This works perfectly; b_list is a dataframe and b_fun is a ton of joins and functions which are to be done on the list.
Because each run takes about 5 minutes and there are 550 elements in b_list, I need this to be faster to be practical.
I try future.lapply but get the following error:
library(future.apply)
options(future.globals.maxSize= 178258920000)
plan(multiprocess, workers = 5) ## Parallelize using five cores
all_necks <- future_lapply(b_list, b_fun)
ERROR:
Error in serialize(data, node$con) : error writing to connection
Then I tried foreach and got the following:
library(doParallel)
cl <- makeCluster(detectCores())
registerDoParallel(cl)
all_necks <- foreach(i = 1:b_list %dopar% {b_fun})
ERROR:
There were 16 warnings (use warnings() to see them)
1: In environment() : closing unused connection 19 (<-DESKTOP-XXX)
2: In environment() : closing unused connection 18 (<-DESKTOP-XXX)
...
I must be doing this incorrectly but I really just want this long lapply to run faster via parallel processing.
I would prefer to do this on 5 cores.
EDIT: Session Info Added
R version 4.0.1 (2020-06-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] future.apply_1.5.0 future_1.17.0 formattable_0.2.0.1 lubridate_1.7.9 data.table_1.12.8 chron_2.3-55
[7] Nmisc_0.3.5 anytime_0.3.7 forcats_0.5.0 stringr_1.4.0 dplyr_1.0.0 purrr_0.3.4
[13] readr_1.3.1 tidyr_1.1.0 tibble_3.0.1 ggplot2_3.3.2 tidyverse_1.3.0 jsonlite_1.6.1
loaded via a namespace (and not attached):
[1] Rcpp_1.0.4.6 lattice_0.20-41 listenv_0.8.0 assertthat_0.2.1 digest_0.6.25 R6_2.4.1
[7] cellranger_1.1.0 backports_1.1.7 reprex_0.3.0 evaluate_0.14 httr_1.4.1 pillar_1.4.4
[13] rlang_0.4.6 readxl_1.3.1 rstudioapi_0.11 furrr_0.1.0 blob_1.2.1 rmarkdown_2.3
[19] htmlwidgets_1.5.1 munsell_0.5.0 tinytex_0.24 broom_0.5.6 compiler_4.0.1 modelr_0.1.8
[25] xfun_0.14 pkgconfig_2.0.3 globals_0.12.5 htmltools_0.5.0 tidyselect_1.1.0 codetools_0.2-16
[31] fansi_0.4.1 crayon_1.3.4 dbplyr_1.4.4 withr_2.2.0 rappdirs_0.3.1 grid_4.0.1
[37] nlme_3.1-148 gtable_0.3.0 lifecycle_0.2.0 DBI_1.1.0 magrittr_1.5 scales_1.1.1
[43] cli_2.0.2 stringi_1.4.6 fs_1.4.1 xml2_1.3.2 ellipsis_0.3.1 generics_0.0.2
[49] vctrs_0.3.1 tools_4.0.1 glue_1.4.1 hms_0.5.3 parallel_4.0.1 colorspace_1.4-1
[55] rvest_0.3.5 knitr_1.28 haven_2.3.1