Send messages from within parallel function (parallel or future framework) - r

Question

I would like to send messages from within a function to the R console during a parallel process using the parallel package or the future framework and pblapply::pblapply().
Here is a reprex that does not send any messages to the R console:
# library
library(pbapply)
library(stringi)
library(parallel)
# make fun
fun_func <- function(x){
cat(paste0("hello world ",x))
return(paste0("hello world ",x))}
# get data
set.seed(23)
d <- stri_rand_strings(100, 2, '[a-z]')
names(d) <- d
# make cluster
cl <- parallel::makeCluster(3)
# load func
clusterExport(cl, c("fun_func"))
# run function
pblapply(cl=cl,X=d,FUN=fun_func) -> res
# stop cluster
parallel::stopCluster(cl)
# show res
head(res)
#> $of
#> [1] "hello world of"
#>
#> $is
#> [1] "hello world is"
#>
#> $vl
#> [1] "hello world vl"
#>
#> $zz
#> [1] "hello world zz"
#>
#> $vz
#> [1] "hello world vz"
#>
#> $ws
#> [1] "hello world ws"
Created on 2022-12-07 with reprex v2.0.2
Update:
I just learned that it mitght be hard to get console message in Windows/RStudio. However, logging the messages with ParallelLogger might be an option.
Unfortunaltely I could not implement it.
So I would be happy to have a solution either for sending the messages to the console or to a file.

Answer 1

If you use a parallelization method on top of the future framework, e.g. future_lapply, furrr, foreach with doFuture, and soon also pbapply (https://github.com/psolymos/pbapply/issues/54), output from cat(), print(), message(), warning(), and so on in parallel workers are captured and truly re-outputted ("relayed") in your main R session as the parallel tasks completed. You can read about this in https://future.futureverse.org/articles/future-2-output.html.
If you want to see near-live output, that is, output that is produced while the parallel tasks are still running, then you use the progressr package. It is designed to send near-live progress updates when using Futureverse. Those updates can also include custom messages. For examples, see https://progressr.futureverse.org/#parallel-processing-and-progress-updates.