Cannot access parameters in C++ code in parallel code called from Snow - r

Question

I am developing a parallel R code using the Snow package, but when calling C++ code using the Rcpp package the program just hangs and is unresponsive.
as an example...
I have the following code in R that is using snow to split into certain number of processes
MyRFunction<-function(i) {
n=i
.Call("CppFunction",n,PACKAGE="MyPackage")
}
if (mpi) {
cl<-getMPIcluster()
clusterExport(cl, list("set.user.Random.seed"))
clusterEvalQ(cl, {library(Rcpp); NULL})
out<-clusterApply(cl,1:mc.cores,MyRFunction)
stopCluster(cl)
}
else
out <- parallel::mclapply(1:mc.cores,MyRFunction)
Whereas my C++ function looks like...
RcppExport SEXP CppFunction(SEXP n) {
int n=as<int>(n);
}
If I run it with mpi=false and mc.cores=[some number of threads] the program runs beautifully BUT
if i run it with mpi=true, therefore using snow, the program just hangs at int=as<int>(n) ?????
On the other hand if I define the C++ function as...
RcppExport SEXP CppFunction(SEXP n) {
CharacterVector nn(n);
int n=boost::lexical_cast<int>(nn[0]);
}
The program runs perfectly on each mpi thread?? The problem is that it works for integers doubles etc, but not matrices
Also, I must use lexical_cast from the boost package to make it works since as<> does not.
Does anybody know why this is, and what I am missing here, so I can load my matrices as well?

Answer 1

It is not entirely clear from your question what you are doing but I'd recommend to
simplify: snow certainly works, and works with Rcpp as it does with other packages
trust packages: I found parallel computing setups easier when all nodes are identical local packages sets
be careful with threading: if you have trouble with explicit threading in the snow context, try it first without it and the add it once the basic mechanics work

Answer 2

Finally the issue was resolved, and the problem seems to lie with getMPICluster() which works perfectly fine for pure R code, but not as well with Rcpp, as explained above.
Instead using makeMPICluster command
mc.cores <- max(1, NumberOfNodes*CoresPerNode-1) # minus one for master
cl <- makeMPIcluster(mc.cores)
cat(sprintf("Running with %d workers\n", length(cl)))
clusterCall(cl, function() { library(MyPackage); NULL })
out<-clusterApply(cl,1:mc.cores,MyRFunction)
stopCluster(cl)
Works great! The problem is that you have to manually define the number of nodes and cores per node within the R code, instead of defining it using the mpirun command.