Can I call an R function while using open FSharp.Collections.ParallelSeq? - r

Question

I have some code that runs with no problems without parallelization. However, the same code generates exceptions if I try to run it using PSeq instead of Seq. The messages I get look a bit random, they are hard to replicate exactly.
Here is the code. When the exception happens the three lines starting with let tmp2 are highlighted.
let frameToRMatrix (df: Frame<'R,string>) =
let foo k df : float list =
df
|> Frame.getCol k
|> Series.values
|> List.ofSeq
let folder acc k = (k, foo k df |> box) :: acc
let tmp =
List.fold folder [] (df.ColumnKeys |> List.ofSeq)
|> namedParams
let sd = df |> Frame.getCol "Vol0" |> Series.lastValue
let sd = sd * 1000.0 |> int
printfn "%s" "I was here"
let rand = System.Random(sd)
let rms = rand.Next(500)
System.Threading.Thread.Sleep rms
let tmp2 =
tmp
|> R.cbind // This line prints something on the console the first time it is executed
printfn "%s" "And here too"
tmp2
The code above includes random number generation and calls to System.Threading.Thread.Sleep. If I do not include this code, which is not needed under sequential execution, I get a message:
System.ArgumentException: 'An item with the same key has already been added.'
and the following on the console:
I was here
I was here
[1] 4095
So execution never gets to the And here too lines.
When I include the random number generator and the call to sleep I get different results, which seem to depend on the build options.
Below are four examples, with the build options, error message and what I see on the console. Notice that in all examples there are four instances of I was here but only three instances of And here too.
---------------------------------------
Any CPU with Prefer 32-bit checked
System.Runtime.InteropServices.SEHException: 'External component has thrown an exception.'
I was here
I was here
[1] 4095
And here too
I was here
And here too
I was here
And here too
Warning: stack imbalance in 'lazyLoadDBfetch', 66 then 65
Error in value[[3L]](cond) : unprotect_ptr: pointer not found
---------------------------------------
Any CPU with Prefer 32-bit unchecked
System.Runtime.InteropServices.SEHException: 'External component has thrown an exception.'
I was here
I was here
[1] 1.759219e+13
And here too
I was here
And here too
I was here
And here too
Error: cons memory exhausted (limit reached?)
Error: cons memory exhausted (limit reached?)
---------------------------------------
x86
System.AccessViolationException" 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'
I was here
I was here
[1] 4095
And here too
I was here
And here too
I was here
And here too
---------------------------------------
x64
Exception thrown: 'System.AccessViolationException' in Unknown Module. Attermpted to read or write protected memory.
$$$ - MachineLearning.signal: Calculating signal for ticker AAPL
$$$ - MachineLearning.signal: Calculating signal for ticker AAPL
I was here
I was here
[1] 1.759219e+13
And here too
I was here
And here too
I was here
And here too
Error in loadNamespace(name) :
no function to return from, jumping to top level

Answer 1

Based on my experience with debugging subtle issues with threading in the R type provider, I think the answer is no - sadly, the R native interop layer is not thread-safe and so you cannot call it from multiple threads in your F# application.
I think that the standard way of running R in parallel is to spawn multiple R.exe processes doing the work. I don't think you can easily initialise multiple independent R processes from F#, so your best bet is probably to create multiple .NET processes that each controls one R engine.