Using blas/Lapack in Fortran file for R - r

I have the following problem. I have a file say file.f90 in which I have implemented some Fortran subroutines, say called foo. Then I compile these functions using "R CMD SHLIB file.f90". To use the function foo in a seperate R file I then use dyn.load("foo.dll") and to call it .Fortran("foo", ...).
So far so good. But now I need to use some functions implemented in Lapack.
I have no idea how to do this or where to have a look. I have only tried calling "R CMD SHLIB file.f90 -llapack" but already there I get an error that llapack has not been found. Any hints would be greatly appreciated!!
EDIT:
I have finally found an answer to my question with the help of everyone here and with looking up much on the internet. I have to say the solution is quite easy but as I am quite a noob when it comes to these things it still took some time. So here s my solution for Windows 11 and R studio 4.1
Assume that our R session/project has the path PATH_PROJ, e.g "C:\Users\Myname\Documents\MyProject". Then I created a new folder named "f90files" in which I intended to save all Fortran functions, so PATH_PROJ\f90files.
Next, I needed the path of my R's Lapack PATH_LAPACK, e.g "C:\Program Files\R\R-4.1.2\bin\x64\Rlapack.dll".
In PATH_PROJ\f90files I then implemented the Fortran subroutine as suggested by Jean-Claude Arbaut:
subroutine eigvals(n, a, vre, vim, info)
implicit none
integer :: n, info
integer, parameter :: lwork = 65536
double precision :: a(n, n), vre(n), vim(n)
double precision, save :: work(lwork)
call dgeev("n", "n", n, a, n, vre, vim, 0d0, 1, 0d0, 1, work, lwork, info)
end subroutine
Following this, I started up Windows command prompt and typed
gfortran -shared PATH_LAPACK PATH_PROJ\f90files\eigvals.f90 -o PATH_PROJ\f90files\eigvals.so -o PATH_PROJ\f90files\eigvals.dll
and further
gfortran -shared PATH_LAPACK PATH_PROJ\f90files\eigvals.f90 -o PATH_PROJ\f90files\eigvals.so
(maybe this can be done in one go?)
With this all was nicely compiled. In R I then loaded the function using
dyn.load("PATH_PROJ\f90files\eigvals.dll")
Finally, using the implementation given below, I ran
eigvals <- function(a) {
if (is.matrix(a) && is.double(a) && nrow(a) == ncol(a)) {
n <- nrow(a)
s <- .Fortran("eigvals", n = as.integer(n), a = a, vre = double(n), vim = double(n), info = 0L)
structure(complex(real = s$vre, imaginary = s$vim), info = s$info)
} else stop("Invalid input")
}
eigvals(a)
and voilà we are done! Thanks again to everyone!

The libraries you are looking for are Rblas.dll and Rlapack.dll in the R-4.2.2\bin\x64 directory (replace 4.2.2 with your version).
Here is an example. Let's compute eigenvalues using LAPACK's dgeev.
Fortran file eigvals.f90. Here to simplify lwork is a constant, but in "real" code you would have to do this more carefully.
subroutine eigvals(n, a, vre, vim, info)
implicit none
integer :: n, info
integer, parameter :: lwork = 65536
double precision :: a(n, n), vre(n), vim(n)
double precision, save :: work(lwork)
call dgeev("n", "n", n, a, n, vre, vim, 0d0, 1, 0d0, 1, work, lwork, info)
end subroutine
Compile with either one of the following commands (change the path as necessary). If you are on Windows, do this from the Rtools bash window. On Linux the extension is .so and not .dll.
gfortran -shared -L/c/App/R/R-4.2.2/bin/x64 eigvals.f90 -lRlapack -o eigvals.dll
R CMD SHLIB eigvals.f90 -lRlapack
In R, you can now do, assuming you have setwd() to the directory containing the DLL:
a <- matrix(c(2, 9, 4, 7, 5, 3, 6, 1, 8), 3, 3, byrow = T)
dyn.load("eigvals.dll")
is.loaded("eigvals")
eigvals <- function(a) {
if (is.matrix(a) && is.double(a) && nrow(a) == ncol(a)) {
n <- nrow(a)
s <- .Fortran("eigvals", n = as.integer(n), a = a, vre = double(n), vim = double(n), info = 0L)
structure(complex(real = s$vre, imaginary = s$vim), info = s$info)
} else stop("Invalid input")
}
eigvals(a)
Pay attention to the number and type of subroutine arguments in .Fortran, otherwise you may crash R. Note also that you must call Fortran subroutines, not functions.
I you are on Windows and using R-4.2 with Rtools 4.2, there is an extra trick: the compiler is no longer in the default directories. See this. You have first to do, in the Rtools bash window:
export PATH=/x86_64-w64-mingw32.static.posix/bin:$PATH
If you are using the compiler from the Windows command prompt, you will have to modify the PATH environment variable accordingly.

Related

*** caught segfault *** address 0x2aaeb4b6f440, cause 'memory not mapped' when Fortran is called by R

I have a very specific error so googling wasn't helpful and I'm sorry I don't know how to provide a simple producible example for this issue. The code below runs perfectly on my local machine but on the HPC it is producing this error:
*** caught segfault ***
address 0x2ad718ba0440, cause 'memory not mapped'
Traceback:
1: array(.Fortran("hus_vertical_interpolation", m = as.integer(DIM[1]), n = as.integer(DIM[2]), o = as.integer(DIM[3]), p = as.integer(DIM[4]), req = as.integer(length(req_press_levels)), hus_on_model_level = as.numeric(spec_hum_data[]), pres = as.numeric(req_press_levels), pressure_full_level = as.numeric(pressure[]), hus_on_press_level = as.numeric(output_array[]))$hus_on_press_level, dim = output_DIM)
2: Specific_humidity_afterburner(spec_hum_file = q_nc.files[x], req_press_levels = required_PLev)
Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:
The code is supposed to:
Loop over a vector of NetCDF files and pass the filename spec_hum_file to function Specific_humidity_afterburner.
The function reads the NetCDF file, extract data pass to the first compiled subroutine, do the math and return the values.
Take the result, pass it to another FORTRAN subroutine and return the second result.
Write the second result to a new NetCDF file.
The error occurs in step 3. The R function is:
Specific_humidity_afterburner<-function(spec_hum_file,req_press_levels){
require(ff)
require(ncdf4)
require(stringi)
require(DescTools)
library(stringr)
library(magrittr)
#1============================================================================
#Reading data from netCDF file
#2============================================================================
#Reading other variables
#3============================================================================
# First Fortran subroutine
#4============================================================================
#load vertical interpolate subroutine for specific humidity
dyn.load("spec_hum_afterburner/vintp2p_afterburner_hus.so")
#check
is.loaded("hus_vertical_interpolation")
DIM<-dim(spec_hum_data)
output_DIM<-c(DIM[1],DIM[2],length(req_press_levels),DIM[4])
output_array<-ff(array(0.00,dim =output_DIM),dim =output_DIM)
result<- array(.Fortran("hus_vertical_interpolation",
m=as.integer(DIM[1]),
n=as.integer(DIM[2]),
o=as.integer(DIM[3]),
p=as.integer(DIM[4]),
req = as.integer(length(req_press_levels)),
pres=as.numeric(req_press_levels),
pressure_full_level=as.numeric(pressure[]),
hus_on_model_level=as.numeric(spec_hum_data[]),
hus_on_press_level=as.numeric(output_array[]))$hus_on_press_level,
dim =output_DIM)
DIMNAMES<-dimnames(spec_hum_data)
DIMNAMES[["lev"]]<-req_press_levels
Specific_humidity<- ff(result, dim = output_DIM,
dimnames =DIMNAMES )
rm(result)
#5============================================================================
# Writing NetCDF file of the interpolated values
}
Fortran subroutine:
subroutine hus_vertical_interpolation(m,n,o,p,req,pres, &
pressure_full_level,hus_on_model_level,hus_on_press_level)
implicit none
integer :: m,n,o,p,req
integer :: x,y,s,t,plev
double precision :: pres(req),hus_on_model_level(m,n,o,p)
double precision :: pressure_full_level(m,n,o,p)
double precision :: delta_hus,delta_p,grad_hus_p,diff_p
double precision, intent(out) :: hus_on_press_level(m,n,req,p)
real :: arg = -1.0,NaN
NaN= sqrt(arg)
do plev=1,req
do t=1,p
do x=1,m
do y=1,n
do s=1,o
!above uppest level
if(pres(plev) .LT. pressure_full_level(x,y,1,t)) then
hus_on_press_level(x,y,plev,t) = NaN
end if
! in between levels
if(pres(plev) .GE. pressure_full_level(x,y,s,t) .AND. pres(plev) .LE. &
pressure_full_level(x,y,s+1,t) ) then
delta_hus = hus_on_model_level(x,y,s,t) - hus_on_model_level(x,y,s+1,t)
delta_p = log(pressure_full_level(x,y,s,t))&
- log(pressure_full_level(x,y,s+1,t))
grad_hus_p = delta_hus /delta_p
diff_p = log(pres(plev)) - log(pressure_full_level(x,y,s,t))
hus_on_press_level(x,y,plev,t) = hus_on_model_level(x,y,s,t)&
+ grad_hus_p * diff_p
end if
! extrapolation below the ground
if(pres(plev) .GT. pressure_full_level(x,y,o,t)) then
hus_on_press_level(x,y,plev,t) = hus_on_model_level(x,y,o,t)
end if
end do
end do
end do
end do
end do
end subroutine hus_vertical_interpolation
Fortran subroutine was compiled with:
gfortran -fPIC -shared -ffree-form vintp2p_afterburner_hus.f90 -o vintp2p_afterburner_hus.so
The error behaviour is unpredictable for example, can happen at index 1, 2, 8, .. etc of the loop. We have tried to hand over the big array to the Fortran subroutine as the last variable, it minimized the occurrence of the error.
Also, the NetCDF files have a size of ~2GB. Another point to mention, The modules are built with EasyBuild so conflicts are not probable as HPC support team stated. We have tried many solutions as far as we know and no progress!

How to compute a modular exponentiation in Crystal?

I want to compute 1_299_709 ** 1_300_751 % 104_729 in Crystal.
In Python, the pow function allows to pass the modulo as third argument:
❯ python
>>> pow(1_299_709, 1_300_751, 104_729)
90827
In Ruby, the same:
❯ irb
irb(main):001:0> 1_299_709.pow(1_300_751, 104_729)
=> 90827
But in Crystal, there seems not to be such a functionality, and naturally, using ** operators quickly overflows:
❯ crystal eval "1_299_709 ** 1_300_751 % 104_729"
Unhandled exception: Arithmetic overflow (OverflowError)
from /usr/lib/crystal/int.cr:0:9 in '**'
from /eval:1:1 in '__crystal_main'
from /usr/lib/crystal/crystal/main.cr:97:5 in 'main_user_code'
from /usr/lib/crystal/crystal/main.cr:86:7 in 'main'
from /usr/lib/crystal/crystal/main.cr:106:3 in 'main'
from __libc_start_main
from _start
from ???
How to compute a modular exponentiation in Crystal?
Edit: To clarify, I'm already using BigInt but that doesn't work. I removed BigInt from my minimal working example for simplicity.
The following Python code contains the actual numbers from my program:
>>> pow(53583115773616729421957814870755484980404298242901134400501331255090818409243, 28948022309329048855892746252171976963317496166410141009864396001977208667916, 115792089237316195423570985008687907853269984665640564039457584007908834671663)
75711134420273723792089656449854389054866833762486990555172221523628676983696
It executes easily and returns the correct result. Same for Ruby:
irb(main):001:0> 53583115773616729421957814870755484980404298242901134400501331255090818409243.pow(2894802230932904885589274625217197696331749616641014100986
4396001977208667916, 115792089237316195423570985008687907853269984665640564039457584007908834671663)
=> 75711134420273723792089656449854389054866833762486990555172221523628676983696
However, Crystal:
a = BigInt.new 53583115773616729421957814870755484980404298242901134400501331255090818409243
e = BigInt.new 28948022309329048855892746252171976963317496166410141009864396001977208667916
p = BigInt.new 115792089237316195423570985008687907853269984665640564039457584007908834671663
y = a ** e % p # overflows with and without BigInt
Is resulting in:
gmp: overflow in mpz type
Program received and didn't handle signal IOT (6)
How to compute such a massive modular exponentiation in Crystal?
Edit: Filed an issue to make sure it's not a bug: crystal-lang/crystal#8612
As stated in the Github issue, this can be easily circumvented by binding mpz_powm_sec from gmp:
This is pretty simple:
https://carc.in/#/r/89qh
require "big/big_int"
a = BigInt.new "53583115773616729421957814870755484980404298242901134400501331255090818409243"
e = BigInt.new "28948022309329048855892746252171976963317496166410141009864396001977208667916"
p = BigInt.new "115792089237316195423570985008687907853269984665640564039457584007908834671663"
#[Link("gmp")]
lib LibGMP
fun mpz_powm_sec = __gmpz_powm_sec(rop : MPZ*, base : MPZ*, exp : MPZ*, mod : MPZ*)
end
#y = a ** e % p
y = BigInt.new
LibGMP.mpz_powm_sec(y, a, e, p)
puts y
# > 75711134420273723792089656449854389054866833762486990555172221523628676983696

optimParallel can not find Rcpp function [duplicate]

I've written a function in Rcpp and compiled it with inline. Now, I want to run it in parallel on different cores, but I'm getting a strange error. Here's a minimal example, where the function funCPP1 can be compiled and runs well by itself, but cannot be called by snow's clusterCall function. The function runs well as a single process, but gives the following error when ran in parallel:
Error in checkForRemoteErrors(lapply(cl, recvResult)) :
2 nodes produced errors; first error: NULL value passed as symbol address
And here is some code:
## Load and compile
library(inline)
library(Rcpp)
library(snow)
src1 <- '
Rcpp::NumericMatrix xbem(xbe);
int nrows = xbem.nrow();
Rcpp::NumericVector gv(g);
for (int i = 1; i < nrows; i++) {
xbem(i,_) = xbem(i-1,_) * gv[0] + xbem(i,_);
}
return xbem;
'
funCPP1 <- cxxfunction(signature(xbe = "numeric", g="numeric"),body = src1, plugin="Rcpp")
## Single process
A <- matrix(rnorm(400), 20,20)
funCPP1(A, 0.5)
## Parallel
cl <- makeCluster(2, type = "SOCK")
clusterExport(cl, 'funCPP1')
clusterCall(cl, funCPP1, A, 0.5)
Think it through -- what does inline do? It creates a C/C++ function for you, then compiles and links it into a dynamically-loadable shared library. Where does that one sit? In R's temp directory.
So you tried the right thing by shipping the R frontend calling that shared library to the other process (which has another temp directory !!), but that does not get the dll / so file there.
Hence the advice is to create a local package, install it and have both snow processes load and call it.
(And as always: better quality answers may be had on the rcpp-devel list which is read by more Rcpp constributors than SO is.)
Old question, but I stumbled across it while looking through the top Rcpp tags so maybe this answer will be of use still.
I think Dirk's answer is proper when the code you've written is fully de-bugged and does what you want, but it can be a hassle to write a new package for such as small piece of code like in the example. What you can do instead is export the code block, export a "helper" function that compiles source code and run the helper. That'll make the CXX function available, then use another helper function to call it. For instance:
# Snow must still be installed, but this functionality is now in "parallel" which ships with base r.
library(parallel)
# Keep your source as an object
src1 <- '
Rcpp::NumericMatrix xbem(xbe);
int nrows = xbem.nrow();
Rcpp::NumericVector gv(g);
for (int i = 1; i < nrows; i++) {
xbem(i,_) = xbem(i-1,_) * gv[0] + xbem(i,_);
}
return xbem;
'
# Save the signature
sig <- signature(xbe = "numeric", g="numeric")
# make a function that compiles the source, then assigns the compiled function
# to the global environment
c.inline <- function(name, sig, src){
library(Rcpp)
funCXX <- inline::cxxfunction(sig = sig, body = src, plugin="Rcpp")
assign(name, funCXX, envir=.GlobalEnv)
}
# and the function which retrieves and calls this newly-compiled function
c.namecall <- function(name,...){
funCXX <- get(name)
funCXX(...)
}
# Keep your example matrix
A <- matrix(rnorm(400), 20,20)
# What are we calling the compiled funciton?
fxname <- "TestCXX"
## Parallel
cl <- makeCluster(2, type = "PSOCK")
# Export all the pieces
clusterExport(cl, c("src1","c.inline","A","fxname"))
# Call the compiler function
clusterCall(cl, c.inline, name=fxname, sig=sig, src=src1)
# Notice how the function now named "TestCXX" is available in the environment
# of every node?
clusterCall(cl, ls, envir=.GlobalEnv)
# Call the function through our wrapper
clusterCall(cl, c.namecall, name=fxname, A, 0.5)
# Works with my testing
I've written a package ctools (shameless self-promotion) which wraps up a lot of the functionality that is in the parallel and Rhpc packages for cluster computing, both with PSOCK and MPI. I already have a function called "c.sourceCpp" which calls "Rcpp::sourceCpp" on every node in much the same way as above. I'm going to add in a "c.inlineCpp" which does the above now that I see the usefulness of it.
Edit:
In light of Coatless' comments, the Rcpp::cppFunction() in fact negates the need for the c.inline helper here, though the c.namecall is still needed.
src2 <- '
NumericMatrix TestCpp(NumericMatrix xbe, int g){
NumericMatrix xbem(xbe);
int nrows = xbem.nrow();
NumericVector gv(g);
for (int i = 1; i < nrows; i++) {
xbem(i,_) = xbem(i-1,_) * gv[0] + xbem(i,_);
}
return xbem;
}
'
clusterCall(cl, Rcpp::cppFunction, code=src2, env=.GlobalEnv)
# Call the function through our wrapper
clusterCall(cl, c.namecall, name="TestCpp", A, 0.5)
I resolved it by sourcing on each cluster cluster node an R file with the wanted C inline function:
clusterEvalQ(cl,
{
library(inline)
invisible(source("your_C_func.R"))
})
And your file your_C_func.R should contain the C function definition:
c_func <- cfunction(...)

Compiling plplot with gfortran

Gfortran compilation fails with plplot graphics library.
FYI: Plplot is a graphics library with which one can plot directly from gfortran (among other languages).
I have installed the following packages (on Xubuntu 18.04)
sudo apt install gfortran libplplot15 libplplot-dev libplplotfortran0 plplot-driver-cairo plplot-driver-qt plplot-driver-wxwidgets plplot-driver-xwin plplot-doc
I updated the local database with the following command: sudo updatedb. When I ran the command locate plplot I get the following relevant lines (along with other lines)
/usr/lib/x86_64-linux-gnu/pkgconfig/plplot-fortran.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/plplot.pc
Then I tried to compile the fortran example code given here (relevant part is given below)
program x00f
use plfortrandemolib
integer, parameter :: NSIZE = 101
real(kind=pl_test_flt), dimension(NSIZE) :: x, y
real(kind=pl_test_flt) :: xmin = 0._pl_test_flt, xmax = 1._pl_test_flt, ymin = 0._pl_test_flt, ymax = 100._pl_test_flt
! integer :: i
integer :: plparseopts_rc
! Prepare data to be plotted.
x = arange(NSIZE) / real(NSIZE-1,pl_test_flt)
y = ymax * x**2
! Or alternatively, using a DO-loop
!do i = 1,NSIZE
! x(i) = real( i - 1, pl_test_flt ) / real( NSIZE - 1, pl_test_flt )
! y(i) = ymax * x(i)**2
!enddo
! Parse and process command line arguments
plparseopts_rc = plparseopts( PL_PARSE_FULL )
if(plparseopts_rc .ne. 0) stop "plparseopts error"
! Initialize plplot
call plinit
! Create a labelled box to hold the plot.
call plenv( xmin, xmax, ymin, ymax, 0, 0 )
call pllab( "x", "y=100 x#u2#d", "Simple PLplot demo of a 2D line plot" )
! Plot the data that was prepared above.
call plline( x, y )
! Close PLplot library
call plend
end program x00f
with the following command
gfortran x00f.f90 $(pkg-config --cflags --libs plplot-fortran)
The output of pkg-config --cflags --libs plplot-fortran is
-I/usr/include/plplot -I/usr/lib/x86_64-linux-gnu/fortran/modules/plplot -I/usr/include/plplot -lplplotfortran
The error that I get is the following:
/tmp/ccAQ0C7A.o: In function `MAIN__':
x00f.f90:(.text+0x65): undefined reference to `__plfortrandemolib_MOD_arange_1'
collect2: error: ld returned 1 exit status
Do I need to install any other packages or is the compilation command is incomplete? Any help will be appreciated.
Answering my own question for future SO users.
The correct compilation command for the above code is
gfortran x00f.f90 -lplfortrandemolib $(pkg-config --cflags --libs plplot-fortran)
Also check VladimirF's comment on the same.

Pass function arguments into Julia non-interactively

I have a Julia function in a file. Let's say it is the below. Now I want to pass arguments into this function. I tried doing
julia filename.jl randmatstat(5)
but this gives an error that '(' token is unexpected. Not sure what the solution would be. I am also a little torn on if there is a main function / how to write a full solution using Julia. For example what is the starting / entry point of a Julia Program?
function randmatstat(t)
n = 5
v = zeros(t)
w = zeros(t)
for i = 1:t
a = randn(n,n)
b = randn(n,n)
c = randn(n,n)
d = randn(n,n)
P = [a b c d]
Q = [a b; c d]
v[i] = trace((P.'*P)^4)
w[i] = trace((Q.'*Q)^4)
end
std(v)/mean(v), std(w)/mean(w)
end
Julia doesn't have an "entry point" as such.
When you call julia myscript.jl from the terminal, you're essentially asking julia to execute the script and exit. As such, it needs to be a script. If all you have in your script is a function definition, then it won't do much unless you later call that function from your script.
As for arguments, if you call julia myscript.jl 1 2 3 4, all the remaining arguments (i.e. in this case, 1, 2, 3 and 4) become an array of strings with the special name ARGS. You can use this special variable to access the input arguments.
e.g. if you have a julia script which simply says:
# in julia mytest.jl
show(ARGS)
Then calling this from the linux terminal will give this result:
<bashprompt> $ julia mytest.jl 1 two "three and four"
UTF8String["1","two","three and four"]
EDIT: So, from what I understand from your program, you probably want to do something like this (note: in julia, the function needs to be defined before it's called).
# in file myscript.jl
function randmatstat(t)
n = 5
v = zeros(t)
w = zeros(t)
for i = 1:t
a = randn(n,n)
b = randn(n,n)
c = randn(n,n)
d = randn(n,n)
P = [a b c d]
Q = [a b; c d]
v[i] = trace((P.'*P)^4)
w[i] = trace((Q.'*Q)^4)
end
std(v)/mean(v), std(w)/mean(w)
end
t = parse(Int64, ARGS[1])
(a,b) = randmatstat(t)
print("a is $a, and b is $b\n")
And then call this from your linux terminal like so:
julia myscript.jl 5
You can try running like so:
julia -L filename.jl -E 'randmatstat(5)'
Add the following to your Julia file:
### original file
function randmatstat...
...
end
### new stuff
if length(ARGS)>0
ret = eval(parse(join(ARGS," ")))
end
println(ret)
Now, you can run:
julia filename.jl "randmatstat(5)"
As attempted originally. Note the additional quotes added to make sure the parenthesis don't mess up the command.
Explanation: The ARGS variable is defined by Julia to hold the parameters to the command running the file. Since Julia is an interpreter, we can join these parameters to a string, parse it as Julia code, run it and print the result (the code corresponds to this description).

Resources