This is a follow up of my previous question. The difference is that instead of a one dimensional array i want to get a 2 dimensional array.
I have the following Fortran subroutine:
subroutine test(d, i, nMCd, DF, X)
integer, intent(in) :: d, i, nMCd
double precision, intent(in), dimension(i,nMCd) :: DF
double precision, intent(out), dimension(i,nMCd) :: X
X = DF + DF
end subroutine test
In R the code is simple:
input <- data.frame(A=c(11,12), B=c(21, 22))
input + input
and I get a 2 by 2 data frame
I am able to compile it for R load it and run it.
system("R CMD SHLIB ./Fortran/mytest.f90")
dyn.load("./Fortran/mytest.so")
X <- .Fortran("test", d = as.integer(1), i = nrow(input), nMCd = ncol(input), DF = unlist(input), X = numeric(nrow(input)*ncol(input)))$X
But I get a vector of length 4 instead of a 2x2 matrix or data frame. I tried X = numeric(nrow(input), ncol(input)) but it does not work
The only solution I can think is running after I run the fortran function
matrix(X,nrow = nrow(input),ncol = ncol(input))
Thanks!
I've reviewed the documentation for .Fortran, .Call and "Writing R extensions" and have not found any instances where a Fortran subroutine returns a matrix. Furthermore, there is no matrix type in the ?.Fortran help page, so I think my comment(s) might be the best solution:
in R you can also coerce a vector to matrix by adding a dimension: dim(X) <- c(2,2).
Or even with more generality: dim(X) <- dim(input)
Obviously I cannot claim this to be from high authority, since I've not done any Fortran programming before. If you were interested in writing code that interacts with R objects in place, you might want to consider studying the data.table package's code. Most such efforts use C or C++. You might also consider the Rcpp or inline package interfaces. The Fortran interface is mostly used to take advantage of the many numerical functions that have been bullet proofed.
Related
I am having trouble handling reals in Fortran, which I use together with R. The following code is written in Fortran:
Subroutine realtest(lol)
implicit none
Real lol
lol = 10.0
End
I compile with the command R CMD SHLIB realtest.f. If I run the shared object in R as:
dyn.load("realtest.so")
res <- .Fortran("realtest",lol= as.numeric(1.2))
The resulting value of lol is 1.2, but it should have been 10. If I do the whole thing with Integers instead, I get the correct value 10.
Try using double precision instead of real; the following works for me:
! realtest.f90
!
subroutine realtest(x)
implicit none
double precision, intent(inout) :: x
x = 10.0
end subroutine realtest
From R,
dyn.load("realtest.so")
res <- .Fortran("realtest", x = as.double(1.2))
res
# $x
# [1] 10
You should declare lol as real*8 since R uses double-precision floating-point number.
I think Julia handles matrices with complex elements correctly.
My task is to modify the spectrum of a Hermitian matrix H and return just the matrix with modified spectrum. i.e. I have a function f(real_vec)->real_vec that modifies the spectrum s(H) of a hermitian matrix H=U[s(H)]U'. I need the result f(H) = U[f(s(H))]U'. I think it is possible to optimize by not computing explicitly the eigfact(H).
Therefore I tried to write my own eigmodif based on the Julia realization of eigfact. That was difficult because I was lost on the line 4816 in lapack.jl, where syevr() is wrapped up.
I need to understand where and how, Julia has converted a COMPLEX HERMITIAN matrix to a REAL-SYMMETRIC one. Theoretically it is possible, since we have a 2n by 2n matrix J that squares to minus identity; for any n by n COMPLEX HERMITIAN matrix H we then turn it into real(H).I + imag(H).J, or in a block form
[ real(H) -imag(H) ]
[ imag(H) real(H) ]
But how does Julia do this?
Not an expert on LAPACK, but perhaps the use of macros in the definition of the eigensolvers is unclear. From linalg/lapack.jl (around line 4900):
# Hermitian eigensolvers
for (syev, syevr, sygvd, elty, relty) in
((:zheev_,:zheevr_,:zhegvd_,:Complex128,:Float64),
(:cheev_,:cheevr_,:chegvd_,:Complex64,:Float32))
#eval begin
# SUBROUTINE ZHEEV( JOBZ, UPLO, N, A, LDA, W, WORK, LWORK, RWORK, INFO )
# * .. Scalar Arguments ..
# CHARACTER JOBZ, UPLO
⋮
⋮
So the macro code uses $syevr as a placeholder to refer to :zheevr_ and :cheevr_ in the two passes through the loop, defining the same syevr! for different type signatures. These are LAPACK functions dedicated to Hermitian matrices and accept complex inputs. So the meat of the calculation and complex number handling goes on inside LAPACK.
I am having trouble handling reals in Fortran, which I use together with R. The following code is written in Fortran:
Subroutine realtest(lol)
implicit none
Real lol
lol = 10.0
End
I compile with the command R CMD SHLIB realtest.f. If I run the shared object in R as:
dyn.load("realtest.so")
res <- .Fortran("realtest",lol= as.numeric(1.2))
The resulting value of lol is 1.2, but it should have been 10. If I do the whole thing with Integers instead, I get the correct value 10.
Try using double precision instead of real; the following works for me:
! realtest.f90
!
subroutine realtest(x)
implicit none
double precision, intent(inout) :: x
x = 10.0
end subroutine realtest
From R,
dyn.load("realtest.so")
res <- .Fortran("realtest", x = as.double(1.2))
res
# $x
# [1] 10
You should declare lol as real*8 since R uses double-precision floating-point number.
New to Julia. Following this blog to do Neural Network:
http://blog.yhathq.com/posts/julia-neural-networks.html
I am confused about data types and error messages in Julia. This is my code (again, following the blog post on Neural Network):
# read in df to train
train_df = readtable("data/winequality-red.csv", separator=';')
# create train and test data splits
y = train_df[:quality]
x = train_df[:, 1:11] # matrix of all except quality
# vector() and matrix() from blog post
n = length(y)
is_train = shuffle([1:n] .> floor(n * .25))
x_train,x_test = x[is_train,:],x[!is_train,:]
y_train,y_test = y[is_train],y[!is_train]
type StandardScalar
mean::Vector{Float64}
std::Vector{Float64}
end
# initialize empty scalar
function StandardScalar()
StandardScalar(Array(Float64, 0), Array(Float64, 0))
end
# compute mean and std of each col
function fit_std_scalar!(std_scalar::StandardScalar, x::Matrix{Float64})
n_rows, n_cols = size(x_test)
std_scalar.std = zeros(n_cols)
std_scalar.mean = zeros(n_cols)
for i = 1:n_cols
std_scalar.mean[i] = mean(x[:,i])
std_scalar.std[i] = std(x[:,i])
end
end
# further vectorize the transformation
function transform(std_scalar::StandardScalar, x::Matrix{Float64})
# element wise subtraction of mean and division of std
(x .- std_scalar.mean') ./ std_scalar.std'
end
# fit and transform
function fit_transform!(std_scalar::StandardScalar, x::Matrix{Float64})
fit_std_scalar!(std_scalar, x)
transform(std_scalar, x)
end
# fit scalar on training data and then transform the test
std_scalar = StandardScalar()
n_rows, n_cols = size(x_test)
# cols before scaling
println("Col means before scaling: ")
for i = 1:n_cols
# C printf function
#printf("%0.3f ", (mean(x_test[:, i])))
end
I am getting the error:
'.-' has no method matching .-(::DataFrame, ::Array{Float64,2}) in fit_transform! ...
For this code:
x_train = fit_transform!(std_scalar, x_train)
x_test = transform(std_scalar, x_test)
# after transforming
println("\n Col means after scaling:")
for i = 1:n_cols
#printf("%0.3f ", (mean(x_test[:,i])))
end
I am new to Julia and am just not understanding what the issue is. Vector() and Matrix() do not work from the blog post. I assume that was from an older version of DataFrame.
What I think my issue is: these functions are taking in ::Matrix{Float64} and I am passing in the DataFrame. I assume that deprecated (?) Matrix() would have fixed this? Not sure. How do I analyze this error and pass these functions the correct types (if that is the problem here)?
Thank you!
The error message says that you're attempting an element-wise subtraction, .-, between a DataFrame and an Array but that operation has no definition for those types. A silly example of this sort of situation:
julia> "a" .- [1, 2, 3]
ERROR: `.-` has no method matching .-(::ASCIIString, ::Array{Int64,1})
My guess is that if you add
println(typeof(x_train))
in front of
x_train = fit_transform!(std_scalar, x_train)
that you'll be told that it's a DataFrame rather than an array that you're trying to work with. I'm not experienced with the DataFrame library but may be able to dig up the conversion tomorrow sometime. This is all I have time for just now.
Added comments after obtaining data file
I retrieved winequality-red.csv and worked with its DataFrame
julia> VERSION
v"0.3.5"
julia> using DataFrames
julia> train_df = readtable("data/winequality-red.csv", separator=';')
julia> y = train_df[:quality]
1599-element DataArray{Int64,1}:
julia> x = train_df[:, 1:11]
1599x11 DataFrame
julia> typeof(x)
DataFrame (constructor with 22 methods)
x and y are at this point array-like objects. The blog post apparently uses vector and matrix to convert these to true arrays, but these functions are unfamiliar to me. As IainDunning points out in his answer (I'd like to cite this properly but haven't puzzled that out yet), this conversion is now done via array. Perhaps this is what you need to do:
julia> y = array(train_df[:quality])
1599-element Array{Int64,1}:
julia> x = array(train_df[:, 1:11])
1599x11 Array{Float64,2}:
I've not followed through with an analysis of all of the other code, so this is a hint at the answer rather than a fully fleshed out and tested solution to your problem. Please let me know how this it works out if you give it a try.
I'm accustomed to seeing and using Array{Float64,1} and Array{Float64,2} rather than Vector{Float64} and Matrix{Float64}. Possibly the vector and matrix synonyms for specific types of arrays is deprecated.
I believe vector(...) and matrix(...) were both replaced with just array(...), but I can't find an issue number to correspond with that change.
I use the bdiag function in the Matrix package in R to generate diagonal matrix, and then I pass the resultant matrix (called mat) into a self-written function but R fails to execute due to the following error:
Error: invalid mode (S4) to pass to Fortran (arg 1)
I checked isS4(mat) and it's TRUE. Thus, I guess there is a way to convert the S4 object somehow in order to be passed to the function. Any advice will be greatly appreciated!
UPDATE: I use the following codes for constructing the block diagonal matrix:
grp.ids <- as.factor(c(rep(1,8), rep(2,4), rep(3,2)))
x <- model.matrix(~grp.ids)
X <- do.call(bdiag, replicate(238, x, simplify=FALSE))
Is there any other way to get a S3 matrix without using the bdiag function? Thanks!
Only the .Call() interface can pass full R objects down to C or C++ code, see Section 5 of the Writing R Extensions manual. With .Fortran() and .C() you are limited to basic vectors of int, double, ... and their corresponding Fortran types.