I am trying to evaluate strings within a for loop within an R script using JuliaCall::julia_eval. While I was able to accomplish this in R using the deSolve package, I am running into issues when converting the code to one that is compatible with Julia. The base code for the correctly functioning R deSolve code is shown below.
library(deSolve)
library(dplyr)
Combine <- c(" - 1*0.4545*(H2O2^1) - 1*27000000*(`$OH`^1)*(H2O2^1)", " - 1*3100000000*(`1,4-dioxane`^1)*(`$OH`^1)",
" - 1*33000*(TOC^1)*(`$OH`^1)", "2*0.4545*(H2O2^1) - 1*3100000000*(`1,4-dioxane`^1)*(`$OH`^1) - 1*33000*(TOC^1)*(`$OH`^1) - 1*27000000*(`$OH`^1)*(H2O2^1) - 1*8500000*(`$OH`^1)*(`HCO3-`^1) - 1*390000000*(`$OH`^1)*(`CO3 2-`^1)",
" - 1*8500000*(`$OH`^1)*(`HCO3-`^1)", " - 1*390000000*(`$OH`^1)*(`CO3 2-`^1)"
)
time <- seq(from=0, to=0.01, by = 1E-4)
State <- c(H2O2 = 0.000294117647058824, `1,4-dioxane` = 0.00000113494,
TOC = 0, `$OH` = 0, `HCO3-` = 0.003766104, `CO3 2-` = 0.0000167638711956647)
ODEcreater2 <- function(t, state, parameters){
with(as.list(c(state)),{
for (i in 1:6) { #
dY[i] <- eval(parse(text=Combine[i]))}
return(list(dY))
} )}
out1<- ode(y = state, times = time, func = ODEcreater2, parms = NULL)
I am trying to use replicate the code and run it in Julia to improve the speed of the ODE solver by using diffeqr vs. deSolve. Unfortunately, I am running into evaluating the string/expression within a for loop in julia_call.
library(diffeqr)
diffeqr::diffeq_setup()
library(JuliaCall)
julia <- julia_setup()
ODEcreater <- JuliaCall::julia_eval("
function (dY,t,state)
for i in 1:6
dY[i] = eval(Meta.parse(:Combine[i]))
end
end")
tspan <- list(1E-6, 1E-3)
sol = diffeqr::ode.solve(ODEcreater,state,tspan, abstol=1e-8, reltol=1e-8)
Does anyone have any insight into the best way to evaluate the strings within the for loop? I have been investigating metaexpressions on the JuliaLang website but am still lost.
As mentioned in the duplicate question https://stackoverflow.com/a/58766919/1544203 , building the string and then doing
sprintf("function f(du,u,p,t)\n%s\nend", paste(Combine, collapse="\n"))
from the R side builds a single string which matches the format that works. This is also optimal since it excludes any extra function calls from the generated function.
from julia docs:
parse(str; raise=true, depwarn=true)
Parse the expression string greedily, returning a single expression. An error is thrown if there are additional
characters after the first expression. If raise is true (default), syntax errors will raise an error; otherwise, parse
will return an expression that will raise an error upon evaluation. If depwarn is false, deprecation warnings will be
suppressed.
julia> Meta.parse("x = 3")
:(x = 3)
so, Meta.parse accepts a string and returns an expression. this should evaluate correctly:
eval(Meta.parse(Combine[i]))
one problem i see is the use of non-valid julia variable names, like $OH
Related
I am coding in R-studio and have a function called saveResults(). It takes:
sce - a Single Cell Experiment object.
opt - a list with five things
clusterLabels - simple dataframe with two columns
The important thing is that I receive an error stating:
Error: unexpected symbol in:
"saveResults(sce = sce, opt = opt, clusteInputs()
zhengMix"
which doesn't agree at all with the parameters I pass into the function. You can see this on the last line of the code block below: I pass in proper parameters, but I receive an error that says I have passed in clusteInputs(), and zhengMix instead of clusterLabels. I don't have a function called clusteInputs(), and zhengMix was several lines above.
# Save the clustering data
InstallAndLoadPackagesForSC3Clustering()
opt <- GetOptionInputs()
zhengMix <- FetchzhengMix(opt)
sce <- CreateSingleCellExperiment(zhengMix)
clusterLabels <- getClusterLabels(sce)
opt <- createNewDirectoriesToSaveData(opt)
saveResults <- function(sce, opt, clusterLabels){
print("Beginning process of saving results...")
maxClusters = ncol(clusterLabels)/2+1
for (n in 2:maxClusters){
savePCAasPDF(sce, opt, numOfClusters = n, clusterLabels)
saveClusterLabelsAsRDS(clusterLabels, numOfClusters = n, opt)
}
saveSilhouetteScores(sce, opt)
print("Done.")
}
saveResults(sce = sce, opt = opt, clusterLabels = clusterLabels)
Does anyone have an idea what is going on? I'm pretty stuck on this.
This isn't the best solution, but I fixed my own problem by removing the code out of the function and running it there caused no issues.
I'm attempting to code the method described here to estimate production functions of metal manufacturers. I've done this in Python and Matlab, but am trying to learn Julia.
spain_clean.csv is a dataset of log capital (lnk), log labor (lnl), log output (lnva), and log materials (lnm) that I am loading. Lagged variables are denoted with an "l" before them.
Code is at the bottom. I am getting an error:
ERROR: LoadError: MethodError: no method matching parseNLExpr_runtime(::JuMP.Model, ::JuMP.GenericQuadExpr{Float64,JuMP.Variable}, ::Array{ReverseDiffSparse.NodeData,1}, ::Int32, ::Array{Float64,1})
I think it has to do with the use of vector sums and arrays going into the non-linear objective, but I do not understand Julia enough to debug this.
using JuMP # Need to say it whenever we use JuMP
using Clp, Ipopt # Loading the GLPK module for using its solver
using CSV # csv reader
# read data
df = CSV.read("spain_clean.csv")
#MODEL CONSTRUCTION
#--------------------
acf = Model(solver=IpoptSolver())
#variable(acf, -10<= b0 <= 10) #
#variable(acf, -5 <= bk <= 5 ) #
#variable(acf, -5 <= bl <= 5 ) #
#variable(acf, -10<= g1 <= 10) #
const g = sum(df[:phihat]-b0-bk* df[:lnk]-bl* df[:lnl]-g1* (df[:lphihat]-b0-bk* df[:llnk]-bl* df[:llnl]))
const gllnk = sum((df[:phihat]-b0-bk* df[:lnk]-bl* df[:lnl]-g1* (df[:lphihat]-b0-bk* df[:llnk]-bl* df[:llnl])).*df[:llnk])
const gllnl = sum((df[:phihat]-b0-bk* df[:lnk]-bl* df[:lnl]-g1* (df[:lphihat]-b0-bk* df[:llnk]-bl* df[:llnl])).*df[:llnl])
const glphihat = sum((df[:phihat]-b0-bk* df[:lnk]-bl* df[:lnl]-g1* (df[:lphihat]-b0-bk* df[:llnk]-bl* df[:llnl])).*df[:lphihat])
#OBJECTIVE
#NLobjective(acf, Min, g* g + gllnk* gllnk + gllnl* gllnk + glphihat* glphihat)
#SOLVE IT
status = solve(acf) # solves the model
println("Objective value: ", getobjectivevalue(acf)) # getObjectiveValue(model_name) gives the optimum objective value
println("b0 = ", getvalue(b0))
println("bk = ", getvalue(bk))
println("bl = ", getvalue(bl))
println("g1 = ", getvalue(g1))
No an expert in Julia, but I think a couple of things are wrong about your code.
first, constant are not supposed to change during iteration and you are making them functions of control variables. Second, what you want to use there are nonlinear expression instead of constants. so instead of the constants what you want to write is
N = size(df, 1)
#NLexpression(acf, g, sum(df[i, :phihat]-b0-bk* df[i, :lnk]-bl* df[i, :lnl]-g1* (df[i, :lphihat]-b0-bk* df[i, :llnk]-bl* df[i, :llnl]) for i=1:N))
#NLexpression(acf, gllnk, sum((df[i,:phihat]-b0-bk* df[i,:lnk]-bl* df[i,:lnl]-g1* (df[i,:lphihat]-b0-bk* df[i,:llnk]-bl* df[i,:llnl]))*df[i,:llnk] for i=1:N))
#NLexpression(acf,gllnl,sum((df[i,:phihat]-b0-bk* df[i,:lnk]-bl* df[i,:lnl]-g1* (df[i,:lphihat]-b0-bk* df[i,:llnk]-bl* df[i,:llnl]))*df[i,:llnl] for i=1:N))
#NLexpression(acf,glphihat,sum((df[i,:phihat]-b0-bk* df[i,:lnk]-bl* df[i,:lnl]-g1* (df[i,:lphihat]-b0-bk* df[i,:llnk]-bl* df[i,:llnl]))*df[i,:lphihat] for i=1:N))
I tested this and it seems to work.
I'm writing a genetic program in order to test the fitness of randomly generated expressions. Shown here is the function to generate the expression as well a the main function. DIV and GT are defined elsewhere in the code:
function create_single_full_tree(depth, fs, ts)
"""
Creates a single AST with full depth
Inputs
depth Current depth of tree. Initially called from main() with max depth
fs Function Set - Array of allowed functions
ts Terminal Set - Array of allowed terminal values
Output
Full AST of typeof()==Expr
"""
# If we are at the bottom
if depth == 1
# End of tree, return function with two terminal nodes
return Expr(:call, fs[rand(1:length(fs))], ts[rand(1:length(ts))], ts[rand(1:length(ts))])
else
# Not end of expression, recurively go back through and create functions for each new node
return Expr(:call, fs[rand(1:length(fs))], create_single_full_tree(depth-1, fs, ts), create_single_full_tree(depth-1, fs, ts))
end
end
function main()
"""
Main function
"""
# Define functional and terminal sets
fs = [:+, :-, :DIV, :GT]
ts = [:x, :v, -1]
# Create the tree
ast = create_single_full_tree(4, fs, ts)
#println(typeof(ast))
#println(ast)
#println(dump(ast))
x = 1
v = 1
eval(ast) # Error out unless x and v are globals
end
main()
I am generating a random expression based on certain allowed functions and variables. As seen in the code, the expression can only have symbols x and v, as well as the value -1. I will need to test the expression with a variety of x and v values; here I am just using x=1 and v=1 to test the code.
The expression is being returned correctly, however, eval() can only be used with global variables, so it will error out when run unless I declare x and v to be global (ERROR: LoadError: UndefVarError: x not defined). I would like to avoid globals if possible. Is there a better way to generate and evaluate these generated expressions with locally defined variables?
Here is an example for generating an (anonymous) function. The result of eval can be called as a function and your variable can be passed as parameters:
myfun = eval(Expr(:->,:x, Expr(:block, Expr(:call,:*,3,:x) )))
myfun(14)
# returns 42
The dump function is very useful to inspect the expression that the parsers has created. For two input arguments you would use a tuple for example as args[1]:
julia> dump(parse("(x,y) -> 3x + y"))
Expr
head: Symbol ->
args: Array{Any}((2,))
1: Expr
head: Symbol tuple
args: Array{Any}((2,))
1: Symbol x
2: Symbol y
typ: Any
2: Expr
[...]
Does this help?
In the Metaprogramming part of the Julia documentation, there is a sentence under the eval() and effects section which says
Every module has its own eval() function that evaluates expressions in its global scope.
Similarly, the REPL help ?eval will give you, on Julia 0.6.2, the following help:
Evaluate an expression in the given module and return the result. Every Module (except those defined with baremodule) has its own 1-argument definition of eval, which evaluates expressions in that module.
I assume, you are working in the Main module in your example. That's why you need to have the globals defined there. For your problem, you can use macros and interpolate the values of x and y directly inside the macro.
A minimal working example would be:
macro eval_line(a, b, x)
isa(a, Real) || (warn("$a is not a real number."); return :(throw(DomainError())))
isa(b, Real) || (warn("$b is not a real number."); return :(throw(DomainError())))
return :($a * $x + $b) # interpolate the variables
end
Here, #eval_line macro does the following:
Main> #macroexpand #eval_line(5, 6, 2)
:(5 * 2 + 6)
As you can see, the values of macro's arguments are interpolated inside the macro and the expression is given to the user accordingly. When the user does not behave,
Main> #macroexpand #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
:((Main.throw)((Main.DomainError)()))
a user-friendly warning message is provided to the user at parse-time, and a DomainError is thrown at run-time.
Of course, you can do these things within your functions, again by interpolating the variables --- you do not need to use macros. However, what you would like to achieve in the end is to combine eval with the output of a function that returns Expr. This is what the macro functionality is for. Finally, you would simply call your macros with an # sign preceding the macro name:
Main> #eval_line(5, 6, 2)
16
Main> #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
ERROR: DomainError:
Stacktrace:
[1] eval(::Module, ::Any) at ./boot.jl:235
EDIT 1. You can take this one step further, and create functions accordingly:
macro define_lines(linedefs)
for (name, a, b) in eval(linedefs)
ex = quote
function $(Symbol(name))(x) # interpolate name
return $a * x + $b # interpolate a and b here
end
end
eval(ex) # evaluate the function definition expression in the module
end
end
Then, you can call this macro to create different line definitions in the form of functions to be called later on:
#define_lines([
("identity_line", 1, 0);
("null_line", 0, 0);
("unit_shift", 0, 1)
])
identity_line(5) # returns 5
null_line(5) # returns 0
unit_shift(5) # returns 1
EDIT 2. You can, I guess, achieve what you would like to achieve by using a macro similar to that below:
macro random_oper(depth, fs, ts)
operations = eval(fs)
oper = operations[rand(1:length(operations))]
terminals = eval(ts)
ts = terminals[rand(1:length(terminals), 2)]
ex = :($oper($ts...))
for d in 2:depth
oper = operations[rand(1:length(operations))]
t = terminals[rand(1:length(terminals))]
ex = :($oper($ex, $t))
end
return ex
end
which will give the following, for instance:
Main> #macroexpand #random_oper(1, [+, -, /], [1,2,3])
:((-)([3, 3]...))
Main> #macroexpand #random_oper(2, [+, -, /], [1,2,3])
:((+)((-)([2, 3]...), 3))
Thanks Arda for the thorough response! This helped, but part of me thinks there may be a better way to do this as it seems too roundabout. Since I am writing a genetic program, I will need to create 500 of these ASTs, all with random functions and terminals from a set of allowed functions and terminals (fs and ts in the code). I will also need to test each function with 20 different values of x and v.
In order to accomplish this with the information you have given, I have come up with the following macro:
macro create_function(defs)
for name in eval(defs)
ex = quote
function $(Symbol(name))(x,v)
fs = [:+, :-, :DIV, :GT]
ts = [x,v,-1]
return create_single_full_tree(4, fs, ts)
end
end
eval(ex)
end
end
I can then supply a list of 500 random function names in my main() function, such as ["func1, func2, func3,.....". Which I can eval with any x and v values in my main function. This has solved my issue, however, this seems to be a very roundabout way of doing this, and may make it difficult to evolve each AST with each iteration.
I have a multidimensional array (B_matrix) that I need to fill up with some random values. Since the dimension depends on two parameters K and C that are user defined, I cannot use nested loop to fill the array, so I have decided to fill it up recursively.
The problem with the recursion function (fillUp) is that that even though the array is declared outside the function, the array is set to NULL after the function is run.
B_dim = rep(2,((K+1+C)*2))
B_matrix = array( dim = B_dim, dimnames = NULL)
string = c()
fillUp<-function(level, string ){
if (level>=1){
for(i in c(1,2)){
Recall(level-1, c(string, i))
}
}else{
B_matrix[string] = 1;
}
}
fillUp(length(B_dim), string)
> sum( B_matrix == 1)
[1] NA
I'm new to R, so I'm not sure if the "global" declaration allows fillUp to change the values of the matrix.
Edit:
Note that the line
B_matrix[string] = 1;
is just a test case, and the original idea is to assign some random value that depends of the position of the array element.
Edit2:
Based on what #Bridgeburners hinted, I'm almost there. Replacing B_matrix[string] = 1, by
assign('str', matrix(string,1), envir=.GlobalEnv)
assign('hl', B_half_length, envir=.GlobalEnv)
rul <-runif(1, 0, sum(str[1:hl]))
with( .GlobalEnv,B_matrix[str] <- rul)
I get the error (last line):
Error in eval(expr, envir, enclos) : object 'rul' not found
The problem, I guess, is that I'm working with variables from two different environments at the same time. I don't know how to proceed here.
This option doesn't work either
assign('str',matrix(string,1), envir=.GlobalEnv)
assign('hl', B_half_length, envir=.GlobalEnv)
assign('ru', runif(1, 0, sum(str[1:hl])), envir=.GlobalEnv)
with( .GlobalEnv,B_matrix[str] <- ru)
Note: no visible binding for global variable 'ru'
Edit3:
I've finally solved it:
assign('str',matrix(string,1), envir=.GlobalEnv)
with( .GlobalEnv, B_matrix[str] <- runif(1, 0, sum(str[1:B_half_length])-B_half_length+1) )
where B_half_length is a global variable
Whenever a process is working within a function, it's working in a different environment. The object "B_matrix" is defined in the global environment. Since you're nesting environments (2*(K+C+1) times) you're not impacting the original object. If you simply replace line
B_matrix[string] = 1;
with
assign('str', matrix(string,1), envir=.GlobalEnv)
with(.GlobalEnv,B_matrix[str] <- 1)
your code will work. You simply need to specify which environment your expression is working in. (In the first line you're passing the local value of "string" to a global object named "str".)
Note, also, that indexing an array with a vector doesn't work.
That is, "B_matrix[2,2,2,2,2,2]" is not the same as "B_matrix[c(2,2,2,2,2,2)]".
But it works with a matrix
What you want can be achieved with the following line code once you have initialised you B_matrix array:
B_matrix[] <- runif(length(B_matrix))
I have the code
INJ.1<-"I01 I02 I03 I04 I05
2.78E+02 1.82E+03 3.62E+02 2.90E+02 7.73E+02
7.92E+02 1.21E+03 9.33E+02 6.32E+02 5.10E+02
2.30E+03 7.54E+02 9.60E+02 6.29E+02 1.05E+03
3.61E+03 3.05E+02 7.77E+02 5.87E+02 1.02E+03
3.89E+02 1.35E+03 7.66E+02 4.00E+02 7.43E+02
1.31E+03 1.63E+03 8.95E+02 3.85E+02 1.10E+02
1.39E+03 1.16E+03 9.07E+02 4.99E+02 2.48E+02
1.94E+03 1.09E+03 8.34E+02 5.22E+02 2.48E+02
2.04E+03 1.11E+03 7.85E+02 2.67E+02 4.27E+02
1.06E+03 1.36E+03 8.80E+02 6.13E+02 7.16E+02
1.40E+03 1.29E+03 8.65E+02 6.17E+02 9.79E+02
1.20E+03 1.68E+03 6.78E+02 6.10E+02 9.30E+02
1.45E+03 1.49E+03 7.66E+02 3.81E+02 1.07E+03
1.16E+03 1.58E+03 1.09E+03 5.33E+02 8.38E+02
1.33E+03 1.38E+03 9.10E+02 6.29E+02 8.80E+02
"
INJ<-as.matrix(read.table(text=INJ.1, header=T))
PRD.1<-"P01
981.32019
1062.5702
1439.7673
1694.0723
1085.1016
1243.6089
1191.5941
1302.2167
1333.5266
1242.0212
1342.6954
1371.2767
1394.1171
1400.7926
1373.1791
"
PRD<-as.matrix(read.table(text=PRD.1, header=T))
tao=as.matrix(c(1,1,1,1,1))
lambda=as.matrix(c(0.0251879,0.1599486,0.1812318,0.2626731,0.3355733,0.3221295,-1.3343501))
i.dash=matrix(ncol=ncol(INJ), nrow=(nrow(INJ)))
fn1 <- function (tao){
for (i in 1:ncol(INJ))
for (j in 1:nrow (INJ))
temp=0
for (k in 1:j)
i.dash[j,i]=(1/tao[i])*exp((k-j)/tao[i])*INJ[k,i]+i.dash[j,i]
target = abs(700-sum(colSums(i.dash)))
}
ini=c(1, 1, 1, 1, 1)
ans1<-optim(par=ini,fn1,hessian=TRUE)
I need to optimize the values of tao as shown in the function. The code keeps giving the same initial values in in addition to that I noticed that the matrix calculation inside the function fn1 wasn't done. I have more than one question in addition to the main question which is how can I solve this case to achieve the min of o target:
Can we issue non calculation commands inside the function for example: assigning and creating matrices, vectors operations and manipulations..etc?
Are these changes going to be available after we exit the function?
In my case I am using the parameters values in some calculation firstly to prepare the objective function and then I do the optimization on them is that an acceptable approach in R?
I would like some one to give me as much as a starting point to start optimizing this function.