What is the correct way to import local modules in Pluto v0.18.0 notebooks with Julia 1.7.2?
The using keyword adds exported members to the main namespace in Julia. Given a sample module, in foo.jl,
module Foo
export bar
function bar()
return "baz"
end
end
The following code cell works in a Pluto Notebook:
# "baz"
begin
include("./foo.jl")
using .Foo
bar()
end
However, if I attempt to call bar from another cell, I get the following error:
# UndefVarError: bar not defined
bar()
Though I notice that Foo.bar() does work.
How do I add my own modules in a way that I get access to their exported members directly in the notebook's namespace?
This discussion gives a possible solution. It describes a better way of getting the module reference than what I have in my question.
If the module is not the last definition in the file, the import function has to be redefined. A suggested name is to have a variant called "ingredients"
#ingredients (generic function with 1 method)
function ingredients(path::String)
# this is from the Julia source code (evalfile in base/loading.jl)
# but with the modification that it returns the module instead of the last object
name = Symbol(basename(path))
m = Module(name)
Core.eval(m,
Expr(:toplevel,
:(eval(x) = $(Expr(:core, :eval))($name, x)),
:(include(x) = $(Expr(:top, :include))($name, x)),
:(include(mapexpr::Function, x) = $(Expr(:top, :include))(mapexpr, $name, x)),
:(include($path))))
m
end
Then the module can be loaded in like so (I don't know the reason why I have to say ingredients().Foo)
#Main.Foo.jl.Foo
Foo = ingredients("My Library/Foo.jl").Foo
and then with the module reference, you have to manually bring all exported variables into the global scope:
#bar = bar (generic function with 1 method)
bar = Foo.bar
Related
I want to import a function present in a julia file somewhere during runtime
just like in python we have importlib.import_module to import module is there something present in julia
I'm new to julia and I'm not sure how to do that.
I have to import a function main from another julia file and want to run it but I have to also check a condition before that if the condition is true then I want to import the function.
EDIT
I have a file
main.jl
function myMain()
s1 = "Hello"
s2 = "World!"
include("functions/hello.jl")
say(s1, s2)
end
myMain()
hello.jl
function say(s1, s2)
print(s1, s2)
end
Error
ERROR: LoadError: MethodError: no method matching say(::String, ::String)
The applicable method may be too new: running in world age 32378, while current world is 32379.
Closest candidates are:
say(::Any, ::Any) at ~/Desktop/julia_including/functions/hello.jl:1 (method too new to be called from this world context.)
Stacktrace:
[1] myMain()
# Main ~/Desktop/julia_including/main.jl:5
[2] top-level scope
# ~/Desktop/julia_including/main.jl:8
in expression starting at /home/shivansh/Desktop/julia_including/main.jl:8
It works fine when I don't use include inside the myMain() function in main.jl
Julia contrary to Python is a compiled language.
Hence for maximum performance all functions should be known at the compilation time so efficient assembly code can be generated.
Hence generally you want to avoid syntax like the one you are proposing and have the inlude outside of the function.
However, if you really know what you are doing, why you are doing this and need such functionality then comes the world of Julia metaprogramming.
The code is the following:
function myMain()
s1 = "Hello"
s2 = "World!"
include("say.jl")
Base.invokelatest(say, s1, s2)
end
Now you can do:
julia> myMain()
HelloWorld!
Honestly, it just sounds like you wanna do this
if some_condition
include("/path/to/some/file/that/you/need.jl")
else
include("/path/to/some/OTHER/file/that/you/need.jl")
end
EDIT
I think though, after seeing the error that you added, what you wanna do is define your function multiple times instead, and with different argument types:
function say(x::Any, y::Any)
println("Default")
end
function say(x::String, y::Any)
println("String in arg1: $x")
end
function say(x::Any, y::String)
println("String in arg2: $y")
end
function say(x::String, y::String)
println("String in args 1 and 2: $x $y")
end
That way using say differs based on the datatype of the arguments:
julia> say("Foo", "Bar")
String in args 1 and 2: Foo Bar
julia> say("Foo", 2)
String in arg1: Foo
julia> say(0, "Bar")
String in arg2: Bar
and if you have two different functions that act on 2 strings you can do this instead:
struct MySayFunctionType{Symbol}
end
function say(x::String, y::String, ::MySayFunctionType{:Type1})
println("Type1 function is being called: $x $y")
end
function say(x::String, y::String, ::MySayFunctionType{:Type2})
println("Type2 function is being called: $x $y")
end
and then use these like this
if condition
say("Foo", "Bar", MySayFunctionType{Symbol("Type1")}() )
else
say("Foo", "Bar", MySayFunctionType{Symbol("Type2")}() )
end
and if you want a default version of this function to be called -- say Type2 -- you can just add a default value for the third argument like this:
function say(x::String, y::String, z::MySayFunctionType{:Type2}=MySayFunctionType{Symbol("Type2")}())
println("Type2 function is being called: $x $y")
end
The reason this works is because "MySayFunctionType{:Type1}" is considered to be a different type than "MySayFunctionType{:Type2}" so multiple dispatch would consider that as well.
Some of the parameters to a simulation I am writing are functions. When the output is generated, I want to put the definition of these functional parameters in the output. I have in mind a macro that somehow saves the definition as a string, and then defines it. For example, here's what I do now:
borda_score_fn(p) = exp(1/p)
global g_borda_score_fn_string = "exp(1/p)"
And then I write g_borda_score_fn_string to my output. But this is really ugly!
What I would like to do is something like this:
#paramfn borda_score_fn(p) = exp(1/p)
And later be able to both call borda_score_fn(p), and have the form (i.e., "exp(1/p)") available for writing to my output log. (The string form might get stashed in a global dict, actually, they both could.)
I have tried many version of this, but can't get the right set of parses and calls to get it to work. Any help would be appreciated.
This may be a bit different than what you have in mind, but one perhaps "Julian" approach might be to have the function itself return the form string via multiple dispatch, rather than defining a whole new global variable just for that. For example, say we have a type
struct Form end
that we can use for dispatch, then we can write
borda_score_fn(p) = exp(1/p)
borda_score_fn(::Form) = "exp(1/p)"
which can then be retrieved just by calling the function with our type
julia> borda_score_fn(2)
1.6487212707001282
julia> borda_score_fn(Form())
"exp(1/p)"
That might actually be not bad on its own. But, if you want a macro to do both parts at once, then something along the lines of
macro paramfn(e)
name = esc(e.args[1].args[1])
str = string(e.args[2].args[2])
f = esc(e)
quote
$name(::Form) = $str
$f
end
end
would let you write
julia> #paramfn borda_score_fn(p) = exp(1/p)
borda_score_fn (generic function with 2 methods)
julia> borda_score_fn(1)
2.718281828459045
julia> borda_score_fn(Form())
"exp(1 / p)"
For completeness, here's how you can do it in a way more similar to your original approach, but more idiomatically than with a global variable:
julia> module FormOf
export formof, #paramfn
function formof end
macro paramfn(expr)
name = esc(expr.args[1].args[1])
form_str = string(expr.args[2].args[2])
quote
$(esc(expr))
$FormOf.formof(::typeof($name)) = $form_str
$name
end
end
end
Main.FormOf
julia> FormOf.#paramfn borda_score_fn(p) = exp(1/p)
borda_score_fn (generic function with 1 method)
julia> FormOf.formof(borda_score_fn)
"exp(1 / p)"
However, since it defines a new method of FormOf.formof, this only works in global scope:
julia> function bla()
FormOf.#paramfn fn(p) = exp(1/p)
fn(10) + 1
end
ERROR: syntax: Global method definition around REPL[45]:10 needs to be placed at the top level, or use "eval".
Stacktrace:
[1] top-level scope
# REPL[50]:1
#cbk's solution does not have this limitation.
I'm trying to code a macro which calls a function defined within the same module (with Julia v1.1). I always get the ERROR: LoadError: UndefVarError: myModel not defined.
The code of the module is the following (Models.jl):
module Models
export Model
export createmodel
export add_variable
export #add_variable
mutable struct Model
variable_symbols::Dict{String, Symbol}
variable_data::Dict{Symbol, Array{Int64, 1}}
Model(; variable_symbols = Dict([]), variable_data = Dict([])) =
new(variable_symbols, variable_data)
Model(variable_symbols, variable_data) = new(variable_symbols,
variable_data)
end
function createmodel()
model = Model()
return model
end
function add_variable(model, label::String, value::Array{Int64, 1})
symbol = Symbol(label)
model.variable_symbols[label] = Symbol(symbol)
model.variable_data[symbol] = value
end
macro add_variable(model, var)
quote
add_variable($model, $(string(var)), $var)
end
end
end
The macro is called as following:
include("Models.jl")
using .Models
x = [1, 2, 3]
myModel = createmodel()
#add_variable(myModel, x)
show(myModel)
I know that this is an escaping/hygiene problem, but after trying many ideas I don't get the expected result. Up to now, my only way to get the expected result was to define the macro outside of the module to get: Model(Dict("x"=>:x), Dict(:x=>[1, 2, 3]))
You just need to escape the variables that get passed into the macro:
macro add_variable(model, var)
quote
add_variable($(esc(model)), $(string(var)), $(esc(var)))
end
end
All names in a macro are resolved in the context of the module its defined in unless you esc them — with an esc they're resolved in the context of the caller. You almost always want to esc the arguments to the macro. Names like add_variable, though, you want to resolve in the place where the macro itself was defined.
Julia by default imports many names into scope. Is there a way to get a warning when I inadvertently overwrite one of them?
In the context of modules and Base functions, Julia already does warn you if you overwrite a name. See the below examples that work on v0.4.5:
MODULES:
In modA.jl:
module modA
export test
function test()
println("modA")
end
end
In modB.jl:
module modB
export test
function test()
println("modB")
end
end
In REPL:
julia> using modA
julia> using modB
WARNING: Using modB.test in module Main conflicts with an existing identifier
julia> test()
"modA"
BASE FUNCTIONS:
In REPL:
julia> function +(x::Float64, y::Float64)
println("my addition")
end
julia> WARNING: module Main should explicitly import + from Base
WARNING: Method definition +(Float64, Float64) in module Base at float.jl:208
overwritten in module Main at none:2.
As far as I am aware, this does not work with user defined functions; see below:
julia> function test(x::Float64, y::Float64)
println("First Definition")
end
julia> test(1.0, 2.0)
First Definition
julia> function test(x::Float64, y::Float64)
println("Second Definition")
end
julia> test(1.0, 2.0)
Second Definition
Did you have a different context in mind for imported names?
I was wondering if there was a way to programmatically or dynamically import a set of modules into Julia? For example, if I have a list of files that conform to some naming convention that are present at startup that I can grab using something like:
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
which might return a list of files ["mod00.jl", "mod02.jl", "mod05.jl"], is there a way to then import each of the modules in those files. This would be equivalent to having:
import mod00
import mod02
import mod05
in the code if I knew that those modules were available when I wrote the code. Or maybe there is some other approach to doing this that is better. Any suggestions would be much appreciated.
Update
I attempted to do this via a macro, but with no luck. For example:
macro import_mod(modn)
quote
import $modn
end
end
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
println(module_files)
for modname in factor_files
modn = modname[1:end-3]
println(modn)
#import_mod modn
end
end
When I run this, I get ERROR: syntax: invalid "import" statement. I tried various escaping strategies but all failed similarly.
NOTE: Please refer to user1712368's (Jameson Nash, Julia dev) answer, this discussion at the julia-users mailing list and this entry of the Julia manual, to know about why this is not the correct answer.
This is how a multiple import expression looks like, using Julia version 0.3.4:
julia> parse("import Foo, Bar")
:($(Expr(:toplevel, :($(Expr(:import, :Foo))), :($(Expr(:import, :Bar))))))
julia> dump(ans)
Expr
head: Symbol toplevel
args: Array(Any,(2,))
1: Expr
head: Symbol import
args: Array(Any,(1,))
1: Symbol Foo
typ: Any
2: Expr
head: Symbol import
args: Array(Any,(1,))
1: Symbol Bar
typ: Any
typ: Any
Here is a macro which does this programmatically, it takes a modules argument that could be a :call or :vcat Expr or a Symbol, which has to evaluate to a Vector{Symbol}:
julia> macro dynamic_import(modules)
(modules = eval(modules))::Vector{Symbol}
ex = Expr(:toplevel)
for m in modules
push!(ex.args, Expr(:import, m))
end
return ex
end
You could also generalize this line:
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
By abstracting it into a function that takes a Regex and a String directory path as arguments and returns a Vector{Symbol}:
julia> function needed_modules(rx::Regex, dir::String=".")
module_files = filter(rx, readdir(dir))
module_syms = map(m -> symbol(split(m, '.')[1]), module_files)
end
needed_modules (generic function with 2 methods)
So you may use it like this:
julia> #dynamic_import [:Mod01, :Mod02] # :vcat expression
julia> rx = r"^Mod[0-9][0-9].jl$";
julia> #dynamic_import needed_modules(rx) # :call expression
julia> modules = needed_modules(rx)
2-element Array{Symbol,1}:
:Mod01
:Mod02
julia> #dynamic_import modules # Symbol
Finally you could wrap it all into a module so you may use using DynamicImport:
Note: currently I get this when trying to run the same examples from a module:
julia> using DynamicImport
julia> #dynamic_import [:mod01, :mod02]
julia> rx = r"^mod[0-9][0-9].jl$";
julia> #dynamic_import needed_modules(rx)
ERROR: rx not defined
julia> modules = needed_modules(rx)
2-element Array{Symbol,1}:
:mod01
:mod02
julia> #dynamic_import modules
ERROR: modules not defined
But it works fine If I define the objects inside the REPL, I guess this is an issue involving hygiene, which is something I'm not experienced with, so I'll ask at the julia-users mailing list.
The function version of import X is require("X"), so you should be able to do:
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
println(module_files)
for modname in factor_files
modn = modname[1:end-3]
println(modn)
require(modn)
end
end
Then, assuming each of these defined a module of the same name, you can collect them into an array:
modules = Module[]
...
if isdefined(Main, symbol(modn))
push!(modules, getfield(Main, symbol(modn))
else
warn("importing $modn did not defined a module")
end
...
return modules
I'm not thrilled with this solution, but it seems to work. Basically I dynamically create a temporary file contains the import statements that I then include it. So:
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
path, f = mktemp()
for modname in module_files
modn = splitext(modname)[1]
write(f, "import $modn\n")
end
close(f)
include(path)
rm(path)
end
I would be curious to know if there is a more idiomatic strategy since I'm new to Julia and just wadding through its capabilities.
Edit
The strategy of setting up a temporary file is likely unnecessary since Julia also provides an include_string method.
On the Julia-user list, #Isaiah also suggests using eval(Expr(:import, symbol(modn)))