I am new to Julia and trying to do a very simple task:
to distribute variable x=1 to every process
change x to a different value only in process 2 (change x local to process 2)
print the new value of remote x in process 2
My code is:
using Distributed
function f()
x=10*x
println(x)
end
#everywhere x=1
remote_do(f,2)
It does not print anything.
Another try with remotecall:
r=remotecall(x->10*x,2,x)
fetch(r)
println(x)
prints 10 (what anonymous function returns) and 1 (x in process 1) as expected.
As I understand remotecall returns a future with result of lambda x->10x but does not change the remote variable. In fact it even does not multiply the remote variable but x in process 1!
Question: How to change and read remote variable in process 1?
First, x is scoped locally to f(), so even running f() on the local process produces an error:
julia> f()
ERROR: UndefVarError: x not defined
If you really want to use a global variable here, you need to tell Julia that:
function f()
global x=10*x
println(x)
end
julia> f()
10
Then, to see why it isn't running remotely, you can try remotecall_fetch to make the call synchronously and see any exceptions (without this exceptions go to stderr on the remote worker).
julia> remotecall_fetch(f, 2)
ERROR: On worker 2:
UndefVarError: #f not defined
The remote worker does not have a definition for f.
#everywhere function f()
global x=10*x
println(x)
end
julia> remote_do(f,2)
julia> From worker 2: 10
To make this easier when you have more code, you could place the code in a module and then call #everywhere using MyModule.
Related
I would like to build a Julia application where a user can specify a function using a configuration file (and therefore as a string). The configuration file then needs to be parsed before the function is evaluated in the program.
The problem is that while the function name is known locally, it is not known in the module containing the parser. One solution I have come up with is to pass the local eval function to the parsing function but that does not seem very elegant.
I have tried to come up with a minimal working example here, where instead of parsing a configuration file, the function name is already contained in a string:
module MyFuns
function myfun(a)
return a+2
end
end
module MyUtil
# in a real application, parseconfig would parse the configuration file to extract funstr
function parseconfig(funstr)
return eval(Meta.parse(funstr))
end
function parseconfig(funstr, myeval)
return myeval(Meta.parse(funstr))
end
end
# test 1 -- succeeds
f1 = MyFuns.myfun
println("test1: $(f1(1))")
# test 2 -- succeeds
f2 = MyUtil.parseconfig("MyFuns.myfun", eval)
println("test2: $(f2(1))")
# test 3 -- fails
f3 = MyUtil.parseconfig("MyFuns.myfun")
println("test3: $(f3(1))")
The output is:
test1: 3
test2: 3
ERROR: LoadError: UndefVarError: MyFuns not defined
So, the second approach works but is there a better way to achieve the goal?
Meta.parse() will transform your string to an AST. What MyFuns.myfun refers to depends on the scope provided by the eval() you use.
The issue with your example is that the eval() inside MyUtil will evaluate in the context of that module. If that is the desired behavior, you simply miss using MyFuns inside MyUtil.
But what you really want to do is write a macro. This allows the code to be included when parsing your program, before running it. The macro will have access to a special argument __module__, which is the context where the macro is used. So __module__.eval() will execute an expression in that very scope.
foo = "outside"
module MyMod
foo = "inside"
macro eval(string)
expr = Meta.parse(string)
__module__.eval(expr)
end
end
MyMod.#eval "foo"
# Output is "outside"
See also this explanation on macros:
https://docs.julialang.org/en/v1/manual/metaprogramming/index.html#man-macros-1
And for the sake of transforming the answer of #MauricevanLeeuwen into the framework of my question, this code will work:
module MyFuns
function myfun(a)
return a+2
end
end
module MyUtil
macro parseconfig(funstr)
__module__.eval(Meta.parse(funstr))
end
end
f4 = MyUtil.#parseconfig "MyFuns.myfun"
println("test4: $(f4(1))")
I have a script that defines a function, and later intended to call the function but forgot to add the parentheses, like this:
function myfunc()
println("inside myfunc")
end
myfunc # This line is silently ignored. The function isn't invoked and there's no error message.
After a while I did figure out that I was missing the parentheses, but since Julia didn't give me an error, I'm wondering what that line is actually doing? I'm assuming that it must be doing something with the myfunc statement, but I don't know Julia well enough to understand what is happening.
I tried --depwarn=yes but don't see a julia command line switch to increase the warning level or verbosity. Please let me know if one exists.
For background context, the reason this came up is that I'm trying to translate a Bash script to Julia, and there are numerous locations where an argument-less function is defined and then invoked, and in Bash you don't need parentheses after the function name to invoke it.
The script is being run from command line (julia stub.jl) and I'm using Julia 1.0.3 on macOS.
It doesn't silently ignore the function. Calling myfunc in an interactive session will show you what happens: the call returns the function object to the console, and thus call's the show method for Function, showing how many methods are currently defined for that function in your workspace.
julia> function myfunc()
println("inside myfunc")
end
myfunc (generic function with 1 method)
julia> myfunc
myfunc (generic function with 1 method)
Since you're calling this in a script, show is never called, and thus you don't see any result. But it doesn't error, because the syntax is valid.
Thanks to DNF for the helpful comment on it being in a script.
It does nothing.
As in c, an expression has a value: in c the expression _ a=1+1; _ has the value _ 2 _ In c, this just fell out of the parser: they wanted to be able to evaluate expressions like _ a==b _
In Julia, it's the result of designing a language where the code you write is handled as a data object of the language. In Julia, the expression "a=1+1" has the value "a=1+1".
In c, the fact that
a=1+1;
is an acceptable line of code means that, accidentally,
a;
is also an acceptable line of code. The same is true in Julia: the compiler expects to see a data value there: any data value you put may be acceptable: even for example the data value that represents the calculated value returned by a function:
myfunc()
or the value that represents the function object itself:
myfunc
As in c, the fact that data values are everywhere in your code just indicates that the syntax allows data values everywhere in your code and that the compiler does nothing with data values that are everywhere in your code.
I want to run multiple 'processes' in parallel in julia
For this, I use #spawn
However I don't see errors that occur in the spawned subprocess
The subprocess dies, but no error message
In a terminal, run julia and then, inside the julia vm(?), type the following:
function sub()
println("1")
error("2")
println("3")
end
subproc = #spawn sub()
It prints '1', but nothing more
Is using #spawn the correct way? Where is the error output stream gone to? How can I see errors?
Thanks
Imran
From your description I guess you didn't really launch more than 1 process, otherwise you will get "From worker X: 1" rather than just "1". You should use julia -p X or addprocs(X) to launch more processes.
To receive the error message or any other data from subprocesses, generally you need a fetch operation. Read the manual for more details.
Here is an example that runs "actual" subprocess and displays the error message.
nprocs()<=1 && addprocs()
#everywhere function sub()
println(1)
error(2)
println(2)
end
subproc = #spawn sub()
wait(subproc)
When generating a not explicitly generated version of a function, #ngenerate runs
eval(quote
local _F_
$localfunc # Definition of _F_ for the requested value of N
_F_
end)
Since eval runs in the scope of the current module, not the function, I wonder what is the effect of local in this context. As far as I know, the languange documentation only mentions the use of local inside function definitions.
To give some background why this question arose: I frequently need to code loops of the form
function foo(n::Int)
s::Int = 0
for i in 1:1000000000
for j in 1:n
s += 1
end
end
return s
end
where n <= 10 (of course, in my actual code the loops are such that they cannot just be reduced to O(1)). Because this code is very simple for the compiler but demanding at runtime, it turns out to be beneficial to simply recompile the loops with the required value of n each time foo is called.
function clever_foo(n::Int)
eval(quote
function clever_foo_impl()
s::Int = 0
for i in 1:1000000000
s += $(Expr(:call,:+,[1 for j in 1:n]...))
end
return s
end
end)
return clever_foo_impl()
end
However, I am not sure whether I am doing this the right way.
It's to prevent _F_ from being visible in the global method cache.
If you'll call clever_foo with the same n repeatedly, you can do even better by saving the compiled function in a Dict. That way you don't have to recompile it each time.
This is just a convenience but I think useful. Note that IPython allows a pure quit as does Matlab. Thus it would be reasonble in Julia to allow aliasing.
Thanks for any ideas as to how to do this.
Quitting in Julia
If you are using Julia from the command line then ctrl-d works. But if your intention is to quit by typing a command this is not possible exactly the way you want it because typing quit in the REPL already has a meaning which is return the value associated with quit, which is the function quit.
julia> quit
quit (generic function with 1 method)
julia> typeof(quit)
Function
Also Python
But that's not rare, for example Python has similar behavior.
>>> quit
Use quit() or Ctrl-D (i.e. EOF) to exit
Using a macro
Using \q might be nice in the Julia REPL like in postgres REPL, but unfortunately \ also already has a meaning. However, if you were seeking a simple way to do this, how about a macro
julia> macro q() quit() end
julia> #q
Causes Julia to Quit
If you place the macro definition in a .juliarc.jl file, it will be available every time you run the interpreter.
As waTeim notes, when you type quit into the REPL, it simply shows the function itself… and there's no way to change this behavior. You cannot execute a function without calling it, and there are a limited number of ways to call functions in Julia's syntax.
What you can do, however, is change how the Functions are displayed. This is extremely hacky and is not guaranteed to work, but if you want this behavior badly enough, here's what you can do: hack this behavior into the display method.
julia> function Base.writemime(io::IO, ::MIME"text/plain", f::Function)
f == quit && quit()
if isgeneric(f)
n = length(f.env)
m = n==1 ? "method" : "methods"
print(io, "$(f.env.name) (generic function with $n $m)")
else
show(io, f)
end
end
Warning: Method definition writemime(IO,MIME{symbol("text/plain")},Function) in module Base at replutil.jl:5 overwritten in module Main at none:2.
writemime (generic function with 34 methods)
julia> print # other functions still display normally
print (generic function with 22 methods)
julia> quit # but when quit is displayed, it actually quits!
$
Unfortunately there's no type more specific than ::Function, so you must completely overwrite the writemime(::IO,::MIME"text/plain",::Function) definition, copying its implementation.
Also note that this is pretty unexpected and somewhat dangerous. Some library may actually end up trying to display the function quit… causing you to lose your work from that session.
Related to Quitting in Julia
I was searching for something simple. This question hasn't been updated since 2017, as I try to learn Julia now, and spend some time googling for something simple and similar to python. Here, what I found:
You can use:
exit()
Note
I use julia 1.53