In OrdinaryDiffEq's solve.jl, supposedly the default for progress_message is ODE_DEFAULT_PROG_MESSAGE (line 58), which by default is set to show dt, t, and the maximum of u as stated here (lines 21-22). However, I'm using it with TerminalLoggers to run the code in Jupyter Lab and/or terminal, and instead, I'm getting the message "ETA: (some time number)".
using Logging: global_logger
using TerminalLoggers: TerminalLogger
global_logger(TerminalLogger())
using OrdinaryDiffEq
function lorenz(du,u,p,t)
du[1] = 10.0*(u[2] - u[1])
du[2] = u[1]*(28.0 - u[3]) - u[2]
du[3] = u[1]*u[2] - (8/3)*u[3]
end
u0 = [1.0; 0.0; 0.0]
tspan = (0.0, 100.0)
prob = ODEProblem(lorenz, u0, tspan)
sol = solve(prob, Tsit5(), progress=true)
Terminal output:
ODE 0%| | ETA: N/A
ODE 77%|████████████████████████████████████▎ | ETA: 0:00:00
ODE 100%|███████████████████████████████████████████████| Time: 0:00:01
The ETA seems to be part of the TerminalLoggers defaults.
The documentation is relatively terse and doesn't explain what to pass to the progress_message solver option to change the message. Passing strings like progress_message="dt" doesn't work. Passing in the exact function for ODE_DEFUALT_PROG_MESSAGE as defined in DiffEqBase.jl/src/common_defaults.jl does not change the behavior at all.
Is this a mistake for which I should file an issue with OrdinaryDiffEq, or is there some way I can manipulate progress_message in the common solver options to actually get dt, t, and u?
The documentation is relatively terse and doesn't explain what to pass to the progress_message solver option to change the message. Passing strings like progress_message="dt" doesn't work. Passing in the exact function for ODE_DEFUALT_PROG_MESSAGE as defined in DiffEqBase.jl/src/common_defaults.jl does not change the behavior at all.
These options work on any logger which supports them. The default logger in Juno(/Atom) supports it and uses these messages. I do not think TerminalLoggers does, which is an issue of TerminalLoggers.
Related
I have a program for doing Fourier series and I wanted to switch to CuArrays to make it faster. The code is as follows (extract):
#Arrays I want to use
coord = CuArray{ComplexF64,1}(complex.(a[:,1],a[:,2]))
t=CuArray{Float64,1}(-L:(2L/(N-1)):L)
#Array of indexes in the form [0,1,-1,2,-2,...]
n=[((-1)^i)div(i,2) for i in 1:grado]
#Array of functions I need for calculations
base= [x -> exp(π * im * i * x / L) / L for i in n]
base[i](1.) #This line is OK
base[i](-1:.1:1) #This line is OK
base[i].(t) #This line gives error!
base[i].(CuArray{Float64,1}(t)) #This line gives error!
And the error is:
GPU broadcast resulted in non-concrete element type Any.
This probably means that the function you are broadcasting contains an error or type instability.
If I change it like this
base= [(x::Float64) -> (exp(π * im * i * x / L) / L)::ComplexF64 for i in n]
the same lines still give error, but the error now is:
UndefVarError: parameters not defined
Any idea how I could fix this?
Thank you in advance!
Package information:
(#v1.6) pkg> st CUDA
Status `C:\Users\marce\.julia\environments\v1.6\Project.toml`
[052768ef] CUDA v2.6.2
P.S.: This other function has the same problem:
function integra(inizio, fine, arr)
N=size(arr,1)
h=(fine-inizio)/N
integrale=sum(arr)
integrale -= (first(arr)+last(arr))/2
integrale *= h
end
L=2
integra(-L,L,coord)
The first and easier problem is that you should take care to declare global variables to be constant so that the compiler can assume a constant type: const L = 2. A mere L = 2 allows you to do something like L = SomeOtherType(), and if that type can be Anything, so must the return type of your functions. On the CPU that's only a performance hit, but it's a no-no for the GPU. If you actually want L to vary in value, pass it in as an argument so the compiler can still infer types within a function.
Your ::ComplexF64 assertion did actually force a concrete return type, though the middle of the function is still type unstable (check with #code_warntype). The second problem you ran into after that patch was probably caused by this recently patched conflict between ExprTools.jl and LLVM.jl. Seems like you just need to update the packages or maybe reinstall them.
I have an ODE that I need to solver over a wide range of parameters.
Previously I have used MATLAB's parfor to divide the parameter ranges between multiple threads. I am new to Julia and need to do the same thing in Julia now. Here is the code that I am using
using DifferentialEquations, SharedArrays, Distributed, Plots
function SingleBubble(du,u,p,t)
du[1]=#. u[2]
du[2]=#. ((-0.5*u[2]^2)*(3-u[2]/(p[4]))+(1+(1-3*p[7])*u[2]/p[4])*((p[6]-p[5])/p[2]+2*p[1]/(p[2]*p[8]))*(p[8]/u[1])^(3*p[7])-2*p[1]/(p[2]*u[1])-4*p[3]*u[2]/(p[2]*u[1])-(1+u[2]/p[4])*(p[6]-p[5]+p[10]*sin(2*pi*p[9]*t))/p[2]-p[10]*u[1]*cos(2*pi*p[9]*t)*2*pi*p[9]/(p[2]*p[4]))/((1-u[2]/p[4])*u[1]+4*p[3]/(p[2]*p[4]))
end
R0=2e-6
f=2e6
u0=[R0,0]
LN=1000
RS = SharedArray(zeros(LN))
P = SharedArray(zeros(LN))
bif = SharedArray(zeros(LN,6))
#distributed for i= 1:LN
ps=1e3+i*1e3
tspan=(0,60/f)
p=[0.0725,998,1e-3,1481,0,1.01e5,7/5,R0,f,ps]
prob = ODEProblem(SingleBubble,u0,tspan,p)
sol=solve(prob,Tsit5(),alg_hints=:stiff,saveat=0.01/f,reltol=1e-8,abstol=1e-8)
RS[i]=maximum((sol[1,5000:6000])/R0)
P[i]=ps
for j=1:6
nn=5500+(j-1)*100;
bif[i,j]=(sol[1,nn]/R0);
end
end
plotly()
scatter(P/1e3,bif,shape=:circle,ms=0.5,label="")#,ma=0.6,mc=:black,mz=1,label="")
When using one worker, the for loop is basically executed as a normal single threaded loop and it works fine. However, when I am using addprocs(n) to add n more workers, nothing gets written into the SharedArrays RS, P and bif. I appreciate any guidance anyone may provide.
These changes are required to make your program work with multiple workers and display the results you need:
Whatever packages and functions are used under #distributed loop must be made available in all the processes using #everywhere as explained here. So, in your case it would be DifferentialEquations and SharedArrays packages as well as the SingleBubble() function.
You need to draw the plot only after all the workers have finished their tasks. For this, you would need to use #sync along with #distributed.
With these changes, your code would look like:
using Distributed, Plots
#everywhere using DifferentialEquations, SharedArrays
#everywhere function SingleBubble(du,u,p,t)
du[1]=#. u[2]
du[2]=#. ((-0.5*u[2]^2)*(3-u[2]/(p[4]))+(1+(1-3*p[7])*u[2]/p[4])*((p[6]-p[5])/p[2]+2*p[1]/(p[2]*p[8]))*(p[8]/u[1])^(3*p[7])-2*p[1]/(p[2]*u[1])-4*p[3]*u[2]/(p[2]*u[1])-(1+u[2]/p[4])*(p[6]-p[5]+p[10]*sin(2*pi*p[9]*t))/p[2]-p[10]*u[1]*cos(2*pi*p[9]*t)*2*pi*p[9]/(p[2]*p[4]))/((1-u[2]/p[4])*u[1]+4*p[3]/(p[2]*p[4]))
end
R0=2e-6
f=2e6
u0=[R0,0]
LN=1000
RS = SharedArray(zeros(LN))
P = SharedArray(zeros(LN))
bif = SharedArray(zeros(LN,6))
#sync #distributed for i= 1:LN
ps=1e3+i*1e3
tspan=(0,60/f)
p=[0.0725,998,1e-3,1481,0,1.01e5,7/5,R0,f,ps]
prob = ODEProblem(SingleBubble,u0,tspan,p)
sol=solve(prob,Tsit5(),alg_hints=:stiff,saveat=0.01/f,reltol=1e-8,abstol=1e-8)
RS[i]=maximum((sol[1,5000:6000])/R0)
P[i]=ps
for j=1:6
nn=5500+(j-1)*100;
bif[i,j]=(sol[1,nn]/R0);
end
end
plotly()
scatter(P/1e3,bif,shape=:circle,ms=0.5,label="")#,ma=0.6,mc=:black,mz=1,label="")
Output using multiple workers:
Following the approach of this answer I am trying to understand what happens exactly and how expressions and generated functions work in Julia within the concept of metaprogramming.
The goal is to optimize a recursive function using expressions and generated functions (for a concrete example you can have a look at the question answered in the link provided above).
Consider the following modified fibonacci function, in which I want to compute the fibonacci series up to n and multiply it by a number p.
The straightforward, recursive implementation would be
function fib(n::Integer, p::Real)
if n <= 1
return 1 * p
else
return n * fib(n-1, p)
end
end
As a first step, I could define a function which returns an expression instead of the computed value
function fib_expr(n::Integer, p::Symbol)
if n <= 1
return :(1 * $p)
else
return :($n * $(fib_expr(n-1, p)))
end
end
which, e.g. returns something like
julia> ex = fib_expr(3, :myp)
:(3 * (2 * (1myp)))
In this way I get an expression which is fully expanded and depends on the value assigned to the symbol myp. In this way I do not see the recursion anymore, basically I am metaprogramming: I created a function that creates another "function" (in this case we call it expression though).
I can now set myp = 0.5 and call eval(ex) to compute the result.
However, this is slower than the first approach.
What I can do though, is to generate a parametric function in the following way
#generated function fib_gen{n}(::Type{Val{n}}, p::Real)
return fib_expr(n, :p)
end
And magically, calling fib_gen(Val{3}, 0.5) gets things done, and is incredibly fast.
So, what is going on?
To my understanding, in the first call to fib_gen(Val{3}, 0.5), the parametric function fib_gen{Val{3}}(...) gets compiled and its content is the fully expanded expression obtained through fib_expr(3, :p), i.e. 3*2*1*p with p substituted with the input value.
The reason why it is so fast then, is because fib_gen is basically just a series of multiplications, whereas the original fib has to allocate on the stack every single recursive call making it slower, am I correct?
To give some numbers, here is my short benchmark using BenchmarkTools.
julia> #benchmark fib(10, 0.5)
...
mean time: 26.373 ns
...
julia> p = 0.5
0.5
julia> #benchmark eval(fib_expr(10, :p))
...
mean time: 177.906 μs
...
julia> #benchmark fib_gen(Val{10}, 0.5)
...
mean time: 2.046 ns
...
I have many questions:
Why the second case is so slow?
What exactly is and means ::Type{Val{n}}? (I copied that from the answer linked above)
Because of the JIT compiler, sometimes I am lost in what happens at compile-time and at run-time, as it is the case here...
Furthermore, I tried to combine fib_expr and fib_gen in a single function according to
#generated function fib_tot{n}(::Type{Val{n}}, p::Real)
if n <= 1
return :(1 * p)
else
return :(n * fib_tot(Val{n-1}, p))
end
end
which however is slow
julia> #benchmark fib_tot(Val{10}, 0.5)
...
mean time: 4.601 μs
...
What am I doing wrong here? Is it even possible to combine fib_expr and fib_gen in a single function?
I realize this is more a monograph rather than a question, however, even though I read the metaprogramming section few times, I am having a hard time to grasp everything, in particular with an applied example such as this one.
A monograph in response:
Metaprogramming basics
It will be easier to start with "normal" macros first. I'll relax the definition you used a bit:
function fib_expr(n::Integer, p)
if n <= 1
return :(1 * $p)
else
return :($n * $(fib_expr(n-1, p)))
end
end
That allows to pass in more than just symbols for p, like integer literals or whole expressions. Given this, we can define a macro for the same functionality:
macro fib_macro(n::Integer, p)
fib_expr(n, p)
end
Now, if #fib_macro 45 1 is used anywhere in the code, at compile time it will first be replaced by a long nested expression
:(45 * (44 * ... * (1 * 1)) ... )
and then compiled normally -- to a constant.
That's all there is to macros, really. Replacing syntax during compile time; and by recursion, this can be an arbitrarily long alteration between compiling, and evaluating functions on expressions. And for things that are essentially constant, but tedious to write otherwise, it is very useful: a bood example example is Base.Math.#evalpoly.
Evaluation at runtime?
But it has the problem that you cannot inspect values which are only known at runtime: you can't implement fib(n) = #fib_macro n 1, since at compile time, n is a symbol representing the parameter, and not a number you can dispatch on.
The next best solution to this would be to use
fib_eval(n::Integer) = eval(fib_expr(n, 1))
which works, but will repeat the compilation process every time it is called -- and that is much more overhead than the original function, since now at runtime, we perform the whole recursion on the expression tree and then call the compiler on the result. Not good.
Method dispatch & compilation
So we need a way to intermingle runtime and compile time. Enter #generated functions. These will at runtime dispatch on a type, and then work like a macro defining the function body.
First about type dispatch. If we have
f(x) = x + 1
and have a function call f(1), about the following will happen:
The type of the argument is determined (Int)
The method table of the function is consulted to find the best matching method
The method body is compiled for the specific Int argument type, if that hasn't been done before
The compiled method is evaluated on the concrete argument
If we then enter f(1.0), the same will happen again, with a new, different specialized method being compiled for Float64, based on the same function body.
Value types & singleton types
Now, Julia has the peculiar feature that you can use numbers as types. That means that the dispatch process outlined above will also work on the following function:
g(::Type{Val{N}}) where N = N + 1
That's a bit tricky. Remember that types are themselves values in Julia: Int isa Type.
Here, Val{N} is for every N a so-called singleton type having exactly one instance, namely Val{N}() -- just like Int is a type having many instances 0, -1, 1, -2, ....
Type{T} is also a singleton type, having as its single instance the type T. Int is a Type{Int}, and Val{3} is a Type{Val{3}} -- in fact, both are the only values of their type.
So, for each N, there is a type Val{N}, being the single instance of Type{Val{N}}. Thus, g will be dispatched and compiled for each single N. This is how we can dispatch on numbers as types. This already allows for optimization:
julia> #code_llvm g(Val{1})
define i64 #julia_g_61158(i8**) #0 !dbg !5 {
top:
ret i64 2
}
julia> #code_llvm f(1)
define i64 #julia_f_61076(i64) #0 !dbg !5 {
top:
%1 = shl i64 %0, 2
%2 = or i64 %1, 3
%3 = mul i64 %2, %0
%4 = add i64 %3, 2
ret i64 %4
}
But remember that it requires compilation for each new N at the first call.
(And fkt(::T) is just short for fkt(x::T) if you don't use x in the body.)
Integrating generating functions and value types
Finally to generated functions. They work as a slight modification of the above dispatch pattern:
The type of the argument is determined (Int)
The method table of the function is consulted to find the best matching method
The method body is treated as a macro and called with the Int argument type as a parameter, if that hasn't been done before. The resulting expression is compiled into a method.
The compiled method is evaluated on the concrete argument
This pattern allows to change the implementation for each type which the function is dispatched on.
For our concrete setting, we want to dispatch on the Val types representing the arguments of the Fibonacci sequence:
#generated function fib_gen{n}(::Type{Val{n}}, p::Real)
return fib_expr(n, :p)
end
You now see that your explanation was exactly right:
in the first call to fib_gen(Val{3}, 0.5), the parametric function
fib_gen{Val{3}}(...) gets compiled and its content is the fully
expanded expression obtained through fib_expr(3, :p), i.e. 3*2*1*p
with p substituted with the input value.
I hope that the whole story has also answered all three of your listed questions:
The implementation using eval replicates the recursion every time, plus the overhead of compilation
Val is a trick to lift numbers to types, and Type{T} the singleton type containing only T -- but I hope the examples were helpful enough
Compile time is not before execution, because of JIT -- it is every time a method gets compiled first time, because it get's called.
First of all, I am joining myself to the comments: your question is very well written & constructive.
I have reproduced your results using Julia 0.7-beta.
Difference between #generated fib_tot (one piece of code) and fib_gen (that calls fib_expr)
With my julia version results are identicals:
julia> #btime fib_tot(Val{10},0.5)
0.042 ns (0 allocations: 0 bytes)
1.8144e6
julia> #btime fib_gen(Val{10},0.5)
0.042 ns (0 allocations: 0 bytes)
1.8144e6
Sometimes breaking a function into multiple parts see official doc:performance tips can be useful, however in your peculiar case I do not see why this could be useful. At compile time Julia has everything it needs to optimize fib_tot. There is a branch if n<=1 however n is known at "compile time" thanks to the Type{Val{n}} trick and this branch should be removed without problem in the generated (specialized) code.
The Type{Val{n}} trick
To specialize functions, Julia inference is performed according to argument type and not according to argument value.
For instance a compiled version of foo(n::Int) = ... is not generated for each n value. You must define a type that depends on n value to reach this goal. This is precisely how Type{Val{n}} works: Val{n} is simply a parametrized empty structure:
struct Val{T} end
Hence, each Val{1}, Val{2}, ... Val{100}, ... is a different type. By consequence, if foo is defined as:
foo(::Type{Val{n}}) where {n} = ...
Each foo(Val{1}), foo(Val{2}), ... foo(Val{100}) will trigger a specialized foo version (because argument type is different).
The eval(fib_expr(n, 1)) case
This
julia> #btime eval(fib_expr(10, :p))
401.651 μs (99 allocations: 6.45 KiB)
1.8144e6
is slow because your expression is (re-)compiled every time. The problem can be avoided if you use a macro instead (see phg answer).
The fib version
.
julia> #btime fib(10,0.5)
30.778 ns (0 allocations: 0 bytes)
1.8144e6
There is only one compiled version of this fib function. By consequence, it must contain all the runtime branch tests etc... This explains how slow it is.
Just a remark about:
foo{n}(::Type{Val{n}}) deprecated syntax
The foo{n}(::Type{Val{n}}) syntax is deprecated, the new one is foo(::Type{Val{n}}) where {n}. You can read Julia doc, parametric methods for further details.
My Julia version:
julia> versioninfo()
Julia Version 0.7.0-beta.0
Commit f41b1ecaec (2018-06-24 01:32 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Xeon(R) CPU E5-2603 v3 # 1.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.0 (ORCJIT, haswell)
I'm new to working with Scilab, and I'm trying to run the code below, but when I try it keeps showing me this error::
test3(1000) //Line that I type to run the code
!--error 4 //First error
Undefined variable: cputime
at line 2 of function test3 called by:
I ran it using MATLAB, and it worked, but I can't figure out how to make it run using Scilab.
For sample code when typed using the Scilab editor, see below.
function test3(n)
t = cputime;
for (j = 1:n)
x(j) = sin(j);
end
disp(cputime - t);
Typing help cputime in the Scilab console will reveal that this is not a Scilab function. The near-equivalent Scilab function is timer(), but its behavior is a bit different:
cputime in Matlab measures time since Matlab started
timer() measures time since the last call to timer()
Here is your function rewritten in Scilab:
function test3(n)
timer()
for j = 1:n
x(j) = sin(j)
end
disp(timer())
endfunction
Note that Scilab functions must end with endfunction, and that semicolons are optional: line-by-line output is suppressed in Scilab by default.
For completeness, I'll mention tic() and toc(), which work just like Matlab's tic and toc, measuring real-world time of computation.
I am in the process of trying to learn a little about using Julia by translating an old example code which solves the time dependent Schrodinger equation. Here is what I have so far:
require("setparams.jl")
require("V.jl")
require("InitialRIM.jl")
#require("expevolve.jl")
function doit()
Nspace, R, Im, x0, width, k0, xmax, xmin, V0, a, dx, dx2, n, dt = setparams()
R, Im = initialRIm(width,n,k0,dt,xmin)
ProbDen = zeros(Nspace)
ProbDen = R.*R + Im.*Im
plot(ProbDen)
#Imold = Im;
t=0.0
#t, R =evolve!(R,Im,t,V0,width,a,dx,dx2,dt,xmin,n)
println("Done")
end
After requiring the above code, I then do using Winston. Then I attempt to run the code by typing doit(). Nothing appears.
Can someone please let me know what I am doing wrong? I can provide the innards of setuparame() if needed, as well as initialRIm() but thought at first I'd ask whether my expectations about what should happen are in fault. Note that if I run setuparams() and initialRIm() in a terminal session, then do the plot(ProbDen), the correct graph appears.
Thanks for your help.
Update:
I have now restarted julia, have done using Winston, and then have done doit() to wit:
julia> using Winston
julia> require("driveSch.jl")
julia> doit()
ERROR: dx not defined
in initialRIm at /Users/comerduncan/juliaexamples/TDSch/InitialRIM.jl:8
in doit at /Users/comerduncan/juliaexamples/TDSch/driveSch.jl:11
However, the call to setparams() sets dx along with all the other things. This I see when I run the setparams() interactively. So I am not understanding what is the problem...
It seems like you use dx in initialRIm, but dx is not one of the arguments you pass to it. If you access a variable that is not a parameter nor assigned inside a Julia function, Julia will look for a variable with the same name in the surrounding scopes. When you run
Nspace, R, Im, x0, width, k0, xmax, xmin, V0, a, dx, dx2, n, dt = setparams()
in the global scope, and you create a global variable dx that initialRIm could access. When you wrap the calls into a function, you create a local variable dx that can not be accessed from initialRIm.