Is there a way I can localize memoization (via Memoize.jl) inside a function? or at least delete the dictionaries created by memoization?
Clarification: suppose I define a define a function f(x, y). I want to start with a fresh table for every new value of y. That is, given y = y0, f( . , y0) iterates on itself for x, x-1, etc but given a new y = y1, I don't need to store the old table for y0, so that memory can be freed up. How can I do that?
Solution:
cachedfib() = begin
global dict = Dict()
global dict2 = Dict()
function _fib(n::Int, a::Int)
if !haskey(dict2, a)
dict2[a] = true
dict = Dict()
end
if haskey(dict, (n, a))
return dict[(n, a)]
elseif n < 2
dict[(0, a)] = 0
dict[(1, a)] = 1
return dict[(n, a)]
else
dict[(n, a)] = a*(_fib(n - 1, a) + _fib(n - 2, a))
return dict[(n, a)]
end
end
end
fib = cachedfib()
fib(10, 1)
fib(10, 2)
now call dict and dict2 and check that the dictionaries are refreshed every time the second argument changes. You can get even better performance when the parameters to store are integers and you use Array instead of Dict
To use memoization technique you can do it with let or closures. Have a look at my rapid implementation of factorial (with closure).
with_cached_factorial() = begin
local _cache = [1] #cache factorial(0)=1
function _factorial(n)
if n < length(_cache)
println("pull out from the cache factorial($n)=$(_cache[n+1])")
_cache[n+1]
else
fres = n * _factorial(n-1)
push!(_cache, fres)
println("put factorial($n)=$fres into the cache of the size=$(sizeof(_cache))") #a
fres
end
end
end
Now, just use it:
julia> myf = with_cached_factorial()
_factorial (generic function with 1 method)
julia> myf(3)
pull out from the cache factorial(0)=1
put factorial(1)=1 into the cache of the size=16
put factorial(2)=2 into the cache of the size=24
put factorial(3)=6 into the cache of the size=32
6
julia> myf(5)
pull out from the cache factorial(3)=6
put factorial(4)=24 into the cache of the size=40
put factorial(5)=120 into the cache of the size=48
120
julia> myf(10)
pull out from the cache factorial(5)=120
put factorial(6)=720 into the cache of the size=56
put factorial(7)=5040 into the cache of the size=64
put factorial(8)=40320 into the cache of the size=72
put factorial(9)=362880 into the cache of the size=80
put factorial(10)=3628800 into the cache of the size=88
3628800
let Aold = nothing
global foo
function foo(A::AbstractArray)
if A == Aold
println("Same as last array")
else
Aold = A
end
nothing
end
end
Results:
julia> A = rand(2,2)
2x2 Array{Float64,2}:
0.272936 0.153311
0.299549 0.703668
julia> B = rand(2,2)
2x2 Array{Float64,2}:
0.6762 0.377428
0.493344 0.240194
julia> foo(A)
julia> foo(A)
Same as last array
julia> foo(B)
julia> foo(A)
julia> foo(A)
Same as last array
julia> Aold
ERROR: UndefVarError: Aold not defined
Related
Suppose I have this function, implemented without StaticArrays
function example_svector_bad(G)
vector_list = [ randn(G) for q in 1:1000]
for i in size(vector_list)
for g in 1:G
vector_list[i][g] = vector_list[i][g] * g
end
end
return vector_list
end
I'm hoping to implement it using StaticArrays for speed gains. However, I don't know how to do it without losing the flexibility of specifying G. For example, I could do
function example_svector()
vector_list = [#SVector randn(3) for q in 1:1000]
for i in size(vector_list)
vector_list[i] = SVector(vector_list[i][1] * 1, vector_list[i][1] * 2,
vector_list[i][1] * 3)
end
return vector_list
end
if I knew that G = 3 and I had to write out SVector(vector_list[i][1] * 1, vector_list[i][1] * 2, vector_list[i][1] * 3).
Is there a way to implement this for any arbitrary number of G?
The size of a static vector or array must be known at the compile time.
At the compile time only types are known (rather than values).
Hence your function could look like this:
function myRandVec(::Val{G}) where G
SVector{G}(rand(G))
end
Note that G is passed as type rather than as value and hence can be used to create a static vector.
This function could be used as:
julia> myRandVec(Val{2}())
2-element SVector{2, Float64} with indices SOneTo(2):
0.7618992223709563
0.5979657793050613
Firstly, there is a mistake in how you are indexing vector_list, where you do
for i in size(vector_list)
Let's see what that does:
julia> x = 1:10;
julia> size(x)
(10,)
The size of x is its length in each dimension, for a vector that is just (10,) since it has only one dimension. Let's try iterating:
julia> for i in size(x)
println(i)
end
10
It just prints out the number 10.
You probably meant
for i in 1:length(vector_list)
but it's better to write
for i in eachindex(vector_list)
since it is more general and safer.
As for your actual question, you can use StaticArrays.SOneTo which provides a static version of [1,2,3]:
function example_svector()
vector_list = [#SVector randn(3) for q in 1:1000]
N = length(eltype(vector_list))
c = SOneTo(N)
for i in eachindex(vector_list)
vector_list[i] = vector_list[i] .* c
end
return vector_list
end
The code at the end of this post constructs a function which is bound to the variables of a given dictionary. Furthermore, the function is not bound to the actual name of the dictionary (as I use the Ref() statement).
An example:
julia> D = Dict(:x => 4, :y => 5)
julia> f= #mymacro4(x+2y, D)
julia> f()
14
julia> DD = D
julia> D = nothing
julia> f()
14
julia> DD[:x] = 12
julia> f()
22
Now I want to be able to construct exactly the same function when I only have access to the expression expr = :(x+2y).
How do I do this? I tried several things, but was not able to find a solution.
julia> f = #mymacro4(:(x+2y), D)
julia> f() ### the function evaluation should also yield 14. But it yields:
:(DR.x[:x] + 2 * DR.x[:y])
(I actually want to use it within another macro in which the dictionary is automatically created. I want to store this dictionary and the function within a struct, such that I'm able to call this function at a later point in time and manipulate the objects in the dictionary. If necessary, I may post the complete example and explain the complete problem.)
_freevars2(literal) = literal
function _freevars2(s::Symbol)
try
if typeof(eval(s)) <: Function
return s
else
return Meta.parse("DR.x[:$s]")
end
catch
return Meta.parse("DR.x[:$s]")
end
end
function _freevars2(expr::Expr)
for (it, s) in enumerate(expr.args)
expr.args[it] = _freevars2(s)
end
return expr
end
macro mymacro4(expr, D)
expr2 = _freevars2(expr)
quote
let DR = Ref($(esc(D)))
function mysym()
$expr2
end
end
end
end
I have issue after calling my macro:
#introspectable square(x) = x * x
Then when calling
square(3)
i should be able to get 9, cause the function call has been specialized to execute an attribute of the structure which is Julia code, however when I enter the macro, the code seems to be directly evaluated.
What i have tried:
struct IntrospectableFunction
name
parameters
native_function
end
(f::IntrospectableFunction)(x) = f.native_function(x)
macro introspectable(expr)
name = expr.args[1].args[1]
parameters = tuple(expr.args[1].args[2:end]...)
body = expr.args[2].args[2]
:( global $name = IntrospectableFunction( :( name ), $parameters, :( body ) ))
end
#introspectable square(x) = x * x
square(3)
The answer should be 9 , however i get "Object of type symbol are not callable ". However if i replace :( body ) with x -> x * x i get the desired result, my objective is generalizing the macro-call.
I usually find it easier to work with expressions in macros (it is not the shortest way to write things, but, from my experience, it is much easier to control what gets generated).
Therefore I would rewrite your code as:
macro introspectable(expr)
name = expr.args[1].args[1]
parameters = expr.args[1].args[2:end]
anon = Expr(Symbol("->"), Expr(:tuple, parameters...), expr.args[2].args[2])
constr = Expr(:call, :IntrospectableFunction, QuoteNode(name), Tuple(parameters), anon)
esc(Expr(:global, Expr(Symbol("="), name, constr)))
end
Now, as you said you wanted generality I would define your functor like this:
(f::IntrospectableFunction)(x...) = f.native_function(x...)
(in this way you allow multiple positional arguments to be passed).
Now let us test our definitions:
julia> #introspectable square(x) = x * x
IntrospectableFunction(:square, (:x,), getfield(Main, Symbol("##3#4"))())
julia> square(3)
9
julia> #macroexpand #introspectable square(x) = x * x
:(global square = IntrospectableFunction(:square, (:x,), ((x,)->x * x)))
julia> #introspectable toarray(x,y) = [x,y]
IntrospectableFunction(:toarray, (:x, :y), getfield(Main, Symbol("##5#6"))())
julia> toarray("a", 10)
2-element Array{Any,1}:
"a"
10
julia> #macroexpand #introspectable toarray(x,y) = [x,y]
:(global toarray = IntrospectableFunction(:toarray, (:x, :y), ((x, y)->[x, y])))
julia> function localscopetest()
#introspectable globalfun(x...) = x
end
localscopetest (generic function with 1 method)
julia> localscopetest()
IntrospectableFunction(:globalfun, (:(x...),), getfield(Main, Symbol("##9#10"))())
julia> globalfun(1,2,3,4,5)
(1, 2, 3, 4, 5)
julia> function f()
v = 100
#introspectable localbinding(x) = (v, x)
end
f (generic function with 1 method)
julia> f()
IntrospectableFunction(:localbinding, (:x,), getfield(Main, Symbol("##11#12")){Int64}(100))
julia> localbinding("x")
(100, "x")
(note that it is useful to use #macroexpand to make sure our macro works as expected)
EDIT - how to handle a minimal multiple dispatch
I am writing a non-macro example because it is related to the data structure:
Use e.g. such a definition:
struct IntrospectableFunction
name::Symbol
method_array::Vector{Pair{Type{<:Tuple}, Function}}
end
function (f::IntrospectableFunction)(x...)
for m in f.method_array
if typeof(x) <: first(m)
return last(m)(x...)
end
end
error("signature not found")
end
and now you can write:
julia> square = IntrospectableFunction(:square, [Tuple{Any}=>x->x*x,Tuple{Any,Any}=>(x,y)->x*y])
IntrospectableFunction(:square, Pair{DataType,Function}[Tuple{Any}=>##9#11(), Tuple{Any,Any}=>##10#12()])
julia> square(3)
9
julia> square(2,3)
6
Keep in mind that the approach I present is not perfect and universal - it just serves to give a very simple example how you could do it.
I have a string, e.g. z[2] and I want to eval it in a context, e.g. Dict(:z => 1:10)
What's the best way to do it?
I can make it sort of work, but it is very slow.
function replace_expr(expr, d::Dict)
return expr
end
function replace_expr(s::Symbol, d::Dict)
get(d, s, s)
end
function replace_expr(expr::Expr, d::Dict)
return Expr(replace_expr(expr.head, d),
[replace_expr(e, d) for e in expr.args]...)
end
function eval_with(context::Dict{Symbol, Any}, expr_string::AbstractString)
# E.g. :abc => :(s[:abc])
d = Dict(k => :(s[$(Meta.quot(k))]) for k in keys(context))
ex = parse("s -> $expr_string")
ex = replace_expr(ex, d)
return eval(ex)(context)
end
The following is the test
function make_context()
x = 1
y = "foo"
z = 2:5
Dict(
:x => x,
:y => y,
:z => z
)
end
const context = make_context()
#test eval_with(context, "x + 3") == 4
#test eval_with(context, "string(1, y, 1)") == "1foo1"
#test eval_with(context, "z[2]") == 3
#time eval_with(context, "z[2]")
# 0.004739 seconds (767 allocations: 40.728 KB)
This seems like a place where you can lean upon more of Julia's built-in expression evaluation machinery. eval takes an optional argument: the module in which the evaluation is to occur.
You can create new modules programmatically:
julia> M = Module()
anonymous
And you can assign values from a dictionary into that module with eval:
julia> context = Dict(
:x => 1,
:y => "foo",
:z => 2:5
);
julia> for (k,v) in context
eval(M, :($k = $v))
end
julia> M.x
1
julia> M.y
"foo"
And now, of course, you can evaluate your custom string within the context of your module.
julia> eval(M, parse("x+3"))
4
julia> eval(M, parse("string(1, y, 1)"))
"1foo1"
Dynamic evaluation like this is not going to be a place where Julia shines. I think this will be about as good as it gets:
julia> #time eval(M, parse("z[2]"))
0.000284 seconds (13 allocations: 672 bytes)
3
Note that this has slightly different semantics from the code you wrote above; the variables within your context only got populated at the beginning… and might be changed by a new evaluation.
And the usual caveats about using eval apply. There are often other, better ways of structuring your program that will be more performant, more understandable, and more maintainable.
If you know the values in advance, you can get around using eval via metaprogramming. A macro for this is provided by Parameters.jl:
d = Dict{Symbol,Any}(:a=>5.0,:b=>2,:c=>"Hi!")
#unpack a, c = d
a == 5.0 #true
c == "Hi!" #true
How could I create variables from keys and values of a dictionary? Is the following extract (like PHP) possible?
julia> d = Dict(
"key1" =>111,
"key2" =>222,
"key3" =>333
);
julia> extract(d)
julia> key1, key2, key3
(111,222,333)
k = collect(keys(d))
v = collect(values(d))
Both keys and values return iterators.
collect then produces an array.
But note that you often do not need to do this and can just iterate through the dictionary using
for (k, v) in d
It is possible to introduce new variables into the global scope with eval:
julia> x = 1
1
julia> function testeval()
eval(:(x = 5))
return x
end
testeval (generic function with 1 method)
julia> testeval()
5
julia> x # the global x has changed!
5
An extract function could look like this:
julia> function extract(d)
expr = quote end
for (k, v) in d
push!(expr.args, :($(Symbol(k)) = $v))
end
eval(expr)
return
end
julia> extract(d)
julia> key1, key2, key3
(111,222,333)
Note that every module has its own global scope. Therefore, this will introduce the variables into the scope of the module where the extract function is defined, i.e., into the Main module if defined at the REPL as in the example.
You should be very careful when using eval and first consider other approaches, e.g, the ones mentioned by David P. Sanders and Dan Getz.