Extracting parameter types in Julia - julia

Suppose I write a function in Julia that takes a Dict{K,V} as an argument, then creates arrays of type Array{K,1} and Array{V,1}. How can I extract the types K and V from the Dict object so that I can use them to create the arrays?

Sven and John's answers are both quite right. If you don't want to introduce method type parameters the way John's code does, you can use the eltype function:
julia> d = ["foo"=>1, "bar"=>2]
["foo"=>1,"bar"=>2]
julia> eltype(d)
(ASCIIString,Int64)
julia> eltype(d)[1]
ASCIIString (constructor with 1 method)
julia> eltype(d)[2]
Int64
julia> eltype(keys(d))
ASCIIString (constructor with 1 method)
julia> eltype(values(d))
Int64
As you can see, there are a few ways to skin this cat, but I think that eltype(keys(d)) and eltype(values(d)) are by far the clearest and since the keys and values functions just return immutable view objects, the compiler is clever enough that this doesn't actually create any objects.

If you're writing a function that will do this for you, you can make the types a parameter of the function, which may save you some run-time lookups:
julia> function foo{K, V}(d::Dict{K, V}, n::Integer = 0)
keyarray = Array(K, n)
valarray = Array(V, n)
# MAGIC HAPPENS
return keyarray, valarray
end
foo (generic function with 2 methods)
julia> x, y = foo(["a" => 2, "b" => 3])
([],[])
julia> typeof(x)
Array{ASCIIString,1}
julia> typeof(y)
Array{Int64,1}

You can use keys and values in combination with typeof:
# an example Dict{K, V}
d = Dict{Int64, ASCIIString}()
# K
typeof(keys(d))
Array{Int64,1}
# V
typeof(values(d))
Array{ASCIIString,1}

If you are just interested in the types, you can use eltype(d), or define even more specific functions
keytype{K}(d::Dict{K}) = K
valuetype{K,V}(d::Dict{K,V}) = V
and find the type immediately through
keytype(d)
valuetype(d)
As far as I understand, this should be pretty efficient because the compiler can deduce most of this at compile time.

For iterables, the eltype approach that is already given in other answers works well enough. E.g.
julia> d = Dict(:a => 1, :b => 2)
Dict{Symbol, Int64} with 2 entries:
:a => 1
:b => 2
julia> K, V = eltype(keys(d)), eltype(values(d))
(Symbol, Int64)
julia> K
Symbol
julia> V
Int64
This approach is generalisable to all iterables. However I would like to offer another approach which will generalise to other kinds of object (and still doesn't use the method type signature approach):
julia> d = Dict(:a => 1, :b => 2)
Dict{Symbol, Int64} with 2 entries:
:a => 1
:b => 2
julia> typeof(d)
Dict{Symbol, Int64}
julia> K, V = typeof(d).parameters
svec(Symbol, Int64)
julia> K
Symbol
julia> V
Int64
You can see that the Dict{Symbol, Int64} type has a field parameters which you can unpack to obtain the type parameters. This works on structs too:
julia> struct MyStruct{S, T}
a::S
b::T
end
julia> x = MyStruct(1, 1.0)
MyStruct{Int64, Float64}(1, 1.0)
julia> S, T = typeof(x).parameters
svec(Int64, Float64)
julia> S
Int64
julia> T
Float64

Related

how to perform reduce(hcat, <Vector{Vector}>) by TensorCast.jl?

Is it possible to do the following via TensorCast.jl as well?
#This is an object of type Vector{Vector{Float64}}
julia> a = [rand(5) for i=1:5];
julia> reduce(hcat, a)
5×5 Matrix{Float64}:
0.0678613 0.266194 0.183221 0.485462 0.873282
0.735101 0.925276 0.956102 0.333281 0.885147
0.323555 0.74204 0.135538 0.26123 0.261068
0.81847 0.917006 0.0118195 0.295497 0.712431
0.420139 0.0197552 0.0617039 0.157477 0.500931
I tried something super silly, I guess (😰):
julia> #reduce b := hcat(a)
ERROR: MethodError: no method matching guesstarget(::Nothing, ::Vector{Any}, ::Vector{Any})
Closest candidates are:
guesstarget(::Expr, ::Any, ::Any) at C:\Users\Shayan\.julia\packages\TensorCast\mQB8h\src\macro.jl:1330
Just do:
#cast b[j,i] := a[i][j]
Example:
julia> a = [rand(3) for _ in 1:2]
2-element Vector{Vector{Float64}}:
[0.20012490537057803, 0.0365551498875093, 0.02494737196890595]
[0.6563493855249903, 0.181706254856571, 0.29210798163726615]
julia> #cast b[j,i] := a[i][j]
3×2 lazystack(::Vector{Vector{Float64}}) with eltype Float64:
0.200125 0.656349
0.0365551 0.181706
0.0249474 0.292108
julia> (#cast b[j,i] := a[i][j]) == reduce(hcat, a)
true
Exaplanation
The variables i and j are used by #cast to define the matrix layout - we have a Vector of Vectors and hence on the right side you can see a[i][j], := defines a new Matrix and b[j,i] says where the corresponding i and j elements should go. Note that you do not declare i and j variables - this is handled by the macro.
Finally, note that if you want the result to be materialized to an actual matrix (rather than a lazystack) you can next do collect(b)

Filtering a dictionary in julia

I want to filter a dictionary using filter() function but I am having trouble with it. What I wish to accomplish is, to return the key for some condition of the value. However I am getting a method error
using Agents: AbstractAgent
# Define types
mutable struct Casualty <: AbstractAgent
id::Int
ts::Int
rescued::Bool
function Casualty(id,ts; rescued = false)
new(id,ts,rescued)
end
end
mutable struct Rescuer <: AbstractAgent
id::Int
actions::Int
dist::Float64
function Rescuer(id; action = rand(1:3) , dist = rand(1)[1])
new(id,action,dist)
end
end
cas1 = Casualty(1,2)
cas2 = Casualty(2,3)
resc1 = Rescuer(3)
agents = Dict(1=> cas1, 2 => cas2, 3 => resc1)
Now to filter
filter((k,v) -> v isa Casualty, agents)
# ERROR: MethodError: no method matching (::var"#22#23")(::Pair{Int64, AbstractAgent})
# what I truly wish to achieve is return the key for some condition of the value
filter((k,v) -> k ? v isa Casualty : "pass", agents)
# ofcourse I am not sure how to "pass" using this format
Any idea how I can achieve this. Thanks
For dictionaries filter gets a key-value pair, so do either (destructuring Pair):
julia> dict = Dict(1=>"a", 2=>"b", 3=>"c")
Dict{Int64, String} with 3 entries:
2 => "b"
3 => "c"
1 => "a"
julia> filter(((k,v),) -> k == 1 || v == "c", dict)
Dict{Int64, String} with 2 entries:
3 => "c"
1 => "a"
or for example (getting Pair as a whole):
julia> filter(p -> first(p) == 1 || last(p) == "c", dict)
Dict{Int64, String} with 2 entries:
3 => "c"
1 => "a"
julia> filter(p -> p[1] == 1 || p[2] == "c", dict)
Dict{Int64, String} with 2 entries:
3 => "c"
1 => "a"
EDIT
Explanation why additional parentheses are needed:
julia> f = (x, y) -> (x, y)
#1 (generic function with 1 method)
julia> g = ((x, y),) -> (x, y)
#3 (generic function with 1 method)
julia> methods(f)
# 1 method for anonymous function "#1":
[1] (::var"#1#2")(x, y) in Main at REPL[1]:1
julia> methods(g)
# 1 method for anonymous function "#3":
[1] (::var"#3#4")(::Any) in Main at REPL[2]:1
julia> f(1, 2)
(1, 2)
julia> f((1, 2))
ERROR: MethodError: no method matching (::var"#1#2")(::Tuple{Int64, Int64})
Closest candidates are:
(::var"#1#2")(::Any, ::Any) at REPL[1]:1
julia> g(1, 2)
ERROR: MethodError: no method matching (::var"#3#4")(::Int64, ::Int64)
Closest candidates are:
(::var"#3#4")(::Any) at REPL[2]:1
julia> g((1, 2))
(1, 2)
As you can see f takes 2 positional argument, while g takes one positional argument that gets destructured (i.e. the assumption is that argument passed to g is iterable and has at least 2 elements).
See also https://docs.julialang.org/en/v1/manual/functions/#Argument-destructuring.
Now comes the tricky part:
julia> h1((x, y)) = (x, y)
h1 (generic function with 1 method)
julia> methods(h1)
# 1 method for generic function "h1":
[1] h1(::Any) in Main at REPL[1]:1
julia> h2 = ((x, y)) -> (x, y)
#1 (generic function with 1 method)
julia> methods(h2)
# 1 method for anonymous function "#1":
[1] (::var"#1#2")(x, y) in Main at REPL[3]:1
In this example h1 is a named function. In this case it is enough to just wrap arguments in extra parentheses to get destructuring behavior. For anonymous functions, because of how Julia parser works an extra , is needed - if you omit it the extra parentheses are ignored.
Now let us check filter docstring:
filter(f, d::AbstractDict)
Return a copy of d, removing elements for which f is false.
The function f is passed key=>value pairs.
As you can see from this docstring f is passed a single argument that is Pair. That is why you need to use either destructuring or define a single argument function and extract its elements inside the function.
The right syntax is:
filter(((k,v),) -> v isa Casualty, agents)
which prints
julia> filter(((k,v),) -> v isa Casualty, agents)
Dict{Int64, AbstractAgent} with 2 entries:
2 => Casualty(2, 3, false)
1 => Casualty(1, 2, false)
About the problem of only getting involved keys... I have no idea beside:
julia> filter(((k,v),) -> v isa Casualty, agents) |> keys
which prints
julia> filter(((k,v),) -> v isa Casualty, agents) |> keys
KeySet for a Dict{Int64, AbstractAgent} with 2 entries. Keys:
2
1

What is the difference between fields and properties in Julia?

Julia has the setter functions setproperty! and setfield! and the getter functions getproperty and getfield that operate on structs. What is the difference between properties and fields in Julia?
For example, the following seems to indicate that they do the same thing:
julia> mutable struct S
a
end
julia> s = S(2)
S(2)
julia> getfield(s, :a)
2
julia> getproperty(s, :a)
2
julia> setfield!(s, :a, 3)
3
julia> s
S(3)
julia> setproperty!(s, :a, 4)
4
julia> s
S(4)
fields are simply the "components" of a struct. The struct
struct A
b
c::Int
end
has the fields b and c. A call to getfield returns the object that is bound to the field:
julia> a = A("foo", 3)
A("foo", 3)
julia> getfield(a, :b)
"foo"
In early versions of Julia, the syntax a.b used to "lower", i.e. be the same as, writing getfield(a, :b). What has changed now is that a.b lowers to getproperty(a, :b) with the default fallback
getproperty(a::Type, v::Symbol) = getfield(a, v)
So by default, nothing has changed. However, authors of structs can overload getproperty (it is not possible to overload getfield) to provide extra functionality to the dot-syntax:
julia> function Base.getproperty(a::A, v::Symbol)
if v == :c
return getfield(a, :c) * 2
elseif v == :q
return "q"
else
return getfield(a, v)
end
end
julia> a.q
"q"
julia> getfield(a, :q)
ERROR: type A has no field q
julia> a.c
6
julia> getfield(a, :c)
3
julia> a.b
"foo"
So we can add extra functionality to the dot syntax (dynamically if we want). As a concrete example where this is useful is for the package PyCall.jl where you used to have to write pyobject[:field] while it is possible now to implement it such that you can write pyobject.field.
The difference between setfield! and setproperty! is analogous to the difference between getfield and getproperty, explained above.
In addition, it is possible to hook into the function Base.propertynames to provide tab completion of properties in the REPL. By default, only the field names will be shown:
julia> a.<TAB><TAB>
b c
But by overloading propertynames we can make it also show the extra property q:
julia> Base.propertynames(::A) = (:b, :c, :q)
julia> a.<TAB><TAB>
b c q

Vectorized indexing for a dictionary

I would like to find a succinct syntax in Julia for indexing a dictionary in a vectorized manner. In R, I would do the following:
dict <- c("a" = 1, "b" = 2)
keys <- c("a", "a", "b", "b", "a")
dict[keys]
In Julia, if I have a dict and keys like this,
dict = Dict(:a => 1, :b => 2)
keys = [:a, :a, :b, :b, :a]
then I can achieve the desired result using a list comprehension:
julia> [dict[key] for key in keys]
5-element Array{Int64,1}:
1
1
2
2
1
Is there a more succinct vectorized syntax, similar to the R syntax?
getindex.(Ref(dict), keys)
You can wrap it in Ref so you don't need to [] it.
Here's a little macro (using the brilliant MacroTools package):
using MacroTools
macro vget(ex)
#capture(ex, dict_.[idxvec_])
:(map(i->$dict[i], $idxvec)) |> esc
end
Then you can:
d = Dict(:a => 1, :b => 2)
ks = [:a, :a, :b, :b, :a]
#vget d.[ks]
I probably wouldn't want to use that in production code, something like your list comprehension, or a simple map: map(i->d[i], ks), is more readable/explicit/standard, but it is fun :D
If you're going to be frequently using the dictionary as a lookup table, then it might be worth creating a closure to use as a lookup function:
make_lookup(dict) = key -> dict[key]
dict = Dict(:a => 1, :b => 2)
lookup = make_lookup(dict)
Then you can use lookup in a vectorized fashion:
julia> keys = [:a, :a, :b, :b, :a];
julia> lookup.(keys)
5-element Array{Int64,1}:
1
1
2
2
1
You can use the vectorized version of getindex:
julia> getindex.([dict], keys)
5-element Array{Int64,1}:
1
1
2
2
1
Note that dict is wrapped in an array so that getindex does not attempt to broadcast over the elements of the dictionary:
julia> getindex.(dict, keys)
ERROR: ArgumentError: broadcasting over dictionaries and `NamedTuple`s is reserved
Stacktrace:
[1] broadcastable(::Dict{Symbol,Int64}) at ./broadcast.jl:615
[2] broadcasted(::Function, ::Dict{Symbol,Int64}, ::Array{Symbol,1}) at ./broadcast.jl:1164
[3] top-level scope at none:0

Best way to eval in a given scope / context which is in the form of a Dict?

I have a string, e.g. z[2] and I want to eval it in a context, e.g. Dict(:z => 1:10)
What's the best way to do it?
I can make it sort of work, but it is very slow.
function replace_expr(expr, d::Dict)
return expr
end
function replace_expr(s::Symbol, d::Dict)
get(d, s, s)
end
function replace_expr(expr::Expr, d::Dict)
return Expr(replace_expr(expr.head, d),
[replace_expr(e, d) for e in expr.args]...)
end
function eval_with(context::Dict{Symbol, Any}, expr_string::AbstractString)
# E.g. :abc => :(s[:abc])
d = Dict(k => :(s[$(Meta.quot(k))]) for k in keys(context))
ex = parse("s -> $expr_string")
ex = replace_expr(ex, d)
return eval(ex)(context)
end
The following is the test
function make_context()
x = 1
y = "foo"
z = 2:5
Dict(
:x => x,
:y => y,
:z => z
)
end
const context = make_context()
#test eval_with(context, "x + 3") == 4
#test eval_with(context, "string(1, y, 1)") == "1foo1"
#test eval_with(context, "z[2]") == 3
#time eval_with(context, "z[2]")
# 0.004739 seconds (767 allocations: 40.728 KB)
This seems like a place where you can lean upon more of Julia's built-in expression evaluation machinery. eval takes an optional argument: the module in which the evaluation is to occur.
You can create new modules programmatically:
julia> M = Module()
anonymous
And you can assign values from a dictionary into that module with eval:
julia> context = Dict(
:x => 1,
:y => "foo",
:z => 2:5
);
julia> for (k,v) in context
eval(M, :($k = $v))
end
julia> M.x
1
julia> M.y
"foo"
And now, of course, you can evaluate your custom string within the context of your module.
julia> eval(M, parse("x+3"))
4
julia> eval(M, parse("string(1, y, 1)"))
"1foo1"
Dynamic evaluation like this is not going to be a place where Julia shines. I think this will be about as good as it gets:
julia> #time eval(M, parse("z[2]"))
0.000284 seconds (13 allocations: 672 bytes)
3
Note that this has slightly different semantics from the code you wrote above; the variables within your context only got populated at the beginning… and might be changed by a new evaluation.
And the usual caveats about using eval apply. There are often other, better ways of structuring your program that will be more performant, more understandable, and more maintainable.
If you know the values in advance, you can get around using eval via metaprogramming. A macro for this is provided by Parameters.jl:
d = Dict{Symbol,Any}(:a=>5.0,:b=>2,:c=>"Hi!")
#unpack a, c = d
a == 5.0 #true
c == "Hi!" #true

Resources