Related
Here is the great example from StatWithJuliaBook (please find the following)
It demos how to smooth a plot of stary sky stars.png
My question is about argmax().I. According to the author, "Note the use of the trailing “.I” at the end of each argmax, which extracts the values of the co-ordinates in column-major."
What does it mean? Is there other parameter? I can't find any description in the document.
According to author, it seems to be the position of column-wise maxmum value, yet when I tried argmax(gImg, dims=2), the result is different.
#julia> yOriginal, xOriginal = argmax(gImg).I
#(192, 168)
#julia> yy, xx = argmax(gImg, dims = 2)
#400×1 Matrix{CartesianIndex{2}}:
# CartesianIndex(1, 187)
# CartesianIndex(2, 229)
⋮
# CartesianIndex(399, 207)
# CartesianIndex(400, 285)
#julia> yy, xx
#(CartesianIndex(1, 187), CartesianIndex(2, 229))
Please advise.
using Plots, Images; pyplot()
img = load("stars.png")
gImg = red.(img)*0.299 + green.(img)*0.587 + blue.(img)*0.114
rows, cols = size(img)
println("Highest intensity pixel: ", findmax(gImg))
function boxBlur(image,x,y,d)
if x<=d || y<=d || x>=cols-d || y>=rows-d
return image[x,y]
else
total = 0.0
for xi = x-d:x+d
for yi = y-d:y+d
total += image[xi,yi]
end
end
return total/((2d+1)^2)
end
end
blurImg = [boxBlur(gImg,x,y,5) for x in 1:cols, y in 1:rows]
yOriginal, xOriginal = argmax(gImg).I
yBoxBlur, xBoxBlur = argmax(blurImg).I
p1 = heatmap(gImg, c=:Greys, yflip=true)
p1 = scatter!((xOriginal, yOriginal), ms=60, ma=0, msw=4, msc=:red)
p2 = heatmap(blurImg, c=:Greys, yflip=true)
p2 = scatter!((xBoxBlur, yBoxBlur), ms=60, ma=0, msw=4, msc=:red)
plot(p1, p2, size=(800, 400), ratio=:equal, xlims=(0,cols), ylims=(0,rows),
colorbar_entry=false, border=:none, legend=:none)
I is a field in an object of type CartesianIndex which is returned by argmax when its argument has more than 1 dimension.
If in doubt always try using dump.
Please consider the code below:
julia> arr = rand(4,4)
4×4 Matrix{Float64}:
0.971271 0.0350186 0.20805 0.284678
0.348161 0.19649 0.30343 0.291894
0.385583 0.990593 0.216894 0.814146
0.283823 0.750008 0.266643 0.473104
julia> el = argmax(arr)
CartesianIndex(3, 2)
julia> dump(el)
CartesianIndex{2}
I: Tuple{Int64, Int64}
1: Int64 3
2: Int64 2
However, getting CartesianIndex object data via its internal structure is not very elegant. The nice Julian way to do it is to use the appropriate method:
julia> Tuple(el)
(3, 2)
Or just access the indices directly:
julia> el[1], el[2]
(3, 2)
I have a model with many parameters where I am passing them as a named tuple. Is there a way to promote the values into the variable scope in my function?
parameters = (
τ₁ = 0.035,
β₁ = 0.00509,
θ = 1,
τ₂ = 0.01,
β₂ = 0.02685,
...
)
And then used like so currently:
function model(init,params) # params would be the parameters above
foo = params.β₁ ^ params.θ
end
Is there a way (marco?) to get the parameters into my variable scope directly so that I can do this:
function model(init,params) # params would be the parameters above
#promote params # hypothetical macro to bring each named tuple field into scope
foo = β₁ ^ θ
end
The latter looks a lot nicer with some math-heavy code.
You can use #unpack from the UnPack.jl package1:
julia> nt = (a = 1, b = 2, c = 3);
julia> #unpack a, c = nt; # selectively unpack a and c
julia> a
1
julia> c
3
1 This was formerly part of the Parameters.jl package, which still exports #unpack and has other similar functionality you might find useful.
Edit: As noted in the comments, writing a general macro #unpack x is not possible since the fieldnames are runtime information. You could however define a macro specific to your own type/namedtuple that unpacks
julia> macro myunpack(x)
return esc(quote
a = $(x).a
b = $(x).b
c = $(x).c
nothing
end)
end;
julia> nt = (a = 1, b = 2, c = 3);
julia> #myunpack nt
julia> a, b, c
(1, 2, 3)
However, I think it is more clear to use the #unpack since this version "hides" assignments and it is not clear where the variables a, b and c comes from when reading the code.
I like the idea of
NamedTuple
a lot, as a middle ground between Tuple and full, user-defined composite types.
I know how to build a named tuple and access one of its fields
julia> nt = (a=1, b=2.0)
(a = 1, b = 2.0)
julia> nt.a
1
however, I don't know much more and don't even know whether it is possible to do more than that. I'm thinking about a lot of ways we can manipulate plain tuples (usually involving splatting), and wonder if some of those apply to named tuples as well. For example, how to:
dynamically build a NamedTuple from lists of fields and values
grow a NamedTuple , i.e add new field-value pairs to it
"update" (in an immutable sense) a field in an existing named tuple
The NamedTupleTools
package contains a lot of tools aiming at making the use of NamedTuples more
straightforward. But here are a few elementary operations that can be performed
on them "manually":
Creation
# regular syntax
julia> nt = (a=1, b=2.)
(a = 1, b = 2.0)
# empty named tuple (useful as a seed that will later grow)
julia> NamedTuple()
NamedTuple()
# only one entry => don't forget the comma
julia> (a=1,)
(a = 1,)
Growth and "modification"
It is possible to
merge two
named tuples to create a new one:
julia> merge(nt, (c=3, d=4.))
(a = 1, b = 2.0, c = 3, d = 4.0)
...or to re-use an existing NamedTuple by splatting it in the creation of a
new one:
julia> (; nt..., c=3, d=4.)
(a = 1, b = 2.0, c = 3, d = 4.0)
When the same field name appears multiple times, the last occurrence is
kept. This allows for a form of "copy with modification":
julia> nt
(a = 1, b = 2.0)
julia> merge(nt, (b=3,))
(a = 1, b = 3)
julia> (; nt..., b=3)
(a = 1, b = 3)
Dynamic manipulations
Using field=>value pairs in the various techniques presented above allows for
more dynamic manipulations:
julia> field = :c;
julia> merge(nt, [field=>1])
(a = 1, b = 2.0, c = 1)
julia> (; nt..., field=>1)
(a = 1, b = 2.0, c = 1)
The same technique can be used to build NamedTuples from existing dynamic data structures
julia> dic = Dict(:a=>1, :b=>2);
julia> (; dic...)
(a = 1, b = 2)
julia> arr = [:a=>1, :b=>2];
julia> (; arr...)
(a = 1, b = 2)
Iteration
Iterating on a NamedTuple iterates on its values:
julia> for val in nt
println(val)
end
1
2.0
Like all key->value structures, the
keys function
can be used to iterate over the fields:
julia> for field in keys(nt)
val = nt[field]
println("$field => $val")
end
a => 1
b => 2.0
In Python, one can use the * operator in the unpacking of an iterable.
In [1]: head, *tail = [1, 2, 3, 4, 5]
In [2]: head
Out[2]: 1
In [3]: tail
Out[3]: [2, 3, 4, 5]
I would like to produce the same behavior in Julia. I figured that the equivalent ... operator would work, but it seems to just produce an error in this context.
julia> head, tail... = [1, 2, 3, 4, 5]
ERROR: syntax: invalid assignment location "tail..."
I was able to produce the results I want using the following, but this is an ugly solution.
julia> head, tail = A[1], A[2:end]
(1,[2,3,4,5])
Can I unpack the array such that tail would contain the rest of the items after head using the splat (...) operator? If not, what is the cleanest alternative?
Edit: This feature has been proposed in #2626. It looks like it will be part of the 1.0 release.
As of Julia 1.6
It is now possible to use ... on the left-hand side of destructured assignments for taking any number of items from the front of an iterable collection, while also collecting the rest.
Example of assigning the first two items while slurping the rest:
julia> a, b, c... = [4, 8, 15, 16, 23, 42]
# 6-element Vector{Int64}:
# 4
# 8
# 15
# 16
# 23
# 42
julia> a
# 4
julia> b
# 8
julia> c
# 4-element Vector{Int64}:
# 15
# 16
# 23
# 42
This syntax is implemented using Base.rest, which can be overloaded to customize its behavior.
Example of overloading Base.rest(s::Union{String, SubString{String}}, i::Int) to slurp a Vector{Char} instead of the default SubString:
julia> a, b... = "hello"
julia> b
# "ello"
julia> Base.rest(s::Union{String, SubString{String}}, i=1) = collect(SubString(s, i))
julia> a, b... = "hello"
julia> b
# 4-element Vector{Char}:
# 'e': ASCII/Unicode U+0065 (category Ll: Letter, lowercase)
# 'l': ASCII/Unicode U+006C (category Ll: Letter, lowercase)
# 'l': ASCII/Unicode U+006C (category Ll: Letter, lowercase)
# 'o': ASCII/Unicode U+006F (category Ll: Letter, lowercase)
That does indeed sound like a job for a macro:
function unpack(lhs, rhs)
len = length(lhs.args)
if len == 1
# just remove the splatting
l, is_splat = remove_splat(lhs.args[1])
return :($l = $(esc(rhs)))
else
new_lhs = :()
new_rhs = quote
tmp = $(esc(rhs))
$(Expr(:tuple))
end
splatted = false
for (i, e) in enumerate(lhs.args)
l, is_splat = remove_splat(e)
if is_splat
splatted && error("Only one splatting operation allowed on lhs")
splatted = true
r = :(tmp[$i:end-$(len-i)])
elseif splatted
r = :(tmp[end-$(len-i)])
else
r = :(tmp[$i])
end
push!(new_lhs.args, l)
push!(new_rhs.args[4].args, r)
end
return :($new_lhs = $new_rhs)
end
end
remove_splat(e::Symbol) = esc(e), false
function remove_splat(e::Expr)
if e.head == :(...)
return esc(e.args[1]), true
else
return esc(e), false
end
end
macro unpack(expr)
if Meta.isexpr(expr, :(=))
if Meta.isexpr(expr.args[1], :tuple)
return unpack(expr.args[1], expr.args[2])
else
return unpack(:(($(expr.args[1]),)), expr.args[2])
end
else
error("Cannot parse expression")
end
end
It is not very well tested, but basic things work:
julia> #unpack head, tail... = [1,2,3,4]
(1,[2,3,4])
julia> #unpack head, middle..., tail = [1,2,3,4,5]
(1,[2,3,4],5)
A few Julia gotchas:
x,y = [1,2,3] #=> x = 1, y = 2
a = rand(3)
a[1:3], y = [1,2,3] #=> a = [1.0,1.0,1.0], y = 2
The macro follows this behavior
#unpack a[1:3], y... = [1,2,3]
#=> a=[1.0,1.0,1.0], y=[2,3]
I have a Array{Tuple{A, B}}, and I want to unzip / transpose it into Tuple{Array{A}, Array{B}}.
typealias MyIndexType Tuple{Bool, Int}
function test(x::MyIndexType)
# prepare for test data set.
myArray = Array{Tuple{MyIndexType, Float64}}(0)
push!(myArray, (x,1))
push!(myArray, (x,1))
push!(myArray, (x,1))
# transform
a, b = (zip(myArray...)...)
[a...]
end
test((true, 1))
>>>
3-element Array{Tuple{Bool,Int64},1}:
(true,1)
(true,1)
(true,1)
However, using #code_warntype, JIT cannot infer the type of a, b ahead of time.
Variables:
x::Tuple{Bool,Int64}
myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1}
a::ANY
b::ANY
#s41::Int64
Body:
begin # In[47], line 6:
myArray = (top(ccall))(:jl_alloc_array_1d,(top(apply_type))(Base.Array,Tuple{Tuple{Bool,Int64},Float64},1)::Type{Array{Tuple{Tuple{Bool,Int64},Float64},1}},(top(svec))(Base.Any,Base.Int)::SimpleVector,Array{Tuple{Tuple{Bool,Int64},Float64},1},0,0,0)::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 7:
(Main.push!)(myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1},(top(tuple))(x::Tuple{Bool,Int64},1)::Tuple{Tuple{Bool,Int64},Int64})::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 8:
(Main.push!)(myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1},(top(tuple))(x::Tuple{Bool,Int64},1)::Tuple{Tuple{Bool,Int64},Int64})::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 9:
(Main.push!)(myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1},(top(tuple))(x::Tuple{Bool,Int64},1)::Tuple{Tuple{Bool,Int64},Int64})::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 10:
GenSym(0) = (top(_apply))((top(getfield))(Main,:call)::F,top(tuple),(top(_apply))((top(getfield))(Main,:call)::F,Main.zip,myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1})::UNION{BASE.ZIP2{TUPLE{TUPLE{BOOL,INT64},FLOAT64},TUPLE{TUPLE{BOOL,INT64},FLOAT64}},TUPLE{TUPLE{BOOL,INT64},FLOAT64},ZIP{I,Z<:BASE.ABSTRACTZIPITERATOR}})::TUPLE
#s41 = 1
GenSym(4) = (Base.getfield)(GenSym(0),1)::ANY
GenSym(5) = (Base.box)(Base.Int,(Base.add_int)(1,1)::ANY)::Int64
a = GenSym(4)
#s41 = GenSym(5)
GenSym(6) = (Base.getfield)(GenSym(0),2)::ANY
GenSym(7) = (Base.box)(Base.Int,(Base.add_int)(2,1)::ANY)::Int64
b = GenSym(6)
#s41 = GenSym(7) # In[47], line 11:
return (top(_apply))((top(getfield))(Main,:call)::F,top(vect),a)::ANY
end::ANY
Is there a way to make zip aware of the resulting types?
Update
Actually there are 2 problems.
It thinks a is of type a::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}, but it is actually of type a::TUPLE{FLOAT64,FLOAT64}
function test{T}(x::T)
A = Tuple{T, Int}[]
for i in 1:3
push!(A, (x, 1))
end
d = zip(A[1], A[2])
a, b = d
a
end
#code_warntype test(3.0)
Variables:
x::Float64
A::Array{Tuple{Float64,Int64},1}
d::Base.Zip2{Tuple{Float64,Int64},Tuple{Float64,Int64}}
a::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
b::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
#s40::Tuple{Int64,Int64}
#s41::Int64
i::Int64
For zip taking more than 2 arguments, note d has a nested zip2 type, which I feel may bring burden to type inference.
function test{T}(x::T)
A = Tuple{T, Int}[]
for i in 1:3
push!(A, (x, 1))
end
d = zip(A[1], A[2], A[3])
a, b = d
a
end
#code_warntype test(3.0)
Variables:
x::Float64
A::Array{Tuple{Float64,Int64},1}
d::Zip{Tuple{Float64,Int64},Base.Zip2{Tuple{Float64,Int64},Tuple{Float64,Int64}}}
a::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
b::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
#s40::Tuple{Int64,Tuple{Int64,Int64}}
#s41::Int64
i::Int64
##c#7879::Tuple{Tuple{Float64,Int64}}
Why do I care about the type?
It takes 10+ seconds to compile a, b = zip(A...) of the following example, and the speed seems to be related to the length of A. (Julia 0.4)
const A = Tuple{Int, Int}[]
for i = 1:200
push!(A, (1, 1))
end
a, b = zip(A...)
a
I opened a bug report here https://github.com/JuliaLang/julia/issues/13722
I believe that it is not a problem that #code_warntype reports that it fails to infer the correct types if it gets it right in the end.
I was nonetheless wondering if this is due to the complexity of your type. But it isn't, as the code below shows (with a simpler type).
Note that you can also simplify your zip expression; and you probably don't need to convert a to an array.
Code:
function test{T}(x::T)
A = Tuple{T, Int}[]
for i in 1:3
push!(A, (x, 1))
end
a, b = zip(A...)
a, b
end
julia> test(3) # now returns a and b
((3,3,3),(2,2,2))
julia> #code_warntype test(3)
Variables:
x::Int64
A::Array{Tuple{Int64,Int64},1}
a::ANY
b::ANY
#s40::ANY
#s41::Int64
i::Int64