In Python, one can use the * operator in the unpacking of an iterable.
In [1]: head, *tail = [1, 2, 3, 4, 5]
In [2]: head
Out[2]: 1
In [3]: tail
Out[3]: [2, 3, 4, 5]
I would like to produce the same behavior in Julia. I figured that the equivalent ... operator would work, but it seems to just produce an error in this context.
julia> head, tail... = [1, 2, 3, 4, 5]
ERROR: syntax: invalid assignment location "tail..."
I was able to produce the results I want using the following, but this is an ugly solution.
julia> head, tail = A[1], A[2:end]
(1,[2,3,4,5])
Can I unpack the array such that tail would contain the rest of the items after head using the splat (...) operator? If not, what is the cleanest alternative?
Edit: This feature has been proposed in #2626. It looks like it will be part of the 1.0 release.
As of Julia 1.6
It is now possible to use ... on the left-hand side of destructured assignments for taking any number of items from the front of an iterable collection, while also collecting the rest.
Example of assigning the first two items while slurping the rest:
julia> a, b, c... = [4, 8, 15, 16, 23, 42]
# 6-element Vector{Int64}:
# 4
# 8
# 15
# 16
# 23
# 42
julia> a
# 4
julia> b
# 8
julia> c
# 4-element Vector{Int64}:
# 15
# 16
# 23
# 42
This syntax is implemented using Base.rest, which can be overloaded to customize its behavior.
Example of overloading Base.rest(s::Union{String, SubString{String}}, i::Int) to slurp a Vector{Char} instead of the default SubString:
julia> a, b... = "hello"
julia> b
# "ello"
julia> Base.rest(s::Union{String, SubString{String}}, i=1) = collect(SubString(s, i))
julia> a, b... = "hello"
julia> b
# 4-element Vector{Char}:
# 'e': ASCII/Unicode U+0065 (category Ll: Letter, lowercase)
# 'l': ASCII/Unicode U+006C (category Ll: Letter, lowercase)
# 'l': ASCII/Unicode U+006C (category Ll: Letter, lowercase)
# 'o': ASCII/Unicode U+006F (category Ll: Letter, lowercase)
That does indeed sound like a job for a macro:
function unpack(lhs, rhs)
len = length(lhs.args)
if len == 1
# just remove the splatting
l, is_splat = remove_splat(lhs.args[1])
return :($l = $(esc(rhs)))
else
new_lhs = :()
new_rhs = quote
tmp = $(esc(rhs))
$(Expr(:tuple))
end
splatted = false
for (i, e) in enumerate(lhs.args)
l, is_splat = remove_splat(e)
if is_splat
splatted && error("Only one splatting operation allowed on lhs")
splatted = true
r = :(tmp[$i:end-$(len-i)])
elseif splatted
r = :(tmp[end-$(len-i)])
else
r = :(tmp[$i])
end
push!(new_lhs.args, l)
push!(new_rhs.args[4].args, r)
end
return :($new_lhs = $new_rhs)
end
end
remove_splat(e::Symbol) = esc(e), false
function remove_splat(e::Expr)
if e.head == :(...)
return esc(e.args[1]), true
else
return esc(e), false
end
end
macro unpack(expr)
if Meta.isexpr(expr, :(=))
if Meta.isexpr(expr.args[1], :tuple)
return unpack(expr.args[1], expr.args[2])
else
return unpack(:(($(expr.args[1]),)), expr.args[2])
end
else
error("Cannot parse expression")
end
end
It is not very well tested, but basic things work:
julia> #unpack head, tail... = [1,2,3,4]
(1,[2,3,4])
julia> #unpack head, middle..., tail = [1,2,3,4,5]
(1,[2,3,4],5)
A few Julia gotchas:
x,y = [1,2,3] #=> x = 1, y = 2
a = rand(3)
a[1:3], y = [1,2,3] #=> a = [1.0,1.0,1.0], y = 2
The macro follows this behavior
#unpack a[1:3], y... = [1,2,3]
#=> a=[1.0,1.0,1.0], y=[2,3]
Related
I want to create an empty lsit and gardually fill that out with tuples. I've tried the following and each returns an error. My question is: how to append or add and element to an empty array?
My try:
A = []
A.append((2,5)) # return Error type Array has no field append
append(A, (2,5)) # ERROR: UndefVarError: append not defined
B = Vector{Tuple{String, String}}
# same error occues
You do not actually want to append, you want to push elements into your vector. To do that use the function push! (the trailing ! indicates that the function modifies one of its input arguments. It's a naming convention only, the ! doesn't do anything).
I would also recommend creating a typed vector instead of A = [], which is a Vector{Any} with poor performance.
julia> A = Tuple{Int, Int}[]
Tuple{Int64, Int64}[]
julia> push!(A, (2,3))
1-element Vector{Tuple{Int64, Int64}}:
(2, 3)
julia> push!(A, (11,3))
2-element Vector{Tuple{Int64, Int64}}:
(2, 3)
(11, 3)
For the vector of string tuples, do this:
julia> B = Tuple{String, String}[]
Tuple{String, String}[]
julia> push!(B, ("hi", "bye"))
1-element Vector{Tuple{String, String}}:
("hi", "bye")
This line in your code is wrong, btw:
B = Vector{Tuple{String, String}}
It does not create a vector, but a type variable. To create an instance you can write e.g. one of these:
B = Tuple{String, String}[]
B = Vector{Tuple{String,String}}() # <- parens necessary to construct an instance
It can also be convenient to use the NTuple notation:
julia> NTuple{2, String} === Tuple{String, String}
true
julia> NTuple{3, String} === Tuple{String, String, String}
true
I want to find the length, nc, of this "vector of functions". I should be 2.
comp(x) = [([x[5], x[6], x[7], x[8],x[9], x[10]], tmp(x)) ; ([x[1],x[2]], [x[3],x[4]])];
nc = ....
I tried with length(comp) and length(comp(x)) but it doesn't work. I get "x not defined" and "no method matching length(::typeof(comp))", respectively.
Pulling together some of the comments to hopefully make things clearer:
What you have written is essentially
function comp(x)
a = [x[5], x[6], x[7], x[8],x[9], x[10]]
b = [x[1],x[2]]
c = [x[3],x[4]]
return [(a, tmp(x)); (b, c)]
end
that is, you have defined a function comp which takes one argument x and then returns a 2-element vector of 2-element tuples, with the first tuple holding values 5 to 10 of x and the result of tmp(x) (this function is not defined in your code so we don't know what it returns), and the second tuple holding the first and second, and third and fourth elements of x, respectively.
To illustrate, assume tmp(x) just sums up the elements of x, then we can pass some array (in the below example a range) of numbers to comp and see it in action:
julia> tmp(x) = sum(x)
tmp (generic function with 1 method)
julia> comp(1:20)
2-element Vector{Tuple{Vector{Int64}, Any}}:
([5, 6, 7, 8, 9, 10], 210)
([1, 2], [3, 4])
and you can get the result of the return value:
julia> length(comp(1:20))
2
Julia 1.4.2
The behaviour that stumps me occurs in the first and fourth cases:
julia> if (val = (1 == 2))
println(val)
end
julia> if (val = (1 == 1))
println(val)
end
true
julia> if (val = (1 == 1))
println(val + 5)
end
6
julia> if (val = (1 == 2))
println(val + 5)
end
julia> 1 == 2
false
julia> 1 == 1
true
julia> println(1==2)
false
julia> println(1==1)
true
I observe this in the REPL and in Jupyter.
My questions are:
Why does this happen?
How can I retrieve the value of val, preferably the "truthy" value, in both cases?
Let's look at the first example in more details
# initially, `val` does not exist
julia> val
ERROR: UndefVarError: val not defined
# 1==2 is false
# the assignment returns the value of the right-hand side -> false
julia> if (val = (1 == 2))
# therefore, this branch is not taken
println(val)
else
# but this one is
println("in the else clause")
end
in the else clause
# `if` returns whatever the branch taken evaluates to.
# In this case, `println` returns nothing, so the `if` block does not
# return anything either
julia> #show ans
ans = nothing
# `if` doesn't introduce a new scope, so the newly created `val` variable
# can still be inspected
julia> val
false
How (is it possible) to annotate the type of arguments when using the splat operator?
f(x, y) = x^2 + y^2
vec = [1.0, 2.0, 'a']
f(vec[1:2]...)
How can I annotate that use of ... in the function call. Also notice that none of the macros to view code (#code_llvm, #code_lowered, #code_native, #code_typed, #code_warntype) work, so it would be very hard to optimize when using the splat?
Because it seems that in the above use-case, macro versions of reflection functions couldn't reach the right argument types, using original function instead of macro, could be helpful:
f(x, y) = x^2 + y^2
vec = [1.0, 2.0, 'a']
#code_warntype(f(vec[1:2]...)) # => Nothing
code_warntype(f,map(typeof,vec[1:2]))
# Variables:
# x::Float64
# y::Float64
# .....
This logic is true for all reflection macros, using their variant function with a (function, collection of types).
references:
The macro #code_warntype has function variant: #code_warntype
How macros generated: macro generator
Util function to reach types: gen_call_with_extracted_types
I think you mean something like this:
julia> foo(args::Float64...) = sum([x^2 for x in args])::Float64
foo (generic function with 1 method)
julia> foo(args::Vector{Float64}) = foo(args...)::Float64
foo (generic function with 2 methods)
julia> foo(args::Tuple{Vararg{Float64}}) = foo(args...)::Float64
foo (generic function with 3 methods)
julia> foo(2.0, 5.5, 7.0)
83.25
julia> v = Float64[2, 5.5, 7.0]
3-element Array{Float64,1}:
2.0
5.5
7.0
julia> foo(v)
83.25
julia> t = tuple(v...)
(2.0,5.5,7.0)
julia> foo(t)
83.25
I've placed type anotations in several places, so you can get the feel of the possibilities.
julia> #which foo(2.0, 5.5, 7.0)
foo(args::Float64...) at none:1
julia> #which foo(v)
foo(args::Array{Float64,1}) at none:1
julia> #which foo(t)
foo(args::Tuple{Vararg{Float64}}) at none:1
#code_warntype, etc.
julia> #code_warntype foo(2.0, 5.5, 7.0)
Variables:
args::Tuple{Float64,Float64,Float64}
#s33::Int64
#s32::Int64
#s31::Int64
x::Float64
#s30::Int64
Body:
begin # none, line 1:
GenSym(1) = (Base.nfields)(args::Tuple{Float64,Float64,Float64})::Int64
0:
GenSym(3) = (top(ccall))(:jl_alloc_array_1d,(top(apply_type))(Base.Array,Float64,1)::Type{Array{Float64,1}},(top(svec))(Base.Any,Base.Int)::SimpleVector,Array{Flo
at64,1},0,GenSym(1),0)::Array{Float64,1}
#s33 = 1
#s32 = 1
#s31 = 0
unless (Base.box)(Base.Bool,(Base.not_int)(#s31::Int64 === GenSym(1)::Bool)::Any)::Bool goto 2
3:
#s31 = (Base.box)(Base.Int,(Base.add_int)(#s31::Int64,1)::Any)::Int64
GenSym(10) = (Base.getfield)(args::Tuple{Float64,Float64,Float64},#s32::Int64)::Float64
GenSym(11) = (Base.box)(Base.Int,(Base.add_int)(#s32::Int64,1)::Any)::Int64
#s30 = 1
GenSym(12) = GenSym(10)
GenSym(13) = (Base.box)(Base.Int,(Base.add_int)(1,1)::Any)::Int64
x = GenSym(12)
#s30 = GenSym(13)
GenSym(14) = GenSym(11)
GenSym(15) = (Base.box)(Base.Int,(Base.add_int)(2,1)::Any)::Int64
#s32 = GenSym(14)
#s30 = GenSym(15)
GenSym(4) = (Base.box)(Base.Float64,(Base.mul_float)(x::Float64,x::Float64)::Any)::Float64
$(Expr(:type_goto, 0, GenSym(4)))
$(Expr(:boundscheck, false))
(Base.arrayset)(GenSym(3),GenSym(4),#s33::Int64)::Array{Float64,1}
$(Expr(:boundscheck, :(Main.pop)))
#s33 = (Base.box)(Base.Int,(Base.add_int)(#s33::Int64,1)::Any)::Int64
4:
unless (Base.box)(Base.Bool,(Base.not_int)((Base.box)(Base.Bool,(Base.not_int)(#s31::Int64 === GenSym(1)::Bool)::Any)::Bool)::Any)::Bool goto 3
2:
1:
GenSym(8) = GenSym(3)
return (Base._mapreduce)($(Expr(:new, :((top(getfield))(Base,:IdFun)::Type{Base.IdFun}))),$(Expr(:new, :((top(getfield))(Base,:AddFun)::Type{Base.AddFun}))),GenSy
m(8))::Float64
end::Float64
julia> #code_warntype foo(v)
Variables:
args::Array{Float64,1}
Body:
begin # none, line 1:
return (top(_apply))((top(getfield))(Main,:call)::F,Main.foo,args::Array{Float64,1})::Float64
end::Float64
julia> #code_warntype foo(t)
Variables:
args::Tuple{Float64,Float64,Float64}
Body:
begin # none, line 1:
return (Main.foo)((top(getfield))(args::Tuple{Float64,Float64,Float64},1)::Float64,(top(getfield))(args::Tuple{Float64,Float64,Float64},2)::Float64,(top(getfield))(args::Tuple{Float64,Float64,Float64},3)::Float64)::Float64
end::Float64
Edit: IJulia notebook, tested at juliabox.org, Julia v0.4.1:
http://nbviewer.ipython.org/gist/Ismael-VC/edeb2f919c2341cb389c
You can also put type annotations when calling a function:
julia> #which foo(t::Tuple{Vararg{Float64}}...)
foo(args::Tuple{Vararg{Float64}}) at none:1
julia> #which foo(v::Vector{Float64}...)
foo(args::Array{Float64,1}) at none:1
I have a Array{Tuple{A, B}}, and I want to unzip / transpose it into Tuple{Array{A}, Array{B}}.
typealias MyIndexType Tuple{Bool, Int}
function test(x::MyIndexType)
# prepare for test data set.
myArray = Array{Tuple{MyIndexType, Float64}}(0)
push!(myArray, (x,1))
push!(myArray, (x,1))
push!(myArray, (x,1))
# transform
a, b = (zip(myArray...)...)
[a...]
end
test((true, 1))
>>>
3-element Array{Tuple{Bool,Int64},1}:
(true,1)
(true,1)
(true,1)
However, using #code_warntype, JIT cannot infer the type of a, b ahead of time.
Variables:
x::Tuple{Bool,Int64}
myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1}
a::ANY
b::ANY
#s41::Int64
Body:
begin # In[47], line 6:
myArray = (top(ccall))(:jl_alloc_array_1d,(top(apply_type))(Base.Array,Tuple{Tuple{Bool,Int64},Float64},1)::Type{Array{Tuple{Tuple{Bool,Int64},Float64},1}},(top(svec))(Base.Any,Base.Int)::SimpleVector,Array{Tuple{Tuple{Bool,Int64},Float64},1},0,0,0)::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 7:
(Main.push!)(myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1},(top(tuple))(x::Tuple{Bool,Int64},1)::Tuple{Tuple{Bool,Int64},Int64})::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 8:
(Main.push!)(myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1},(top(tuple))(x::Tuple{Bool,Int64},1)::Tuple{Tuple{Bool,Int64},Int64})::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 9:
(Main.push!)(myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1},(top(tuple))(x::Tuple{Bool,Int64},1)::Tuple{Tuple{Bool,Int64},Int64})::Array{Tuple{Tuple{Bool,Int64},Float64},1} # In[47], line 10:
GenSym(0) = (top(_apply))((top(getfield))(Main,:call)::F,top(tuple),(top(_apply))((top(getfield))(Main,:call)::F,Main.zip,myArray::Array{Tuple{Tuple{Bool,Int64},Float64},1})::UNION{BASE.ZIP2{TUPLE{TUPLE{BOOL,INT64},FLOAT64},TUPLE{TUPLE{BOOL,INT64},FLOAT64}},TUPLE{TUPLE{BOOL,INT64},FLOAT64},ZIP{I,Z<:BASE.ABSTRACTZIPITERATOR}})::TUPLE
#s41 = 1
GenSym(4) = (Base.getfield)(GenSym(0),1)::ANY
GenSym(5) = (Base.box)(Base.Int,(Base.add_int)(1,1)::ANY)::Int64
a = GenSym(4)
#s41 = GenSym(5)
GenSym(6) = (Base.getfield)(GenSym(0),2)::ANY
GenSym(7) = (Base.box)(Base.Int,(Base.add_int)(2,1)::ANY)::Int64
b = GenSym(6)
#s41 = GenSym(7) # In[47], line 11:
return (top(_apply))((top(getfield))(Main,:call)::F,top(vect),a)::ANY
end::ANY
Is there a way to make zip aware of the resulting types?
Update
Actually there are 2 problems.
It thinks a is of type a::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}, but it is actually of type a::TUPLE{FLOAT64,FLOAT64}
function test{T}(x::T)
A = Tuple{T, Int}[]
for i in 1:3
push!(A, (x, 1))
end
d = zip(A[1], A[2])
a, b = d
a
end
#code_warntype test(3.0)
Variables:
x::Float64
A::Array{Tuple{Float64,Int64},1}
d::Base.Zip2{Tuple{Float64,Int64},Tuple{Float64,Int64}}
a::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
b::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
#s40::Tuple{Int64,Int64}
#s41::Int64
i::Int64
For zip taking more than 2 arguments, note d has a nested zip2 type, which I feel may bring burden to type inference.
function test{T}(x::T)
A = Tuple{T, Int}[]
for i in 1:3
push!(A, (x, 1))
end
d = zip(A[1], A[2], A[3])
a, b = d
a
end
#code_warntype test(3.0)
Variables:
x::Float64
A::Array{Tuple{Float64,Int64},1}
d::Zip{Tuple{Float64,Int64},Base.Zip2{Tuple{Float64,Int64},Tuple{Float64,Int64}}}
a::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
b::TUPLE{UNION{FLOAT64,INT64},UNION{FLOAT64,INT64},UNION{FLOAT64,INT64}}
#s40::Tuple{Int64,Tuple{Int64,Int64}}
#s41::Int64
i::Int64
##c#7879::Tuple{Tuple{Float64,Int64}}
Why do I care about the type?
It takes 10+ seconds to compile a, b = zip(A...) of the following example, and the speed seems to be related to the length of A. (Julia 0.4)
const A = Tuple{Int, Int}[]
for i = 1:200
push!(A, (1, 1))
end
a, b = zip(A...)
a
I opened a bug report here https://github.com/JuliaLang/julia/issues/13722
I believe that it is not a problem that #code_warntype reports that it fails to infer the correct types if it gets it right in the end.
I was nonetheless wondering if this is due to the complexity of your type. But it isn't, as the code below shows (with a simpler type).
Note that you can also simplify your zip expression; and you probably don't need to convert a to an array.
Code:
function test{T}(x::T)
A = Tuple{T, Int}[]
for i in 1:3
push!(A, (x, 1))
end
a, b = zip(A...)
a, b
end
julia> test(3) # now returns a and b
((3,3,3),(2,2,2))
julia> #code_warntype test(3)
Variables:
x::Int64
A::Array{Tuple{Int64,Int64},1}
a::ANY
b::ANY
#s40::ANY
#s41::Int64
i::Int64