Nested list comprehensions in Julia - julia

In python I can do nested list comprehensions, for instance I can flatten the following array thus:
a = [[1,2,3],[4,5,6]]
[i for arr in a for i in arr]
to get [1,2,3,4,5,6]
If I try this syntax in Julia I get:
julia> a
([1,2,3],[4,5,6],[7,8,9])
julia> [i for arr in a for i in arr]
ERROR: syntax: expected ]
Are nested list comprehensions in Julia possible?

This feature has been added in julia v0.5:
julia> a = ([1,2,3],[4,5,6],[7,8,9])
([1,2,3],[4,5,6],[7,8,9])
julia> [i for arr in a for i in arr]
9-element Array{Int64,1}:
1
2
3
4
5
6
7
8
9

List comprehensions work a bit differently in Julia:
> [(x,y) for x=1:2, y=3:4]
2x2 Array{(Int64,Int64),2}:
(1,3) (1,4)
(2,3) (2,4)
If a=[[1 2],[3 4],[5 6]] was a multidimensional array, vec would flatten it:
> vec(a)
6-element Array{Int64,1}:
1
2
3
4
5
6
Since a contains tuples, this is a bit more complicated in Julia. This works, but likely isn't the best way to handle it:
function flatten(x, y)
state = start(x)
if state==false
push!(y, x)
else
while !done(x, state)
(item, state) = next(x, state)
flatten(item, y)
end
end
y
end
flatten(x)=flatten(x,Array(Any, 0))
Then, we can run:
> flatten([(1,2),(3,4)])
4-element Array{Any,1}:
1
2
3
4

You can get some mileage out of using the splat operator with the array constructor here (transposing to save space)
julia> a = ([1,2,3],[4,5,6],[7,8,9])
([1,2,3],[4,5,6],[7,8,9])
julia> [a...]'
1x9 Array{Int64,2}:
1 2 3 4 5 6 7 8 9

Any reason why you're using a tuple of vectors? It's much simpler with arrays, as Ben has already shown with vec. But you can also use comprehensions pretty simply in either case:
julia> a = ([1,2,3],[4,5,6],[7,8,9]);
julia> [i for i in hcat(a...)]
9-element Array{Any,1}:
1
2
⋮
The expression hcat(a...) "splats" your tuple and concatenates it into an array. But remember that, unlike Python, Julia uses column-major array semantics. You have three column vectors in your tuple; is that what you intend? (If they were row vectors — delimited by spaces — you could just use [a...] to do the concatenation). Arrays are iterated through all elements, regardless of their dimensionality.

Don't have enough reputation for comment so posting a modification #ben-hammer. Thanks for the example of flatten(), it was helpful to me.
But it did break if the tuples/arrays contained strings. Since strings are iterables the function would further break them down to characters. I had to insert condition to check for ASCIIString to fix that. The code is below
function flatten(x, y)
state = start(x)
if state==false
push!(y, x)
else
if typeof(x) <: String
push!(y, x)
else
while (!done(x, state))
(item, state) = next(x, state)
flatten(item, y)
end
end
end
y
end
flatten(x)=flatten(x,Array(Any, 0))

Related

How to delete an element from a list in Julia?

v = range(1e10, -1e10, step=-1e8) # velocities [cm/s]
deleteat!(v, findall(x->x==0,v))
I want to delete the value 0 from v. Following this tutorial, I tried deleteat! but I get the error
MethodError: no method matching deleteat!(::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, ::Vector{Int64})
What am I missing here?
Notice the type that is returned by the function range.
typeof(range(1e10, -1e10, step=-1e8))
The above yields to
StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}
Calling the help function for the function deleteat!.
? deleteat!()
deleteat!(a::Vector, inds)
Remove the items at the indices given by inds, and return the > modified a. Subsequent items are shifted to fill the resulting gap.
inds can be either an iterator or a collection of sorted and > unique integer indices, or a boolean vector of the same length as a with true indicating entries to delete.
We can convert the returned type of range using collect. Try the following code.
v = collect(range(1e10, -1e10, step=-1e8))
deleteat!(v,findall(x->x==0,v))
Notice that we can shorten x->x==0 to iszero which yields to
v = collect(range(1e10, -1e10, step=-1e8))
deleteat!(v,findall(iszero,v))
Use filter! or filter:
julia> filter!(!=(0), [1,0,2,0,4])
3-element Vector{Int64}:
1
2
4
In case of a range you can collect it or use:
julia> filter(!=(0), range(2, -2, step=-1))
4-element Vector{Int64}:
2
1
-1
-2
However for big ranges you might just not want to materialize them to save the memory footprint. In that case you could use:
(x for x in range(2, -2, step=-1) if x !== 0)
To see what is being generated you need to collect it:
julia> collect(x for x in range(2, -2, step=-1) if x !== 0)
4-element Vector{Int64}:
2
1
-1
-2

Julia 1.0.0: What does the `=>` operator do?

I see this Stackoverflow code for =>, but when I search Julia 1.0.0 on-line help for "=>", I get zero hits.
replace!(x, 0=>4) # The last expression is the focus of this question.
In the REPL help I get:
help?> =>
search: =>
Pair(x, y)
x => y
Construct a Pair object with type Pair{typeof(x), typeof(y)}. The elements are stored in the fields first and second.
They can also be accessed via iteration.
See also: Dict
Examples
≡≡≡≡≡≡≡≡≡≡
julia> p = "foo" => 7
"foo" => 7
julia> typeof(p)
Pair{String,Int64}
julia> p.first
"foo"
julia> for x in p
println(x)
end
foo
7
What does => do in replace!(x, 0=>4)? Does it create a pair, a replacement of all zeros by fours, or what? Why do I seem to not find it in the Julia 1.0.0 on-line docs?
EDIT
Code added to help me understand #Bill's helpful answer below:
julia> x = [1, 0, 3, 2, 0]
5-element Array{Int64,1}:
1
0
3
2
0
julia> replace!(x, 0=>4)
5-element Array{Int64,1}:
1
4
3
2
4
Edit 2
Besides #Bill's accepted answer, I found #Steven's answer helpful as well. Sorry I could not check them both, but Bill's came in first and they both offered useful information.
"What does => do in replace!(x, 0=>4)? Does it create a pair, a replacement of all zeros by fours, or what?"
It creates a Pair. In the function replace, a Pair in the second argument position means the multiple dispatch of replace() chooses a version of the replace function where, given a numeric array or string x, all items within x fitting the first part of the Pair are replaced with an instance of the second part of the Pair.
You can check the REPL docs for replace for details.
This small example should show how "=>" makes a pair
julia> replace("julia", Pair("u", "o"))
"jolia"
julia> replace("julia", "u" => "o")
"jolia"
"=>" operator means "Change into"
so
julia> replace("hello world",'l' => 'z')
"hezzo worzd"
means Change the string "hello world" using "change" 'l' "into" 'z'
and producing the resultant string "hezzo worzd"
julia> replace( [1,2,3,4,5], 3 => 666 )
5-element Array{Int64,1}:
1
2
666
4
5

Transform nested array into new dimension

Given an array as follows:
A = Array{Array{Int}}(2,2)
A[1,1] = [1,2]
A[1,2] = [3,4]
A[2,1] = [5,6]
A[2,2] = [7,8]
We then have that A is a 2x2 array with elements of type Array{Int}:
2×2 Array{Array{Int64,N} where N,2}:
[1, 2] [3, 4]
[5, 6] [7, 8]
It is possible to access the entries with e.g. A[1,2] but A[1,2,2] would not work since the third dimension is not present in A. However, A[1,2][2] works, since A[1,2] returns an array of length 2.
The question is then, what is a nice way to convert A into a 3-dimensional array, B, so that B[i,j,k] refers the the i,j-th array and the k-th element in that array. E.g. B[2,1,2] = 6.
There is a straightforward way to do this using 3 nested loops and reconstructing the array, element-by-element, but I'm hoping there is a nicer construction. (Some application of cat perhaps?)
You can construct a 3-d array from A using an array comprehension
julia> B = [ A[i,j][k] for i=1:2, j=:1:2, k=1:2 ]
2×2×2 Array{Int64,3}:
[:, :, 1] =
1 3
5 7
[:, :, 2] =
2 4
6 8
julia> B[2,1,2]
6
However a more general solution would be to overload the getindex function for arrays with the same type of A. This is more efficient since there is no need to copy the original data.
julia> import Base.getindex
julia> getindex(A::Array{Array{Int}}, i::Int, j::Int, k::Int) = A[i,j][k]
getindex (generic function with 179 methods)
julia> A[2,1,2]
6
With thanks to Dan Getz's comments, I think the following works well and is succinct:
cat(3,(getindex.(A,i) for i=1:2)...)
where 2 is the length of the nested array. It would also work for higher dimensions.
permutedims(reshape(collect(Base.Iterators.flatten(A)), (2,2,2)), (2,3,1))
also does the job and appears to be faster than the accepted cat() answer for me.
EDIT: I'm sorry, I just saw that this has already been suggested in the comments.

Find indices of non-zero elements from [1,2,0,0,4,0] in julia and create an Arrray with them

I know there is a function that does this, for example:
A = [1,2,0,0,4,0]
find(A)
3-element Array{Int64,1}:
1
2
5
I am trying to do it on my own way, however, I am stuck here
for i=1:endof(A)
if A[i] != 0
[]
end
end
Thanks in advance.
Here's one alternative:
function myfind(c)
a = similar(c, Int)
count = 1
#inbounds for i in eachindex(c)
a[count] = i
count += (c[i] != zero(eltype(c)))
end
return resize!(a, count-1)
end
It actually outperformed find for all the cases I tested, though for the very small example vector you posted, the difference was negligible. There is perhaps some performance advantage to avoiding the branch and dynamically growing the index array.
I have notice that the question is really confusion (because is poorly formulated, sorry about that). Therefore, there are two possible answers: one is [1,2,4]which is an array with the non-zero elements; the other is [1,2,5] which is an array of the indices of the non-zero elements.
Let´s begin with the first option
A = [1,2,0,0,4,0]
B = []
for i=1:endof(A)
if A[i] != 0
push!(B,i)
end
end
println(B)
The output is Any[1,2,5]
However, the type is not the one I wanted. Using typeof(B) it shows Array{Any,1} so I added this code:
B = Array{Int64}(B)
println(B)
typeof(B)
And the result is the desired
[1,2,5]
Array{Int64,1}
To improve its efficiency, following with the recommendations in the comments, I have specified the type of B with the eltype() function before the loop as follows:
A1 = [1,2,0,0,4,0] #The default type is Array{Int64,1}
B1 = eltype(A1)[] #Define the type of the 0 element array B with the type of A
#B1 = eltype(typeof(A))[] this is also valid
for i=1:endof(A1)
if A1[i] != 0
push!(B1::Array{Int64,1},i::Int64)
end
end
println(B1)
typeof(B1)
Then, the output is again the desired
[1,2,5]
Array{Int64,1}
The simplest way of doing this is using the function find(). However, since I´m a beginner, I wanted to do it in another way. However, there is another alternative provided by #DNF that outperform find() for the cases he has tested it (see below answers).
The second option, which creates an output matrix with the non-zero elements has been provided by other users (#Harrison Grodin and #P i) in this discussion.
Thanks all of you for the help!
You have a few options here.
Using the strategy you started with, you can use push! inside the for loop.
julia> B = Int[]
0-element Array{Int64,1}
julia> for element = A
if element != 0
push!(B, element)
end
end
julia> B
3-element Array{Int64,1}:
1
2
4
You can also opt to use short-circuit evaluation.
julia> for element = A
element != 0 && push!(B, element)
end
julia> for element = A
element == 0 || push!(B, element)
end
Both filter and list comprehensions are valid, as well!
julia> B = [element for element = A if element != 0]
3-element Array{Int64,1}:
1
2
4
julia> filter(n -> n != 0, A)
3-element Array{Int64,1}:
1
2
4
Edit: Thanks to the OP's comment, I have realized that the desired result is the indices of the nonzero elements, not the elements themselves. This can be achieved simply with the following. :)
julia> find(A)
3-element Array{Int64,1}:
1
2
5
Just because I don't see a simple solution posted here, this is approach I took to the same problem. I think it is the simplest and most elegant solution. Hope this closes the thread!
julia> A = [1,2,0,0,4,0];
julia> findall(!iszero,A)
3-element Vector{Int64}:
1
2
5

Re. partitions()

Why is
julia> collect(partitions(1,2))
0-element Array{Any,1}
returned instead of
2-element Array{Any,1}:
[0,1]
[1,0]
and do I really have to
x = collect(partitions(n,m));
y = Array(Int64,length(x),length(x[1]));
for i in 1:length(x)
for j in 1:length(x[1])
y[i,j] = x[i][j];
end
end
to convert the result to a two-dimensional array?
From the wikipedia:
In number theory and combinatorics, a partition of a positive integer n, also called an integer partition, is a way of writing n as a sum of positive integers.
For array conversion, try:
julia> x = collect(partitions(5,3))
2-element Array{Any,1}:
[3,1,1]
[2,2,1]
or
julia> x = partitions(5,3)
Base.FixedPartitions(5,3)
then
julia> hcat(x...)
3x2 Array{Int64,2}:
3 2
1 2
1 1
Here's another approach to your problem that I think is a little simpler, using the Combinatorics.jl library:
multisets(n, k) = map(A -> [sum(A .== i) for i in 1:n],
with_replacement_combinations(1:n, k))
This allocates a bunch of memory, but I think your current approach does too. Maybe it would be useful to make a first-class version and add it to Combinatorics.jl.
Examples:
julia> multisets(2, 1)
2-element Array{Array{Int64,1},1}:
[1,0]
[0,1]
julia> multisets(3, 5)
21-element Array{Array{Int64,1},1}:
[5,0,0]
[4,1,0]
[4,0,1]
[3,2,0]
[3,1,1]
[3,0,2]
[2,3,0]
[2,2,1]
[2,1,2]
[2,0,3]
⋮
[1,2,2]
[1,1,3]
[1,0,4]
[0,5,0]
[0,4,1]
[0,3,2]
[0,2,3]
[0,1,4]
[0,0,5]
The argument order is backwards from yours to match mathematical convention. If you prefer the other way, that can easily be changed.
one robust solution can be achieved using lexicographic premutations generation algorithm, originally By Donald Knuth plus classic partitions(n).
that is lexicographic premutations generator:
function lpremutations{T}(a::T)
b=Vector{T}()
sort!(a)
n=length(a)
while(true)
push!(b,copy(a))
j=n-1
while(a[j]>=a[j+1])
j-=1
j==0 && return(b)
end
l=n
while(a[j]>=a[l])
l-=1
end
tmp=a[l]
a[l]=a[j]
a[j]=tmp
k=j+1
l=n
while(k<l)
tmp=a[k]
a[k]=a[l]
a[l]=tmp
k+=1
l-=1
end
end
end
The above algorithm will generates all possible unique
combinations of an array elements with repetition:
julia> lpremutations([2,2,0])
3-element Array{Array{Int64,1},1}:
[0,2,2]
[2,0,2]
[2,2,0]
Then we will generate all integer arrays that sum to n using partitions(n) (forget the length of desired arrays m), and resize them to the lenght m using resize_!
function resize_!(x,m)
[x;zeros(Int,m-length(x))]
end
And main function looks like:
function lpartitions(n,m)
result=[]
for i in partitions(n)
append!(result,lpremutations(resize_!(i, m)))
end
result
end
Check it
julia> lpartitions(3,4)
20-element Array{Any,1}:
[0,0,0,3]
[0,0,3,0]
[0,3,0,0]
[3,0,0,0]
[0,0,1,2]
[0,0,2,1]
[0,1,0,2]
[0,1,2,0]
[0,2,0,1]
[0,2,1,0]
[1,0,0,2]
[1,0,2,0]
[1,2,0,0]
[2,0,0,1]
[2,0,1,0]
[2,1,0,0]
[0,1,1,1]
[1,0,1,1]
[1,1,0,1]
[1,1,1,0]
The MATLAB script from http://www.mathworks.com/matlabcentral/fileexchange/28340-nsumk actually behaves the way I need, and is what I though that partitions() would do from the description given. The Julia version is
# k - sum, n - number of non-negative integers
function nsumk(k,n)
m = binomial(k+n-1,n-1);
d1 = zeros(Int16,m,1);
d2 = collect(combinations(collect((1:(k+n-1))),n-1));
d2 = convert(Array{Int16,2},hcat(d2...)');
d3 = ones(Int16,m,1)*(k+n);
dividers = [d1 d2 d3];
return diff(dividers,2)-1;
end
julia> nsumk(3,2)
4x2 Array{Int16,2}:
0 3
1 2
2 1
3 0
using daycaster's lovely hcat(x...) tidbit :)
I still wish there would be a more compact way of doing this.
The the first mention of this approach seem to be https://au.mathworks.com/matlabcentral/newsreader/view_thread/52610, and as far as I can understand it is based on the "stars and bars" method https://en.wikipedia.org/wiki/Stars_and_bars_(combinatorics)

Resources