List comprehensions and tuples in Julia - julia

I am trying to do in Julia what this Python code does. (Find all pairs from the two lists whose combined value is above 7.)
#Python
def sum_is_large(a, b):
return a + b > 7
l1 = [1,2,3]
l2 = [4,5,6]
l3 = [(a,b) for a in l1 for b in l2 if sum_is_large(a, b)]
print(l3)
There is no if for list comprehensions in Julia. And if I use filter(), I'm not sure if I can pass two arguments. So my best suggestion is this:
#Julia
function sum_is_large(pair)
a, b = pair
return a + b > 7
end
l1 = [1,2,3]
l2 = [4,5,6]
l3 = filter(sum_is_large, [(i,j) for i in l1, j in l2])
print(l3)
I don't find this very appealing. So my question is, is there a better way in Julia?

Using the very popular package Iterators.jl, in Julia:
using Iterators # install using Pkg.add("Iterators")
filter(x->sum(x)>7,product(l1,l2))
is an iterator producing the pairs. So to get the same printout as the OP:
l3iter = filter(x->sum(x)>7,product(l1,l2))
for p in l3iter println(p); end
The iterator approach is potentially much more memory efficient. Ofcourse, one could just l3 = collect(l3iter) to get the pair vector.
#user2317519, just curious, is there an equivalent iterator form for python?

Guards (if) are now available in Julia v0.5 (currently in the release-candidate stage):
julia> v1 = [1, 2, 3];
julia> v2 = [4, 5, 6];
julia> v3 = [(a, b) for a in v1, b in v2 if a+b > 7]
3-element Array{Tuple{Int64,Int64},1}:
(3,5)
(2,6)
(3,6)
Note that generators are also now available:
julia> g = ( (a, b) for a in v1, b in v2 if a+b > 7 )
Base.Generator{Filter{##18#20,Base.Prod2{Array{Int64,1},Array{Int64,1}}},##17#19}(#17,Filter{##18#20,Base.Prod2{Array{Int64,1},Array{Int64,1}}}(#18,Base.Prod2{Array{Int64,1},Array{Int64,1}}([1,2,3],[4,5,6])))

Another option similar to the one of #DanGetz using also Iterators.jl:
function expensive_fun(a, b)
return (a + b)
end
Then, if the condition is also complicated, it can be defined as a function:
condition(x) = x > 7
And last, filter the results:
>>> using Iterators
>>> result = filter(condition, imap(expensive_fun, l1, l2))
result is an iterable that is only computed when needed (inexpensive) and can be collected collect(result) if required.
The one-line if the filter condition is simple enough would be:
>>> result = filter(x->(x > 7), imap(expensive_fun, l1, l2))
Note: imap works natively for arbitrary number of parameters.

Perhaps something like this:
julia> filter(pair -> pair[1] + pair[2] > 7, [(i, j) for i in l1, j in l2])
3-element Array{Tuple{Any,Any},1}:
(3,5)
(2,6)
(3,6)
although I'd agree it doesn't look like it ought to be the best way...

I'm surprised nobody mentions the ternary operator to implement the conditional:
julia> l3 = [sum_is_large((i,j)) ? (i,j) : nothing for i in l1, j in l2]
3x3 Array{Tuple,2}:
nothing nothing nothing
nothing nothing (2,6)
nothing (3,5) (3,6)
or even just a normal if block within a compound statement, i.e.
[ (if sum_is_large((x,y)); (x,y); end) for x in l1, y in l2 ]
which gives the same result.
I feel this result makes a lot more sense than filter(), because in julia the a in A, b in B construct is interpreted dimensionally, and therefore the output is in fact an "array comprehension" with appropriate dimensionality, which clearly in many cases would be advantageous and presumably the desired behaviour (whether we include a conditional or not).
Whereas filter will always return a vector. Obviously, if you really want a vector result you can always collect the result; or for a conditional list comprehension like the one here, you can simply remove nothing elements from the array by doing l3 = l3[l3 .!= nothing].
Presumably this is still clearer and no less efficient than the filter() approach.

You can use the #vcomp (vector comprehension) macro in VectorizedRoutines.jl to do Python-like comprehensions:
using VectorizedRoutines
Python.#vcomp Int[i^2 for i in 1:10] when i % 2 == 0 # Int[4, 16, 36, 64, 100]

Related

Why (; [(:x, 1), (:y, 2)]...) creates a NamedTuple?

I'm still learning Julia, and I recently came across the following code excerpt that flummoxed me:
res = (; [(:x, 10), (:y, 20)]...) # why the semicolon in front?
println(res) # (x = 10, y = 20)
println(typeof(res)) # NamedTuple{(:x, :y), Tuple{Int64, Int64}}
I understand the "splat" operator ..., but what happens when the semicolon appear first in a tuple? In other words, how does putting a semicolon in (; [(:x, 10), (:y, 20)]...) create a NamedTuple? Is this some undocumented feature/trick?
Thanks for any pointers.
Yes, this is actually a documented feature, but perhaps not a very well known one. As the documentation for NamedTuple notes:
help?> NamedTuple
search: NamedTuple #NamedTuple
NamedTuple
NamedTuples are, as their name suggests, named Tuples. That is, they're a tuple-like
collection of values, where each entry has a unique name, represented as a Symbol.
Like Tuples, NamedTuples are immutable; neither the names nor the values can be
modified in place after construction.
Accessing the value associated with a name in a named tuple can be done using field
access syntax, e.g. x.a, or using getindex, e.g. x[:a]. A tuple of the names can be
obtained using keys, and a tuple of the values can be obtained using values.
[... some other non-relevant parts of the documentation omitted ...]
In a similar fashion as to how one can define keyword arguments programmatically, a
named tuple can be created by giving a pair name::Symbol => value or splatting an
iterator yielding such pairs after a semicolon inside a tuple literal:
julia> (; :a => 1)
(a = 1,)
julia> keys = (:a, :b, :c); values = (1, 2, 3);
julia> (; zip(keys, values)...)
(a = 1, b = 2, c = 3)
As in keyword arguments, identifiers and dot expressions imply names:
julia> x = 0
0
julia> t = (; x)
(x = 0,)
julia> (; t.x)
(x = 0,)

Return a list of even numbers from a list of integer pairs in sml

I have the following question "Given a list of integer pairs, write a function to return a list of even numbers in that list in sml".
this is what I've achieved so far
val x = [(6, 2), (3, 4), (5, 6), (7, 8), (9, 10)];
fun isEven(num : int) =
if num mod 2 = 0 then num else 0;
fun evenNumbers(list : (int * int) list) =
if null list then [] else
if isEven(#1 (hd list)) <> 0
then if isEven(#2 (hd list)) <> 0
then #1 (hd list) :: #1 (hd list) :: evenNumbers(tl list)
else []
else if isEven(#2 (hd list)) <> 0
then #1 (hd list) :: evenNumbers(tl list)
else [];
evenNumbers(x);
the result should be like this [6,2,4,6,8,10]
any help would be appreciated.
I see two obvious problems.
If both the first and second number are even, you do
#1 (hd list) :: #1 (hd list) :: evenNumbers(tl list)
which adds the first number twice and ignores the second.
If the first number is odd and the second even, you do
#1 (hd list) :: evenNumbers(tl list)
which adds the number that you know is odd and ignores the one you know is even.
Programming with selectors and conditionals gets complicated very quickly (as you've noticed).
With pattern matching, you could write
fun evenNumbers [] = []
| evenNumber ((x,y)::xys) = ...
and reduce the risk of using the wrong selector.
However, this still makes for complicated logic, and there is a better way.
Consider the simpler problem of filtering the odd numbers out of a list of numbers, not pairs.
If you transform the input into such a list, you only need to solve that simpler problem (and there's a fair chance that you've already solved something very similar in a previous exercise).
Exercise: implement this transformation. Its type will be ('a * 'a) list -> 'a list.
Also, your isEven is more useful if it produces a truth value (if you ask someone, "is 36 even?", "36" is a very strange answer).
fun isEven x = x mod 2 = 0
Now, evenNumbers can be implemented as "just" a combination of other, more general, functions.
So running your current code,
- evenNumbers [(6, 2), (3, 4), (5, 6), (7, 8), (9, 10)];
val it = [6,6,3,5,7,9] : int list
suggests that you're not catching all even numbers, and that you're catching some odd numbers.
The function isEven sounds very much like you want to have the type int -> bool like so:
fun isEven n =
n mod 2 = 0
Instead of addressing the logic error of your current solution, I would like to propose a syntactically much simpler approach which is to use pattern matching and fewer explicit type annotations. One basis for such a solution could look like:
fun evenNumbers [] = ...
| evenNumbers ((x,y)::pairs) = ...
Using pattern matching is an alternative to if-then-else: the [] pattern is equivalent to if null list ... and the (x,y)::pairs pattern matches when the input list is non-empty (holds at least one element, being (x,y). At the same time, it deconstructs this one element into its parts, x and y. So in the second function body you can express isEven x and isEven y.
As there is a total of four combinations of whether x and y are even or not, this could easily end up with a similarly complicated nest of if-then-else's. For this I might do either one of two things:
Use case-of (and call evenNumbers recursively on pairs):
fun evenNumbers [] = ...
| evenNumbers ((x,y)::pairs) =
case (isEven x, isEven y) of
... => ...
| ... => ...
Flatten the list of pairs into a list of integers and filter it:
fun flatten [] = ...
| flatten ((x,y)::pairs) = ...
val evenNumbers pairs = ...

Get a number from an array of digits

To split a number into digits in a given base, Julia has the digits() function:
julia> digits(36, base = 4)
3-element Array{Int64,1}:
0
1
2
What's the reverse operation? If you have an array of digits and the base, is there a built-in way to convert that to a number? I could print the array to a string and use parse(), but that sounds inefficient, and also wouldn't work for bases > 10.
The previous answers are correct, but there is also the matter of efficiency:
sum([x[k]*base^(k-1) for k=1:length(x)])
collects the numbers into an array before summing, which causes unnecessary allocations. Skip the brackets to get better performance:
sum(x[k]*base^(k-1) for k in 1:length(x))
This also allocates an array before summing: sum(d.*4 .^(0:(length(d)-1)))
If you really want good performance, though, write a loop and avoid repeated exponentiation:
function undigit(d; base=10)
s = zero(eltype(d))
mult = one(eltype(d))
for val in d
s += val * mult
mult *= base
end
return s
end
This has one extra unnecessary multiplication, you could try to figure out some way of skipping that. But the performance is 10-15x better than the other approaches in my tests, and has zero allocations.
Edit: There's actually a slight risk to the type handling above. If the input vector and base have different integer types, you can get a type instability. This code should behave better:
function undigits(d; base=10)
(s, b) = promote(zero(eltype(d)), base)
mult = one(s)
for val in d
s += val * mult
mult *= b
end
return s
end
The answer seems to be written directly within the documentation of digits:
help?> digits
search: digits digits! ndigits isdigit isxdigit disable_sigint
digits([T<:Integer], n::Integer; base::T = 10, pad::Integer = 1)
Return an array with element type T (default Int) of the digits of n in the given base,
optionally padded with zeros to a specified size. More significant digits are at higher
indices, such that n == sum([digits[k]*base^(k-1) for k=1:length(digits)]).
So for your case this will work:
julia> d = digits(36, base = 4);
julia> sum([d[k]*4^(k-1) for k=1:length(d)])
36
And the above code can be shortened with the dot operator:
julia> sum(d.*4 .^(0:(length(d)-1)))
36
Using foldr and muladd for maximum conciseness and efficiency
undigits(d; base = 10) = foldr((a, b) -> muladd(base, b, a), d, init=0)

smlnj - Function That Adds Even and Odd Elements In An Int List

I am fairly new to functional programming and I do not understand my error here. I am trying to make a function that takes an integer list and returns both the sum of the even elements and the sum of the odd elements. The error I am getting is in line 1, and it states: "Error: right-hand-side of clause doesn't agree with function result type [overload conflict] ...". I don't understand the error, and I would appreciate any help in understanding my error.
fun add(nil) = 0
| add([x]) = x
| add(x :: xs) =
let
val evenList = xs;
val oddList = x :: xs
in
(hd evenList + add(tl(tl(evenList))), hd oddList + add(tl(tl(oddList))))
end;
The reason for the type error is that the function should return a pair, but your base cases don't.
I suspect you got to that code by thinking about skipping every other element, dividing the list by skipping.
There's a different way to approach this.
Consider the list [a,b,c,d].
Counting from 1, the elements are numbered
1 2 3 4
a b c d
Now consider the positions in the tail of the list.
They are
1 2 3
b c d
That is, odd positions in the tail are even positions in the entire list, and even positions in the tail are odd in the entire list.
This means that if we recursively compute "odds and evens" in the tail, we will get the sums from the tail, where its "odds" is our "evens", and if we add our head to the tail's "evens", we will get the "odds" we want.
All we need now is a good base case – and the sums of an empty list must be (0, 0).
Something like this:
fun add [] = (0,0)
| add (x::xs) = case add xs of
(odds, evens) => (x + evens, odds)
or, you can deconstruct the recursive result with a let-binding instead of case:
fun add [] = (0,0)
| add (x::xs) = let val (odds, evens) = add xs
in
(x + evens, odds)
end

Re. partitions()

Why is
julia> collect(partitions(1,2))
0-element Array{Any,1}
returned instead of
2-element Array{Any,1}:
[0,1]
[1,0]
and do I really have to
x = collect(partitions(n,m));
y = Array(Int64,length(x),length(x[1]));
for i in 1:length(x)
for j in 1:length(x[1])
y[i,j] = x[i][j];
end
end
to convert the result to a two-dimensional array?
From the wikipedia:
In number theory and combinatorics, a partition of a positive integer n, also called an integer partition, is a way of writing n as a sum of positive integers.
For array conversion, try:
julia> x = collect(partitions(5,3))
2-element Array{Any,1}:
[3,1,1]
[2,2,1]
or
julia> x = partitions(5,3)
Base.FixedPartitions(5,3)
then
julia> hcat(x...)
3x2 Array{Int64,2}:
3 2
1 2
1 1
Here's another approach to your problem that I think is a little simpler, using the Combinatorics.jl library:
multisets(n, k) = map(A -> [sum(A .== i) for i in 1:n],
with_replacement_combinations(1:n, k))
This allocates a bunch of memory, but I think your current approach does too. Maybe it would be useful to make a first-class version and add it to Combinatorics.jl.
Examples:
julia> multisets(2, 1)
2-element Array{Array{Int64,1},1}:
[1,0]
[0,1]
julia> multisets(3, 5)
21-element Array{Array{Int64,1},1}:
[5,0,0]
[4,1,0]
[4,0,1]
[3,2,0]
[3,1,1]
[3,0,2]
[2,3,0]
[2,2,1]
[2,1,2]
[2,0,3]
⋮
[1,2,2]
[1,1,3]
[1,0,4]
[0,5,0]
[0,4,1]
[0,3,2]
[0,2,3]
[0,1,4]
[0,0,5]
The argument order is backwards from yours to match mathematical convention. If you prefer the other way, that can easily be changed.
one robust solution can be achieved using lexicographic premutations generation algorithm, originally By Donald Knuth plus classic partitions(n).
that is lexicographic premutations generator:
function lpremutations{T}(a::T)
b=Vector{T}()
sort!(a)
n=length(a)
while(true)
push!(b,copy(a))
j=n-1
while(a[j]>=a[j+1])
j-=1
j==0 && return(b)
end
l=n
while(a[j]>=a[l])
l-=1
end
tmp=a[l]
a[l]=a[j]
a[j]=tmp
k=j+1
l=n
while(k<l)
tmp=a[k]
a[k]=a[l]
a[l]=tmp
k+=1
l-=1
end
end
end
The above algorithm will generates all possible unique
combinations of an array elements with repetition:
julia> lpremutations([2,2,0])
3-element Array{Array{Int64,1},1}:
[0,2,2]
[2,0,2]
[2,2,0]
Then we will generate all integer arrays that sum to n using partitions(n) (forget the length of desired arrays m), and resize them to the lenght m using resize_!
function resize_!(x,m)
[x;zeros(Int,m-length(x))]
end
And main function looks like:
function lpartitions(n,m)
result=[]
for i in partitions(n)
append!(result,lpremutations(resize_!(i, m)))
end
result
end
Check it
julia> lpartitions(3,4)
20-element Array{Any,1}:
[0,0,0,3]
[0,0,3,0]
[0,3,0,0]
[3,0,0,0]
[0,0,1,2]
[0,0,2,1]
[0,1,0,2]
[0,1,2,0]
[0,2,0,1]
[0,2,1,0]
[1,0,0,2]
[1,0,2,0]
[1,2,0,0]
[2,0,0,1]
[2,0,1,0]
[2,1,0,0]
[0,1,1,1]
[1,0,1,1]
[1,1,0,1]
[1,1,1,0]
The MATLAB script from http://www.mathworks.com/matlabcentral/fileexchange/28340-nsumk actually behaves the way I need, and is what I though that partitions() would do from the description given. The Julia version is
# k - sum, n - number of non-negative integers
function nsumk(k,n)
m = binomial(k+n-1,n-1);
d1 = zeros(Int16,m,1);
d2 = collect(combinations(collect((1:(k+n-1))),n-1));
d2 = convert(Array{Int16,2},hcat(d2...)');
d3 = ones(Int16,m,1)*(k+n);
dividers = [d1 d2 d3];
return diff(dividers,2)-1;
end
julia> nsumk(3,2)
4x2 Array{Int16,2}:
0 3
1 2
2 1
3 0
using daycaster's lovely hcat(x...) tidbit :)
I still wish there would be a more compact way of doing this.
The the first mention of this approach seem to be https://au.mathworks.com/matlabcentral/newsreader/view_thread/52610, and as far as I can understand it is based on the "stars and bars" method https://en.wikipedia.org/wiki/Stars_and_bars_(combinatorics)

Resources