Write a variable number of arguments for IF - julia

I am trying to write a function that would solve any general version of this problem:
If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.
Find the sum of all the multiples of 3 or 5 below 1000.
The way I solved it for this instance was:
multiples = Array(Int, 0)
[ (i % 3 == 0 || i % 5 == 0) && push!(multiples, i) for i in 1:1000 ]
sum(multiples)
I want to write a function that will take an array of multiples (in this case, [3,5]) and the final number (in this case, 1000). The point is that the array can consist of arbitrarily many numbers, not just two (e.g., [3,5,6]). Then the function should run i % N == 0 for each N.
How do I do that most efficiently? Could that involve metaprogramming? (The code doesn't have to be in a list comprehension format.)
Thanks!

Very first thing that popped into my head was the following, using modulo division and a functional style:
v1(N,M) = sum(filter(k->any(j->k%j==0,M), 1:N))
But I wanted to investigate some alternatives, as this has two problems:
Two levels of anonymous function, something Julia doesn't optimize well (yet).
Creates a temporary array from the range, then sums.
So here is the most obvious alternative, the C-style version of the one-liner:
function v2(N,M)
sum_so_far = 0
for k in 1:N
for j in M
if k%j==0
sum_so_far += k
break
end
end
end
return sum_so_far
end
But then I thought about it some more, and remembered reading somewhere that modulo division is a slow operation. I wanted to see how IntSets perform - a set specialized for integers. So here is another one-liner, IntSets without using any module division, and a functional style!
v3(N,M) = sum(union(map(j->IntSet(j:j:N), M)...))
Expanding the map into a for loop and repeatedly applying union! to a single IntSet was not much better, so I won't include that here. To break this down:
IntSet(j:j:N) is all the multiples of j between j and N
j->IntSet(j:j:N) is an anonymous function that returns that IntSet
map(j->IntSet(j:j:N), M) applies that function to each j in M, and returns a Vector{IntSet}.
The ... "splats" the vector out into arguments of union
union creates an IntSet that is the union of its arguments - in this case, all multiples of the numbers in M
Then we sum it to finish
I benchmarked these with
N,M = 10000000, [3,4,5]
which gives you
One-liner: 2.857292874 seconds (826006352 bytes allocated, 10.49% gc time)
C-style: 0.190581908 seconds (176 bytes allocated)
IntSet no modulo: 0.121820101 seconds (16781040 bytes allocated)
So you can definitely even beat C-style code with higher level objects - modulo is that expensive I guess! The neat thing about the no modulo one is it parallelizes quite easily:
addprocs(3)
#everywhere worker(j,N) = IntSet(j:j:N)
v4(N,M) = sum(union(pmap(j->worker(j,N),M)...))
#show v4(1000, [3,5])
#time v3(1000000000,[3,4,5]); # bigger N than before
#time v4(1000000000,[3,4,5]);
which gives
elapsed time: 12.279323079 seconds (2147831540 bytes allocated, 0.94% gc time)
elapsed time: 10.322364457 seconds (1019935752 bytes allocated, 0.71% gc time)
which isn't much better, but its something I suppose.

Okay, here is my updated answer.
Based on the benchmarks in the answer of #IainDunning, the method to beat is his v2. My approach below appears to be much faster, but I'm not clever enough to generalize it to input vectors of length greater than 2. A good mathematician should be able to improve on my answer.
Quick intuition: For the case of length(M)=2, the problem reduces to the sum of all multiples of M[1] up to N added to the sum of all multiples of M[2] up to N, where, to avoid double-counting, we then need to subtract the sum of all multiples of M[1]*M[2] up to N. A similar algorithm could be implemented for M > 2, but the double-counting issue becomes much more complicated very quickly. I suspect a general algorithm for this would definitely exist (it is the kind of issue that crops up all the time in the field of combinatorics) but I don't know it off the top of my head.
Here is the test code for my approach (f1) versus v2:
function f1(N, M)
if length(M) > 2
error("I'm not clever enough for this case")
end
runningSum = 0
for c = 1:length(M)
runningSum += sum(M[c]:M[c]:N)
end
for c1 = 1:length(M)
for c2 = c1+1:length(M)
temp1 = M[c1]*M[c2]
runningSum -= sum(temp1:temp1:N)
end
end
return(runningSum)
end
function v2(N, M)
sum_so_far = 0
for k in 1:N
for j in M
if k%j==0
sum_so_far += k
break
end
end
end
return sum_so_far
end
f1(1000, [3,5])
v2(1000, [3,5])
N = 10000000
M = [3,5]
#time f1(N, M)
#time v2(N, M)
Timings are:
elapsed time: 4.744e-6 seconds (96 bytes allocated)
elapsed time: 0.201480996 seconds (96 bytes allocated)
Sorry, this is an interesting problem but I'm afraid I have to get back to work :-) I'll check back in later if I get a chance...

Related

Schroders Big number sequence

I am implementing a recursive program to calculate the certain values in the Schroder sequence, and I'm having two problems:
I need to calculate the number of calls in the program;
Past a certain number, the program will generate incorrect values (I think it's because the number is too big);
Here is the code:
let rec schroder n =
if n <= 0 then 1
else if n = 1 then 2
else 3 * schroder (n-1) + sum n 1
and sum n k =
if (k > n-2) then 0
else schroder k * schroder (n-k-1) + sum n (k+1)
When I try to return tuples (1.), the function sum stops working because it's trying to return int when it has type int * int;
Regarding 2., when I do schroder 15 it returns:
-357364258
when it should be returning
3937603038.
EDIT:
firstly thanks for the tips, secondly after some hours of deep struggle, i manage to create the function, now my problem is that i'm struggling to install zarith. I think I got it installed, but ..
in terminal when i do ocamlc -I +zarith test.ml i get an error saying Required module 'Z' is unavailable.
in utop after doing #load "zarith.cma";; and #install_printer Z.pp_print;; i can compile, run the function and it works. However i'm trying to implement a Scanf.scanf so that i can print different values of the sequence. With this being said whenever i try to run the scanf, i dont get a chance to write any number as i get a message saying that '\\n' is not a decimal digit.
With this being said i will most probably also have problems with printing the value, because i dont think that i'm going to be able to print such a big number with a %d. The let r1,c1 = in the following code, is a example of what i'm talking about.
Here's what i'm using :
(function)
..
let v1, v2 = Scanf.scanf "%d %d" (fun v1 v2-> v1,v2);;
let r1,c1 = schroder_a (Big_int_Z.of_int v1) in
Printf.printf "%d %d\n" (Big_int_Z.int_of_big_int r1) (Big_int_Z.int_of_big_int c1);
let r2,c2 = schroder_a v2 in
Printf.printf "%d %d\n" r2 c2;
P.S. 'r1' & 'r2' stands for result, and 'c1' and 'c2' stands for the number of calls of schroder's recursive function.
P.S.S. the prints are written differently because i was just testing, but i cant even pass through the scanf so..
This is the third time I've seen this problem here on StackOverflow, so I assume it's some kind of school assignment. As such, I'm just going to make some comments.
OCaml doesn't have a function named sum built in. If it's a function you've written yourself, the obvious suggestion would be to rewrite it so that it knows how to add up the tuples that you want to return. That would be one approach, at any rate.
It's true, ints in OCaml are subject to overflow. If you want to calculate larger values you need to use a "big number" package. The one to use with a modern OCaml is Zarith (I have linked to the description on ocaml.org).
However, none of the other people solving this assignment have mentioned overflow as a problem. It could be that you're OK if you just solve for representable OCaml int values.
3937603038 is larger than what a 32-bit int can hold, and will therefore overflow. You can fix this by using int64 instead (until you overflow that too). You'll have to use int64 literals, using the L suffix, and operations from the Int64 module. Here's your code converted to compute the value as an int64:
let rec schroder n =
if n <= 0 then 1L
else if n = 1 then 2L
else Int64.add (Int64.mul 3L (schroder (n-1))) (sum n 1)
and sum n k =
if (k > n-2) then 0L
else Int64.add (Int64.mul (schroder k) (schroder (n-k-1))) (sum n (k+1))
I need to calculate the number of calls in the program;
...
the function 'sum' stops working because it's trying to return 'int' when it has type 'int * int'
Make sure that you have updated all the recursive calls to shroder. Remember it is now returning a pair not a number, so you can't, for example, just to add it and you need to unpack the pair first. E.g.,
...
else
let r,i = schroder (n-1) (i+1) in
3 * r + sum n 1 and ...
and so on.
Past a certain number, the program will generate incorrect values (I think it's because the number is too big);
You need to use an arbitrary-precision numbers, e.g., zarith

Translating recursion to divide and conquer

I'm trying to return the three smallest items in a list. Below is an O(n) solution I've written:
def three_smallest(L):
# base case
if (len(L) == 3):
return sorted(L)
current = L[0]
(first_smallest,second_smallest,third_smallest) = three_smallest(L[1:])
if (current < first_smallest):
return (current, first_smallest, second_smallest)
elif (current < second_smallest):
return (first_smallest, current, second_smallest)
elif (current < third_smallest):
return (first_smallest, second_smallest, current)
else:
return (first_smallest,second_smallest,third_smallest)
Now I'm trying to write a divide and conquer approach but I'm not sure how I should divide the list. Any help would be appreciated.
Note that this solution (to my understanding) is NOT divide and conquer. It is just a basic recursive solution as divide and conquer involves dividing a list of length n by integer b, and calling the algorithm on those parts.
For divide and conquer you generally want to divide a set (roughly) in half rather than whittle it away one by one. Perhaps the following would meet your needs?
def three_smallest(L):
if (len(L) <= 3): # base case, technically up to 3 (can be fewer)
return sorted(L)
mid = len(L) // 2 # find the midpoint of L
# find (up to) the 3 smallest in first half, ditto for the second half, pool
# them to a list with no more than 6 items, sort, and return the 3 smallest
return sorted(three_smallest(L[:mid]) + three_smallest(L[mid:]))[:3]
Note that due to the inequality in the base case, this implementation does not have a minimum list size requirement.
At the other end of the scale, your original implementation is limited to lists with fewer than a thousand values or it will blow out the recursion stack. The implementation given above was able to handle a list of a million values with no problem.

Julia push! - is not the right command while trying to collect data

I am struggling with Julia every time that I need to collect data in an array "outside" functions.
If I use push!(element, array) I can collect data in the array, but if the code is inside a loop then the array "grows" each time.
What do you recommend?
I know is quite basic :) but thank you!
I assume the reason why you do not want to use push! is because you have used Matlab before where this sort of operation is painfully slow (to be precise, it is an O(n^2) operation, so doubling n quadruples the runtime). This is not true for Julia's push!, since push! uses the algorithm described here which is only O(n) (so doubling n only doubles the runtime).
You can easily check this experimentally. In Matlab we have
>> n = 100000; tic; a = []; for i = 1:n; a = [a;0]; end; toc
Elapsed time is 2.206152 seconds.
>> n = 200000; tic; a = []; for i = 1:n; a = [a;0]; end; toc
Elapsed time is 8.301130 seconds.
so indeed the runtime quadruples for an n of twice the size. In contrast, in Julia we have
julia> using BenchmarkTools
function myzeros(n)
a = Vector{Int}()
for i = 1:n
push!(a,0)
end
return a
end
#btime myzeros(100_000);
#btime myzeros(200_000);
486.054 μs (17 allocations: 2.00 MiB)
953.982 μs (18 allocations: 3.00 MiB)
so indeed the runtimes only doubles for an n of twice the size.
Long story short: if you know the size of the final array, then preallocating the array is always best (even in Julia). However, if you don't know the final array size, then you can use push! without losing too much performance.

Counting the number of restricted Integer partitions

Original problem:
Let N be a positive integer (actually, N <= 2000) and P - set of all possible partitions of the N, where with and . Let A be the number of partitions . Find the A.
Input: N. Output: A - the number of partitions .
What have I tried:
I think that this problem can be solved by dynamic-based algorithm. Let p(n,a,b) be the function, which returns the number of partitons of n using only numbers a. . .b. Then we can compute the A with the code like:
int Ans = 2; // the 1+1+...+1=N & N=N partitions
for(int a = 2; a <= N/2; a += 1){ //a - from 2 to N/2
int b = a*2-1;
Ans += p[N][a][b]; // add all partitions using a..b to Answer
if(a < (a-1)*2-1){ // if a < previous b [ (a-1)*2-1 ]
Ans -= p[N][a][(a-1)*2-1]; // then we counted number of partitions
} // using numbers a..prev_b twice.
}
Next I tried to find the dynamic algorithm computing p(n,a,b) for any integer a <= b <= n. This paper (.pdf) provides the folowing algorithm:
, were I(n<=b) = 1 if n<=b and =0 otherwise.
Question(s):
How should I realize the algorithm from the paper? I'm new at d-p problems and as I can see, this problem has 3 dimensions (n,a & b), which is quite tricky for me.
How actually that algorithm works? I know how work the algorithms for computing p(n,0,b) or p(n,a,n), but a little explanation for p(n,a,b) will be very helpful.
Does original problem have simpler solution? I'm quite sure that there's another clean solution, but I didn't found it.
I calculated all A(1)-A(600) in 23 seconds with memoization approach (top-down dynamic programming). 3D table requires 1.7 GB of memory.
For reference: A[50] = 278, A(200)=465202, A(600)=38860513616
N=2000 requires too large table for 32-bit environment, and map approach worked too slow.
I can make 2D table with reasonable size, but this approach requires table zeroing at every iteration of external loop - slow again.
A(1000) = 107292471486730 in 131 sec. And I think that long arithmetic might be needed for larger values to avoid Int64 overflow.

Dynamic Programming, Arithmetic Progression, Time complexity issues

Given an array of size n (1<=n<=200000) with each entry a[i] ( 1<=a[i]<=100), find all the number of subsequences that form Arithmetic progression.
Subsequences are the sequences where you can leave any number of elements in the original sequence.
For example, the sequence A,B,D is a subsequence of A,B,C,D,E,F obtained after removal of elements C, E and F. The relation of one sequence being the subsequence of another is a preorder.
I have written O(n^2) solution using DP. But n^2 = 10^10. So, It'll not get accepted.
Here is what I did.
Pseudocode:
for every element A[i]:
for every element A[k] such that k<i:
diff = A[i] - A[k] + 100: (adding 100, -ve differences A.P.)
dp[i][diff] += dp[i-1][diff] + 1;
for every element A[i]:
for every diff, d:
ans = ans + dp[i][d];
return ans;
This is giving correct output but TLE for 3 big cases.
P.S. Please suggest better solution..!!
Is divide and conquer optimization DP required here?? If yes, tell me how to build the solution.

Resources