I am trying to implement parallel processing for nested loops. However I am getting the following syntax error.
I am modifying the example from here ( Parallel computing in Julia - running a simple for-loop on multiple cores )
This works
for N in 1:5:20, H in 1:5:20
println("The N of this iteration in $N, $H")
end
This is giving syntax error
using Distributed
#distributed for N in 1:5:20, H in 1:5:20
println("The N of this iteration in $N, $H")
end
The #distributed macro supports iterating only over one parameter. Hence you could do:
#distributed for (N,H) in collect(Iterators.product(1:5:20,1:5:20))
println("The N of this iteration in $N, $H")
end
Another option is of course having a nested loop. In that case only one of the loops should be #distributed.
An alternative (for single node, multi core parallelization) would be multithreading. This is often easier / more efficient due to lower overhead and shared memory (note that there is no such thing like a Python GIL in Julia).
The corresponding Julia commands are Threads.#threads (in front of for loops) or Threads.#spawn for spawning tasks to be executed parallel.
Edit: example added
Threads.#threads for (N,H) in collect(Iterators.product(1:5:20,1:5:20))
println("The N of this iteration in $N, $H")
end
2nd case:
f(x) = x^2
futures = [Threads.#spawn f(x) for x=1:10]
fetch.(futures)
Related
I have the following code:
mutable struct T
a::Vector{Int}
end
N = 100
len = 10
Ts = [T(fill(i, len)) for i in 1:N]
for i = 1:N
destructive_function!(Ts[i])
end
destructive_function! changes a in each element of Ts.
Is there any way to parallelize the for loop? SharedArray does not seem available for user-defined types.
In this case it would be simpler to use threads rather than processes to parallelize the loop
Threads.#threads for i = 1:N
destructive_function!(Ts[i])
end
Just make sure that you set JULIA_NUM_THREADS environment variable before starting Julia. You can check number of threads running with Threads.nthreads() function.
If you want to use processes rather than threads then probably it is simplest to make destructive_function! return the modified value and then run pmap(destructive_function!, Ts) to collect the modified values in a new array (but this would not modify the Ts in place).
In my Julia 0.5 script I use srand(1234) to get the same results from rand() each time I re-run the script. However, I get different results. What do I wrong?
As #Dan Getz mentioned in the comments, this is likely to because you have some code that calls random functions without you knowing about it.
If you call the same rand() function with the same seed set, you get the same results as expected:
julia> for i in 1:3
srand(1)
println(rand())
end
0.23603334566204692
0.23603334566204692
0.23603334566204692
However, if you have another call in your script to rand that may or may not be called, then your random number generator will be at different stages when you get to the investigated rand() call. Here's an example to illustrate this:
julia> for i in 1:3
srand(1)
if i == 2
rand()
end
println(rand())
end
0.23603334566204692
0.34651701419196046
0.23603334566204692
Notice how in the second iteration of the loop there's an extra rand() call that offsets the random number generator and results in a different value.
In addition to the answer given by #niczky12 I would recommend that you define your own generator and use that for better reproducibility, that way you always keep control of "your" generator, and calls to other functions (perhaps not in your control) that uses the global one will not affect the random numbers you obtain.
For example, creating a MersenneTwister with seed 1234:
rng = MersenneTwister(1234)
Then you simply pass this generator to your rand calls:
julia> rng = MersenneTwister(1234);
julia> rand(rng)
0.5908446386657102
julia> rand(rng, 2, 3)
2×3 Array{Float64,2}:
0.766797 0.460085 0.854147
0.566237 0.794026 0.200586
When I use a for loop, I typically have if constructs with next and break statements. Solving some problems and logical steps just requires that. However I am unable to use next, break statements in the foreach package. How can I use these statements inside of foreach looping structure?
The general idea when using the foreach package is that every iteration can be performed in parallel; so if you had N iteration and N CPUs you would get (ignoring thread communication) perfect speed-up.
So instead of using break, return an NA or 0 as early as possible. For example
library("foreach")
f = function(i) if(i < 3) sqrt(i) else NA
foreach(i=1:5) %do% f(i)
Now you could argue that you have wasted resources for i=4 and i=5, but this amounts to nano/microseconds and your total computation is measured in seconds/minutes.
Given the function to generate fibonacci numbers:
def fibonacci(x, memory = {0:0, 1:1}):
if x not in memory:
memory[x] = fibonacci(x-1, memory)+fibonacci(x-2, memory)
return memory[x]
When I try some arbitrarily large number, say 4000th fibonacci number, I get an error:
RuntimeError: maximum recursion depth exceeded
What is the error caused by? What can be done as a work around without sacrificing the efficiency? I prefer this to iteration, since by iterating, calculating even the 50th position takes astronomically long for a computer (read: half a minute).
As the others have mentioned you have hit the stack memory limit. You usually have a maximum of 50-100 maximum nested recursive calls before you hit this limit.
I think you might have a misconception about iteration (unrolling recursive functions)
def fib():
a=0
b=1;
c=1;
for x in range(4000):
c = a + b;
a = b;
b = c;
print c;
fib();
There is no way this function would take longer than your recursive one.
Iteration is lot better than recursion in my opinion. Because
Iteration is lot more easily readable and traceable than recursion.
You can use concepts such as Generators to reduce the memory load that may not be feasible with recursive functions.
As for what is Recursion Limit, you can see here. It gives a basic explanation of what and why recursion limit is there. As shown there, you can give sys.setrecursionlimit to any value you desired.
Using iterator, this can be used for fibonacci series:
def fib(a, b):
return b, a+b
for i in range(20):
if i in (0, 1):
print i
else:
print a
a, b = fib(a, b)
I think you hit the recursion limit. Here is a link to a similar post, which might help. I would recommend that you use iteration, but I also came across this code to increase the stack size, but have not tested it:
import sys
sys.setrecursionlimit(10000) # change the '10000' to something that works
What is the difference? Are these the same? If not, can someone please give me an example?
MW:
Iteration - 1 : the action or a process of iterating or repeating: as a : a procedure in which repetition of a sequence of operations yields results successively closer to a desired result b : the repetition of a sequence of computer instructions a specified number of times or until a condition is met
Recursion - 3 : a computer programming technique involving the use of a procedure, subroutine, function, or algorithm that calls itself one or more times until a specified condition is met at which time the rest of each repetition is processed from the last one called to the first
We can distinguish (as is done in SICP) recursive and iterative procedures from recursive and iterative processes. The former are as your definition describes, where recursion is basically the same as mathematical recursion: a recursive procedure is defined in terms of itself. An iterative procedure repeats a block of code with a loop statement. A recursive process, however, is one that takes non-constant (e.g. O(n) or O(lg(n)) space) to execute, while an iterative process takes O(1) (constant) space.
For mathematical examples, the Fibonacci numbers are defined recursively:
Sigma notation is analogous to iteration:
as is Pi notation. Similar to how some (mathematical) recursive formulae can be rewritten as iterative ones, some (but not all) recursive processes have iterative equivalents. All recursive procedures can be transformed into iterative ones by keeping track of partial results in your own data structure, rather than using the function call stack.
[Hurry and trump this!]
One form can be converted to the other, with one notable restriction: many "popular" languages (C/Java/normal Python) do not support TCO/TCE (tail-call-optimization/tail-call-elimination) and thus using recursion will 'add extra data to the stack' each time a method calls itself recursively.
So in C and Java, iteration is idiomatic, whereas in Scheme or Haskell, recursion is idiomatic.
As per the definitions you mentioned, these 2 are very different. In iteration, there is no self-calling, but in recursion, a function calls itself
For example. Iterative algorithm for factorial calculation
fact=1
For count=1 to n
fact=fact*count
end for
And the recursive version
function factorial(n)
if (n==1) return 1
else
n=n*factorial(n-1)
end if
end function
Generally
Recursive code is more succinct but uses a larger amount of memory. Sometimes recursion can be converted to iterations using dynamic programming.
Here's a Lisp function for finding the length of a list. It is recursive:
(defun recursive-list-length (L)
"A recursive implementation of list-length."
(if (null L)
0
(1+ (recursive-list-length (rest L)))))
It reads "the length of a list is either 0 if that list is empty, or 1 plus the length of the sub-list starting with the second element).
And this is an implementation of strlen - the C function finding the length of a nul-terminated char* string. It is iterative:
size_t strlen(const char *s)
{
size_t n;
n = 0;
while (*s++)
n++;
return(n);
}
You goal is to repeat some operation. Using iteration, you employ an explicit loop (like the while loop in the strlen code). Using recursion, your function calls itself with (usually) a smaller argument, and so on until a boundary condition (null L in the code above) is met. This also repeats the operation, but without an explicit loop.
Recursion:
Eg: Take fibonacci series for example. to get any fibonacci number we have to know the previous one. So u will berform the operation (same one) on every number lesser than the given and each of this inturn calling the same method.
fib(5) = Fib (4) + 5
fib(4) = Fib (3) + 4
.
.
i.e reusing the method fib
Iteration is looping like you add 1+1+1+1+1(iteratively adding) to get 5 or 3*3*3*3*3 (iteratively multiplying)to get 3^5.
For a good example of the above, consider recursive v. iterative procedures for depth-first search. It can be done using language features via recursive function calls, or in an iterative loop using a stack, but the process is inherently recursive.
For difference between recursive vs non-recursive;
recursive implementations are a bit easier to verify for correctness; non-
recursive implementations are a bit more efficient.
Algorithms (4th Edition)