So I have this function that I'm trying to convert from a recursive algorithm to an iterative algorithm. I'm not even sure if I have the right subproblems but this seems to determined what I need in the correct way, but recursion can't be used you need to use dynamic programming so I need to change it to iterative bottom up or top down dynamic programming.
The basic recursive function looks like this:
Recursion(i,j) {
if(i > j) {
return 0;
}
else {
// This finds the maximum value for all possible
// subproblems and returns that for this problem
for(int x = i; x < j; x++) {
if(some subsection i to x plus recursion(x+1,j) is > current max) {
max = some subsection i to x plus recursion(x+1,j)
}
}
}
}
This is the general idea, but since recursions typically don't have for loops in them I'm not sure exactly how I would convert this to iterative. Does anyone have any ideas?
You have a recursive function that can be summarised as this:
recursive(i, j):
if stopping condition:
return value
loop:
if test current value involving recursive call passes:
set value based on recursive call
return value # this appears to be missing from your example
(I am going to be pretty loose with the pseudo code here, to emphasize the structure of the code rather than the specific implementation)
And you want to flatten it to a purely iterative approach. First it would be good to describe exactly what this involves in the general case, as you seem to be interested in that. Then we can move on to flattening the pseudo code above.
Now flattening a primitive recursive function is quite straightforward. When you are given code that is like:
simple(i):
if i has reached the limit: # stopping condition
return value
# body of method here
return simple(i + 1) # recursive call
You can quickly see that the recursive calls will continue until i reaches the predefined limit. When this happens the value will be returned. The iterative form of this is:
simple_iterative(start):
for (i = start; i < limit; i++):
# body here
return value
This works because the recursive calls form the following call tree:
simple(1)
-> simple(2)
-> simple(3)
...
-> simple(N):
return value
I would describe that call tree as a piece of string. It has a beginning, a middle, and an end. The different calls occur at different points on the string.
A string of calls like that is very like a for loop - all of the work done by the function is passed to the next invocation and the final result of the recursion is just passed back. The for loop version just takes the values that would be passed into the different calls and runs the body code on them.
Simple so far!
Now your method is more complex in two ways:
There are multiple separate statements that make recursive calls
Those statements themselves are within a for loop
So your call tree is something like:
recursive(i, j):
for (v in 1, 2, ... N):
-> first_recursive_call(i + v, j):
-> ... inner calls ...
-> potential second recursive call(i + v, j):
-> ... inner calls ...
As you can see this is not at all like a string. Instead it really is like a tree (or a bush) in that each call results in two more calls. At this point it is actually very hard to turn this back into an entirely iterative function.
This is because of the fundamental relationship between loops and recursion. Any loop can be restated as a recursive call. However not all recursive calls can be transformed into loops.
The class of recursive calls that can be transformed into loops are called primitive recursion. Your function initially appears to have transcended that. If this is the case then you will not be able to transform it into a purely iterative function (short of actually implementing a call stack and similar within your function).
This video explains the difference between primitive recursion and fundamentally recursive types that follow:
https://www.youtube.com/watch?v=i7sm9dzFtEI
I would add that your condition and the value that you assign to max appear to be the same. If this is the case then you can remove one of the recursive calls, allowing your function to become an instance of primitive recursion wrapped in a loop. If you did so then you might be able to flatten it.
well unless there is an issue with the logic not included yet, it should be fine
for & while are ok in recursion
just make sure you return in every case that may occur
Related
I've been recently learning about functional languages and how many don't include for loops. While I don't personally view recursion as more difficult than a for loop (and often easier to reason out) I realized that many examples of recursion aren't tail recursive and therefor cannot use simple tail recursion optimization in order to avoid stack overflows. According to this question, all iterative loops can be translated into recursion, and those iterative loops can be transformed into tail recursion, so it confuses me when the answers on a question like this suggest that you have to explicitly manage the translation of your recursion into tail recursion yourself if you want to avoid stack overflows. It seems like it should be possible for a compiler to do all the translation from either recursion to tail recursion, or from recursion straight to an iterative loop with out stack overflows.
Are functional compilers able to avoid stack overflows in more general recursive cases? Are you really forced to transform your recursive code in order to avoid stack overflows yourself? If they aren't able to perform general recursive stack-safe compilation, why aren't they?
Any recursive function can be converted into a tail recursive one.
For instance, consider the transition function of a Turing machine, that
is the mapping from a configuration to the next one. To simulate the
turing machine you just need to iterate the transition function until
you reach a final state, that is easily expressed in tail recursive
form. Similarly, a compiler typically translates a recursive program into
an iterative one simply adding a stack of activation records.
You can also give a translation into tail recursive form using continuation
passing style (CPS). To make a classical example, consider the fibonacci
function.
This can be expressed in CPS style in the following way, where the second
parameter is the continuation (essentially, a callback function):
def fibc(n, cont):
if n <= 1:
return cont(n)
return fibc(n - 1, lambda a: fibc(n - 2, lambda b: cont(a + b)))
Again, you are simulating the recursion stack using a dynamic data structure:
in this case, lambda abstractions.
The use of dynamic structures (lists, stacks, functions, etc.) in all previous
examples is essential. That is to say, that in order to simulate a generic
recursive function iteratively, you cannot avoid dynamic memory allocation,
and hence you cannot avoid stack overflow, in general.
So, memory consumption is not only related to the iterative/recursive
nature of the program. On the other side, if you prevent dynamic memory
allocation, your
programs are essentially finite state machines, with limited computational
capabilities (more interesting would be to parametrise memory according to
the dimension of inputs).
In general, in the same way as you cannot predict termination, you cannot
predict an unbound memory consumption of your program: working with
a Turing complete language, at compile time
you cannot avoid divergence, and you cannot avoid stack overflow.
Tail Call Optimization:
The natural way to do arguments and calls is to sort out the cleaning up when exiting or when returning.
For tail calls to work you need to alter it so that the tail call inherits the current frame. Thus instead of making a new frame it massages the frame so that the next call returns to the current functions caller instead of this function, which really only cleans up and returns if it's a tail call.
Thus TCO is all about cleaning up before the last call.
Continuation Passing Style - make tail calls out of everything
A compiler can change the code such that it only does primitive operations and pass it to continuations. Thus the stack usage gets moved onto the heap since the computation to be continued is made a function.
An example is:
function hypotenuse(k1, k2) {
return sqrt(add(square(k1), square(k2)))
}
becomes
function hypotenuse(k, k1, k2) {
(function (sk1) {
(function (sk2) {
(function (ar) {
k(sqrt(ar));
}(add(sk1,sk2));
}(square(k2));
}(square(k1));
}
Notice every function has exactly one call now and the order of evaluation is set.
According to this question, all iterative loops can be translated into recursion
"Translated" might be a bit of a stretch. The proof that for every iterative loop there is an equivalent recursive program is trivial if you understand Turing completeness: since a Turing machine can be implemented using strictly iterative structures and strictly recursive structures, every program that can be expressed in an iterative language can be expressed in a recursive language, and vice-versa. This means that for every iterative loop there is an equivalent recursive construct (and the other way around). However, that doesn't mean we have some automated way of transforming one into the other.
and those iterative loops can be transformed into tail recursion
Tail recursion can perhaps be easily transformed into an iterative loop, and the other way around. But not all recursion is tail recursion. Here's an example. Suppose we have some binary tree. It consists of nodes. Each node can have a left and a right child and a value. If a node has no children, then isLeaf returns true for it. We'll assume there's some function max that returns the maximum of two values, and if one of the values is null it returns the other one. Now we want to define a function that finds the maximum value among all the leaf nodes. Here it is in some pseudo-code I cooked up.
findmax(node) {
if (node == null) {
return null
}
if (node.isLeaf) {
return node.value
} else {
return max(findmax(node.left), findmax(node.right))
}
}
There's two recursive calls in the max function, so we can't optimize for tail recursion. We need the results of both before we can supply them to the max function and determine the result of the call for the current node.
Now, there may be a way of getting the same result, using recursion and only a single tail-recursive call. It is functionally equivalent, but it is a different algorithm. Compilers can do a lot of transformations to create a functionally equivalent program with lots of optimizations, but they're not quite clever enough to create functionally equivalent algorithms.
Even the transformation of a function that only calls itself recursively once into a tail-recursive version would be far from trivial. Such an adaptation usually employs some argument passed into the recursive invocation that is used as an "accumulator" for the current results.
Look at the next naive implementation for calculating a factorial of a number (e.g. fact(5) = 5*4*3*2*1):
fact(number) {
if (number == 1) {
return 1
} else {
return number * fact(number - 1)
}
}
It's not tail-recursive. But it can be made so in this way:
fact(number, acc) {
if (number == 1) {
return acc
} else {
return fact(number - 1, number * acc)
}
}
// Helper function
fact(number) {
return fact(number, 1)
}
This requires an interpretation of what is being done. Recognizing the case for stuff like this is easy enough, but what if you call a function instead of a multiplication? How will the compiler know that for the initial call the accumulator must be 1 and not, say, 0? How do you translate this program?
recsub(number) {
if (number == 1) {
return 1
} else {
return number - recsub(number - 1)
}
}
This is as of yet outside the scope of the sort of compiler we have now, and may in fact always be.
Maybe it would be interesting to ask this on the computer science Stack Exchange to see if they know of some papers or proofs that investigate this more in-depth.
I am new to the site and am not familiar with how and where to post so please excuse me. I am currently studying recursion and am having trouble understanding the output of this program. Below is the method body.
public static int Asterisk(int n)
{
if (n<1)
return;
Asterisk(n-1);
for (int i = 0; i<n; i++)
{
System.out.print("*");
}
System.out.println();
}
This is the output
*
**
***
****
*****
it is due to the fact that the "Asterisk(n-1)" lies before the for loop.
I would think that the output should be
****
***
**
*
This is the way head recursion works. The call to the function is made before execution of other statements. So, Asterisk(5) calls Asterisk(4) before doing anything else. This further cascades into serial function calls from Asterisk(3) → Asterisk(2) → Asterisk(1) → Asterisk(0).
Now, Asterisk(0) simply returns as it passes the condition n<1. The control goes back to Asterisk(1) which now executes the rest of its code by printing n=1 stars. Then it relinquishes control to Asterisk(2) which again prints n=2 stars, and so on. Finally, Asterisk(5) prints its n=5 stars and the function calls end. This is why you see the pattern of ascending number of stars.
There are two ways to create programming loops. One is using imperative loops normally native to the language (for, while, etc) and the other is using functions (functional loops). In your example the two kinds of loops are presented.
One loop is the unrolling of the function
Asterisk(int n)
This unrolling uses recursion, where the function calls itself. Every functional loop must know when to stop, otherwise it goes on forever and blows up the stack. This is called the "stopping condition". In your case it is :
if (n<1)
return;
There is bidirectional equivalence between functional loops and imperative loops (for, while, etc). You can turn any functional loop into a regular loop and vice versa.
IMO this particular exercise was meant to show you the two different ways to build loops. The outer loop is functional (you could substitute it for a for loop) and the inner loop is imperative.
Think of recursive calls in terms of a stack. A stack is a data structure which adds to the top of a pile. A real world analogy is a pile of dishes where the newest dish goes on the top. Therefore recursive calls add another layer to the top of the stack, then once some criteria is met which prevents further recursive calls, the stack starts to unwind and we work our way back down to the original item (the first plate in pile of dishes).
The input of a recursive method tends towards a base case which is the termination factor and prevents the method from calling itself indefinitely (infinite loop). Once this base condition is met the method returns a value rather than calling itself again. This is how the stack in unwound.
In your method, the base case is when $n<1$ and the recursive calls use the input $n-1$. This means the method will call itself, each time decreasing $n$ by 1, until $n<1$ i.e. $n=0$. Once the base condition is met, the value 0 is returned and we start to execute the $for$ loop. This is why the first line contains a single asterix.
So if you run the method with an input of 5, the recursive calls build a stack of values of $n$ as so
0
1
2
3
4
5
Then this stack is unwound starting with the top, 0, all the way down to 5.
Given code that looks something like this (pseudocode):
f(args)
result = simple_case
if (base_case(args))
return result
new_args = args
for each x in args list
remove x from new_args
if (need_to_update_result_based_on_x(result,x))
result = f(new_args)
return result
I already know how to remove tail-recursion from a function, but I am unsure if there are straightforward techniques for removing recursion that itself is inside of a loop, as above.
In particular, I am especially wondering if it is it possible to rewrite it so that it is purely iterative? By which I mean that it does not even need to effectively emulate the recursive design by simply implementing it's own stack? If not, what would generally be the most economical (in terms of storage and time) way to rewrite such a function?
The args list is being reduced by each iteration for the for loop in the function, and the reduced list is also passed to the next recursive call. So, when the recursive call returns, the loop picks up with the list that it had when it made the recursive call, but the recursive call itself has already worked out results for the reduced versions of that list.
To avoid redoing work already done, you can cache the results of the previous recursive calls so that future recursive calls need not redo the complete computation. This technique is often referred to as memoization.
So, to remove the recursion altogether, the base case results can be computed upfront and remembered (memoized). These memoizations can be used to compute results for the base case plus one of the other arguments in the list, and these results also memoized (at this point, the base case results can be tossed). You can repeatedly compute memoizations for each successive larger list size until you achieve a result for the desired argument list.
I have read a lot about continuations and a very common definition I saw is, it returns the control state.
I am taking a functional programming course taught in SML.
Our professor defined continuations to be:
"What keeps track of what we still have to do"
; "Gives us control of the call stack"
A lot of his examples revolve around trees. Before this chapter, we did tail recursion. I understand that tail recursion lets go of the stack to hold the recursively called functions by having an additional argument to "build" up the answer. Reversing a list would be built in a new accumulator where we append to it accordingly. Also, he said something about functions are called(but not evaluated) except till we reach the end where we replace backwards. He said an improved version of tail recursion would be using CPS(Continuation Programming Style).
Could someone give a simplified explanation of what continuations are and why they are favoured over other programming styles?
I found this stackoverflow link that helped me, but still did not clarify the idea for me:
I just don't get continuations!
Continuations simply treat "what happens next" as first class objects that can be used once unconditionally, ignored in favour of something else, or used multiple times.
To address what Continuation Passing Style is, here is some expression written normally:
let h x = f (g x)
g is applied to x and f is applied to the result.
Notice that g does not have any control. Its result will be passed to f no matter what.
in CPS this is written
let h x next = (g x (fun result -> f result next))
g not only has x as an argument, but a continuation that takes the output of g and returns the final value. This function calls f in the same manner, and gives next as the continuation.
What happened? What changed that made this so much more useful than f (g x)? The difference is that now g is in control. It can decide whether to use what happens next or not. That is the essence of continuations.
An example of where continuations arise are imperative programming languages where you have control structures. Whiles, blocks, ordinary statements, breaks and continues are all generalized through continuations, because these control structures take what happens next and decide what to do with it, for example we can have
...
while(condition1) {
statement1;
if(condition2) break;
statement2;
if(condition3) continue;
statement3;
}
return statement3;
...
The while, the block, the statement, the break and the continue can all be described in a functional model through continuations. Each construct can be considered to be a function that accepts the
current environment containing
the enclosing scopes
optional functions accepting the current environment and returning a continuation to
break from the inner most loop
continue from the inner most loop
return from the current function.
all the blocks associated with it (if-blocks, while-block, etc)
a continuation to the next statement
and returns the new environment.
In the while loop, the condition is evaluated according to the current environment. If it is evaluated to true, then the block is evaluated and returns the new environment. The result of evaluating the while loop again with the new environment is returned. If it is evaluated to false, the result of evaluating the next statement is returned.
With the break statement, we lookup the break function in the environment. If there is no function found then we are not inside a loop and we give an error. Otherwise we give the current environment to the function and return the evaluated continuation, which would be the statement after the the while loop.
With the continue statement the same would happen, except the continuation would be the while loop.
With the return statement the continuation would be the statement following the call to the current function, but it would remove the current enclosing scope from the environment.
What happens with the stack calls and so on and so forth when executing a recursive function? Does recursion even use a stack in the first place? I would appreciate an answer that helps to visualize better what happens during recursion.
Generally, a recursive call is the same as any other function call. It creates a new stack frame, saves old variables and ultimately returns to the caller, just like any old function call. This means that a recursive function can cause a stack overflow. (In fact, that's probably the easiest way to overflow your stack!)
In some languages, however, there is an exception for tail recursion. Tail recursion involves a recursive call that is the very last thing a function does (ie a call in tail position). This means the function can't do anything to the result of the recursive call except returning it directly. Compare these two silly examples:
// Not tail-recursive: we add 1 to the result of foo()
function foo(x) {
if (x > 0) {
return 1 + foo(x - 1)
} else {
return 0;
}
}
// Tail recursive: we return foo() directly
// (`x - 1' happens *before* foo is called)
function foo(x) {
if (x > 0) {
return foo(x - 1);
} else {
return 0;
}
}
If a function is tail recursive, there is not point in allocating a stack frame at each iteration since no information needs to be preserved. Instead, the existing stack frame can be reused or the whole thing can be rewritten into a loop.
Some languages like Scala do this, which means you can write iterative procedures in a recursive style without hitting stack overflows.
However, there is really nothing special about recursion. If a function call is in tail position, we don't need the stack even if it's a call to a different function. We can just implement tail calls as jumps. This is mandated by certain languages (like Scheme) but cannot be implemented in Scala because of Java compatibility reasons.
Proper tail calls like this are important for enabling mutual recursion and continuation passing style without worrying about stack overflows.
So really, there is nothing fundamentally special about recursive calls as opposed to normal calls except that certain languages can only optimize direct recursion in tail position, not tail calls in general.