I've got an internal state, which has to be updated continuously. Following functional programming, the state is immutable and the loop is implemented using recursion. An example way of how this could be implemented in a hypothetical (C based) language is then:
void RunUpdateLoop(WorldState world)
{
WorldState newWorld = Update(world);
RunUpdateLoop(newWorld);
}
However, this would quickly result in a stack overflow (assuming tail recursion optimization doesn't happen). How can this be implemented in a functional way without resulting in a stack overflow?
If there was a tail recursion optimization, the generated code would be very similar to something like
void RunUpdateLoop(WorldState world) {
while(1) { world = RunUpdateLoop(world); }
}
No?
Of course you could say that's not "functional" enough, but you're asking for a "functional" solution in a procedural language. So, not quite sure what you are after.
You can always simulate a call stack in the heap with a stack-like data structure, and do the tail-recursion optimization yourself, but in this case it will once again collapse to something very similar to the above.
Related
I'm referring to this article https://codeburst.io/a-beginner-friendly-intro-to-functional-programming-4f69aa109569. I've been confused with how functional programming differs from procedural programming for a while and want to understand it.
function getOdds2(arr){
return arr.filter(num => num % 2 !== 0)
}
"we simply define the outcome we want to see by using a method called filter, and allow the machine to take care of all the steps in between. This is a more declarative approach."
They say they define the outcome we want to see. How is that any different from procedural programming where you call a function (a sub routine) and get the output you want to see. How is this "declarative approach" any different than just calling a function. The only difference I see between functional and procedural programming is the style.
Procedural programming uses procedures (i.e. calls functions, aka subroutines) and potentially (refers to, and/or changes in place aka "mutates") lots of global variables and/or data structures, i.e. "state".
Functional programming seeks to have all the relevant state in the function's arguments and return values, to minimize if not outright forbid the state external to the functions used (called) in a program.
So, it seeks to have few (or none at all) global variables and data structures, just have functions (subroutines) passing data along until the final one produces the result.
Both are calling functions / subroutines, yes. That's just structured programming, though.
"Pure" functional programming doesn't want its functions to have (and maintain) any internal state (between the calls) either. That way the programming functions / subroutines / procedures become much like mathematical functions, always producing the same output for the same input(s). The operations of such functions become reliable, allowing for the more declarative style of programming. (when it might do something different each time it's called, there's no name we can give it, no concept to call it for).
With functional programming, everything revolves around functions. In the example you gave above, you are defining and executing a function without even knowing it.
You're passing a lambda (anonymous function) to the filter function. With functional programming, that's really the point - defining everything in terms of functions, passing functions as arguments to functions, etc.
So yes, it is about style. It's about ease of expressing your ideas in code and being additionally terse.
Take this for example:
function getOdds(array) {
odds = [];
for(var i = 0; i < array.length; i++) {
if(isOdd(array[i]) array.push(i);
}
}
function isOdd(n) {
return n % 2 != 0;
}
it gets the job done but it's really verbose. Now compare it to:
function getOdds(array) {
array.filter(n => n % 2 != 0)
}
Here you're defining the isOdd function anonymously as a lambda and passing it directly to the filter function. Saves a lot of keystrokes.
I've been recently learning about functional languages and how many don't include for loops. While I don't personally view recursion as more difficult than a for loop (and often easier to reason out) I realized that many examples of recursion aren't tail recursive and therefor cannot use simple tail recursion optimization in order to avoid stack overflows. According to this question, all iterative loops can be translated into recursion, and those iterative loops can be transformed into tail recursion, so it confuses me when the answers on a question like this suggest that you have to explicitly manage the translation of your recursion into tail recursion yourself if you want to avoid stack overflows. It seems like it should be possible for a compiler to do all the translation from either recursion to tail recursion, or from recursion straight to an iterative loop with out stack overflows.
Are functional compilers able to avoid stack overflows in more general recursive cases? Are you really forced to transform your recursive code in order to avoid stack overflows yourself? If they aren't able to perform general recursive stack-safe compilation, why aren't they?
Any recursive function can be converted into a tail recursive one.
For instance, consider the transition function of a Turing machine, that
is the mapping from a configuration to the next one. To simulate the
turing machine you just need to iterate the transition function until
you reach a final state, that is easily expressed in tail recursive
form. Similarly, a compiler typically translates a recursive program into
an iterative one simply adding a stack of activation records.
You can also give a translation into tail recursive form using continuation
passing style (CPS). To make a classical example, consider the fibonacci
function.
This can be expressed in CPS style in the following way, where the second
parameter is the continuation (essentially, a callback function):
def fibc(n, cont):
if n <= 1:
return cont(n)
return fibc(n - 1, lambda a: fibc(n - 2, lambda b: cont(a + b)))
Again, you are simulating the recursion stack using a dynamic data structure:
in this case, lambda abstractions.
The use of dynamic structures (lists, stacks, functions, etc.) in all previous
examples is essential. That is to say, that in order to simulate a generic
recursive function iteratively, you cannot avoid dynamic memory allocation,
and hence you cannot avoid stack overflow, in general.
So, memory consumption is not only related to the iterative/recursive
nature of the program. On the other side, if you prevent dynamic memory
allocation, your
programs are essentially finite state machines, with limited computational
capabilities (more interesting would be to parametrise memory according to
the dimension of inputs).
In general, in the same way as you cannot predict termination, you cannot
predict an unbound memory consumption of your program: working with
a Turing complete language, at compile time
you cannot avoid divergence, and you cannot avoid stack overflow.
Tail Call Optimization:
The natural way to do arguments and calls is to sort out the cleaning up when exiting or when returning.
For tail calls to work you need to alter it so that the tail call inherits the current frame. Thus instead of making a new frame it massages the frame so that the next call returns to the current functions caller instead of this function, which really only cleans up and returns if it's a tail call.
Thus TCO is all about cleaning up before the last call.
Continuation Passing Style - make tail calls out of everything
A compiler can change the code such that it only does primitive operations and pass it to continuations. Thus the stack usage gets moved onto the heap since the computation to be continued is made a function.
An example is:
function hypotenuse(k1, k2) {
return sqrt(add(square(k1), square(k2)))
}
becomes
function hypotenuse(k, k1, k2) {
(function (sk1) {
(function (sk2) {
(function (ar) {
k(sqrt(ar));
}(add(sk1,sk2));
}(square(k2));
}(square(k1));
}
Notice every function has exactly one call now and the order of evaluation is set.
According to this question, all iterative loops can be translated into recursion
"Translated" might be a bit of a stretch. The proof that for every iterative loop there is an equivalent recursive program is trivial if you understand Turing completeness: since a Turing machine can be implemented using strictly iterative structures and strictly recursive structures, every program that can be expressed in an iterative language can be expressed in a recursive language, and vice-versa. This means that for every iterative loop there is an equivalent recursive construct (and the other way around). However, that doesn't mean we have some automated way of transforming one into the other.
and those iterative loops can be transformed into tail recursion
Tail recursion can perhaps be easily transformed into an iterative loop, and the other way around. But not all recursion is tail recursion. Here's an example. Suppose we have some binary tree. It consists of nodes. Each node can have a left and a right child and a value. If a node has no children, then isLeaf returns true for it. We'll assume there's some function max that returns the maximum of two values, and if one of the values is null it returns the other one. Now we want to define a function that finds the maximum value among all the leaf nodes. Here it is in some pseudo-code I cooked up.
findmax(node) {
if (node == null) {
return null
}
if (node.isLeaf) {
return node.value
} else {
return max(findmax(node.left), findmax(node.right))
}
}
There's two recursive calls in the max function, so we can't optimize for tail recursion. We need the results of both before we can supply them to the max function and determine the result of the call for the current node.
Now, there may be a way of getting the same result, using recursion and only a single tail-recursive call. It is functionally equivalent, but it is a different algorithm. Compilers can do a lot of transformations to create a functionally equivalent program with lots of optimizations, but they're not quite clever enough to create functionally equivalent algorithms.
Even the transformation of a function that only calls itself recursively once into a tail-recursive version would be far from trivial. Such an adaptation usually employs some argument passed into the recursive invocation that is used as an "accumulator" for the current results.
Look at the next naive implementation for calculating a factorial of a number (e.g. fact(5) = 5*4*3*2*1):
fact(number) {
if (number == 1) {
return 1
} else {
return number * fact(number - 1)
}
}
It's not tail-recursive. But it can be made so in this way:
fact(number, acc) {
if (number == 1) {
return acc
} else {
return fact(number - 1, number * acc)
}
}
// Helper function
fact(number) {
return fact(number, 1)
}
This requires an interpretation of what is being done. Recognizing the case for stuff like this is easy enough, but what if you call a function instead of a multiplication? How will the compiler know that for the initial call the accumulator must be 1 and not, say, 0? How do you translate this program?
recsub(number) {
if (number == 1) {
return 1
} else {
return number - recsub(number - 1)
}
}
This is as of yet outside the scope of the sort of compiler we have now, and may in fact always be.
Maybe it would be interesting to ask this on the computer science Stack Exchange to see if they know of some papers or proofs that investigate this more in-depth.
I've had this question on my mind for a really long time but I can't figure out the answer. The question is, if does every recursive function have an iterative function that does the same?
For example,
factorial(n) {
if (n==1) { return 1 }
else { return factorial(n-1) }
}
This can be easily rewritten iteratively:
factorial(n) {
result = 1;
for (i=1; i<=n; i++) {
result *= i
}
return result
}
But there are many other, more complicated recursive functions, so I don't know the answer in general. This might also be a theoretical computer science question.
Yes, a recursive function can always be written as an iteration, from a theoretical point of view - this has been discussed before. Quoting from the linked post:
Because you can build a Turing complete language using strictly iterative structures and a Turning complete language using only recursive structures, then the two are therefore equivalent.
Explaining a bit: we know that any computable problem can be solved by a Turing machine. And it's possible to construct a programming language A without recursion, that is equivalent to a Turing machine. Similarly, it's possible to build a programming language B without iteration, equal in computational power to a Turing machine.
Therefore, if both A and B are Turing-complete we can conclude that for any iterative program there must exist an equivalent recursive program, and vice versa. This is a theoretical result, in the sense that it doesn't give you any hints on how to derive one recursive program from an arbitrary iterative program, or vice versa.
Without going to theory, it is easy to convince oneself that any recursive function can have an iterative equivalent by observing that processors (such as Pentium) just run iteratively.
I've been writing (unsophisticated) code for a decent while, and I feel like I have a somewhat firm grasp on while and for loops and if/else statements. I should also say that I feel like I understand (at my level, at least) the concept of recursion. That is, I understand how a method keeps calling itself until the parameters of an iteration match a base case in the method, at which point the methods begin to terminate and pass control (along with values) to previous instances and eventually an overall value of the first call is determined. I may not have explained it very well, but I think I understand it, and I can follow/make traces of the structured examples I've seen. But my question is on creating recursive methods in the wild, ie, in unstructured circumstances.
Our professor wants us to write recursively at every opportunity, and has made the (technically inaccurate?) statement that all loops can be replaced with recursion. But, since many times recursive operations are contained within while or for loops, this means, to state the obvious, not every loop can be replaced with recursion. So...
For unstructured/non-classroom situations,
1) how can I recognize that a loop situation can/cannot be turned into a recursion, and
2) what is the overall idea/strategy to use when applying recursion to a situation? I mean, how should I approach the problem? What aspects of the problem will be used as recursive criteria, etc?
Thanks!
Edit 6/29:
While I appreciate the 2 answers, I think maybe the preamble to my question was too long because it seems to be getting all of the attention. What I'm really asking is for someone to share with me, a person who "thinks" in loops, an approach for implementing recursive solutions. (For purposes of the question, please assume I have a sufficient understanding of the solution, but just need to create recursive code.) In other words, to apply a recursive solution, what am I looking for in the problem/solution that I will then use for the recursion? Maybe some very general statements about applying recursion would be helpful too. (note: please, not definitions of recursion, since I think I pretty much understand the definition. It's just the process of applying them I am asking about.) Thanks!
Every loop CAN be turned into recursion fairly easily. (It's also true that every recursion can be turned into loops, but not always easily.)
But, I realize that saying "fairly easily" isn't actually very helpful if you don't see how, so here's the idea:
For this explanation, I'm going to assume a plain vanilla while loop--no nested loops or for loops, no breaking out of the middle of the loop, no returning from the middle of the loop, etc. Those other things can also be handled but would muddy up the explanation.
The plain vanilla while loop might look like this:
1. x = initial value;
2. while (some condition on x) {
3. do something with x;
4. x = next value;
5. }
6. final action;
Then the recursive version would be
A. def Recursive(x) {
B. if (some condition on x) {
C. do something with x;
D. Recursive(next value);
E. }
F. else { # base case = where the recursion stops
G. final action;
H. }
I.
J. Recursive(initial value);
So,
the initial value of x in line 1 became the orginial argument to Recursive on line J
the condition of the loop on line 2 became the condition of the if on line B
the first action inside the loop on line 3 became the first action inside the if on line C
the next value of x on line 4 became the next argument to Recursive on line D
the final action on line 6 became the action in the base case on line G
If more than one variable was being updated in the loop, then you would often have a corresponding number of arguments in the recursive function.
Again, this basic recipe can be modified to handle fancier situations than plain vanilla while loops.
Minor comment: In the recursive function, it would be more common to put the base case on the "then" side of the if instead of the "else" side. In that case, you would flip the condition of the if to its opposite. That is, the condition in the while loop tests when to keep going, whereas the condition in the recursive function tests when to stop.
I may not have explained it very well, but I think I understand it, and I can follow/make traces of the structured examples I've seen
That's cool, if I understood your explanation well, then how you think recursion works is correct at first glance.
Our professor wants us to write recursively at every opportunity, and has made the (technically inaccurate?) statement that all loops can be replaced with recursion
That's not inaccurate. That's the truth. And the inverse is also possible: every time a recursive function is used, that can be rewritten using iteration. It may be hard and unintuitive (like traversing a tree), but it's possible.
how can I recognize that a loop can/cannot be turned into a recursion
Simple:
what is the overall idea/strategy to use when doing the conversion?
There's no such thing, unfortunately. And by that I mean that there's no universal or general "work-it-all-out" method, you have to think specifically for considering each case when solving a particular problem. One thing may be helpful, however. When converting from an iterative algorithm to a recursive one, think about patterns. How long and where exactly is the part that keeps repeating itself with a small difference only?
Also, if you ever want to convert a recursive algorithm to an iterative one, think about that the overwhelmingly popular approach for implementing recursion at hardware level is by using a (call) stack. Except when solving trivially convertible algorithms, such as the beloved factorial or Fibonacci functions, you can always think about how it might look in assembler, and create an explicit stack. Dirty, but works.
for(int i = 0; i < 50; i++)
{
for(int j = 0; j < 60; j++)
{
}
}
Is equal to:
rec1(int i)
{
if(i < 50)
return;
rec2(0);
rec1(i+1);
}
rec2(int j)
{
if(j < 60)
return;
rec2(j + 1);
}
Every loop can be recursive. Trust your professor, he is right!
This question already has answers here:
Can every recursion be converted into iteration?
(18 answers)
Closed 2 years ago.
As far as I know, most recursive functions can be rewritten using loops. Some may be harder than others, but most of them can be rewritten.
Under which conditions does it become impossible to rewrite a recursive function using a loop (if such conditions exist)?
When you use a function recursively, the compiler takes care of stack management for you, which is what makes recursion possible. Anything you can do recursively, you can do by managing a stack yourself (for indirect recursion, you just have to make sure your different functions share that stack).
So, no, there is nothing that can be done with recursion and that cannot be done with a loop and a stack.
Any recursive function can be made to iterate (into a loop) but you need to use a stack yourself to keep the state.
Normally, tail recursion is easy to convert into a loop:
A(x) {
if x<0 return 0;
return something(x) + A(x-1)
}
Can be translated into:
A(x) {
temp = 0;
for i in 0..x {
temp = temp + something(i);
}
return temp;
}
Other kinds of recursion that can be translated into tail recursion are also easy to change. The other require more work.
The following:
treeSum(tree) {
if tree=nil then 0
else tree.value + treeSum(tree.left) + treeSum(tree.right);
}
Is not that easy to translate. You can remove one piece of the recursion, but the other one is not possible without a structure to hold the state.
treeSum(tree) {
walk = tree;
temp = 0;
while walk != nil {
temp = temp + walk.value + treeSum(walk.right);
walk = walk.left;
}
}
Every recursive function can be implemented with a single loop.
Just think what a processor does, it executes instructions in a single loop.
I don't know about examples where recursive functions cannot be converted to an iterative version, but impractical or extremely inefficient examples are:
tree traversal
fast Fourier
quicksorts (and some others iirc)
Basically, anything where you have to start keeping track of boundless potential states.
It's not so much a matter of that they can't be implemented using loops, it's the fact that the way the recursive algorithm works, it's much clearer and more concise (and in many cases mathematically provable) that a function is correct.
Many recursive functions can be written to be tail loop recursive, which can be optimised to a loop, but this is dependent on both the algorithm and the language used.
They all can be written as an iterative loop (but some might still need a stack to keep previous state for later iterations).
One example which would be extremely difficult to convert from recursive to iterative would be the Ackermann function.
Indirect recursion is still possible to convert to a non-recursive loop; just start with one of the functions, and inline the calls to the others until you have a directly recursive function, which can then be translated to a loop that uses a stack structure.
In SICP, the authors challenge the reader to come up with a purely iterative method of implementing the 'counting change' problem (here's an example one from Project Euler).
But the strict answer to your question was already given - loops and stacks can implement any recursion.
You can always use a loop, but you may have to create more data structure (e.g. simulate a stack).