What is the theoretical/practical limit to the recursion depth in languages implementing Tail Call optimisation? (Please assume that the recurring function is properly tail call-ed).
My guess is that the theoretical limit is NONE, as there is no recursive process, even though it is recursive procedure. Practical limit would be that allowed by the main memory available to be used. Please clarify or correct if I am wrong somewhere.
When a tail recursive function is optimized, it'll essentially become an iterative function. The compiler reuses the stack frame of the original call for subsequent calls, so you won't run out of stack space. If you are not allocating any heap memory (or any other kind of memory that's not on the stack, for that matter), you can have infinitely deep (as long as you are patient enough ;)) recursion (think of it as an infinite loop; it has the same characteristics).
To summarize, there's no practical limit.
In addition to what #Mehrdad Afshari wrote, I just want to point out that it is actually very important that tail recursion (or more generally a chain of tail calls) can be potentially infinite, because otherwise you couldn't write a web server, an operating system, an interpreter, a REPL, or really any kind of event processing loop in a functional language.
After all, an operating system is nothing but an infinite loop, and the way to write a loop in a functional language is using tail recursion. If tail recursion weren't infinite, the loop wouldn't be infinite. Therefore, you could not only not write an operating system, the language wouldn't even be Turing-complete.
Basically, this is how you write a web server in a functional language:
def loop(queue) = {
// handle first request in queue
loop(queue)
}
Without infinite tail recursion, this would quickly run out of memory.
Related
We can print a linked list in reverse order with a stack as well as using recursion. My teacher said that using explicit stack is better since recursion also uses stack but has to maintain a lot of other parameters. Even if we use std::stack from stack, doesn't referring to an external library also take up time? How does using an explicit stack save time/space compared to using a recursive solution?
Recursion involves the use of implicit stacks. This is implemented in the background by the compiler being used to compile your code.
This background stack created by the compiler is known as a ‘Call stack’. Call Stack can be implemented using stack data structure which stores information about the active subroutines of a computer program.
Each subroutine call uses a frame inside the call stack called the stack frame. When a function returns a value, it’s stack frame is popped off the call stack.
Recursion's Call Stack vs Explicit Call Stack?
Stack overflow
The fundamental difference between the 2 stacks is that the space allocated by the compiler for the call stack of a program is fixed. This means there will be a stack overflow if you’re not sure about the maximum no. of recursive function calls expected and there are way too many calls than the space allocated to the stack can handle at a given point of time.
On the other hand, if you define an explicit stack, it’s implemented on the heap space allocated to the program by the compiler at run time. And guess what, the heap size is not fixed and can increase dynamically during run-time when required. You don’t really have to worry about the explicit stack overflowing.
Space and Time
Which one will be faster for a given situation?
Iterating on an explicit stack can be faster than recursion in languages that don’t support recursion related optimizations such as tail call optimization for tail recursion.
What’s Tail recursion?
Tail recursion is a special case of recursion where the recursive function doesn’t do any more computation after the recursive function call i.e. the last step of the function is a call to the recursive function.
What’s Tail-call optimization (TCO)?
Tail-call optimization is where you are able to avoid allocating a new stack frame for a function because the calling function will simply return the value that it gets from the called function.
So, compilers/languages that support tail-call optimizations implement the call to the recursive function with only a single stack frame in the call stack. If your compiler/language doesn’t support this, then using an explicit stack will save you a LOT of space and time.
Python doesn’t support tail call optimization. The main reason for this is to have a complete and clear stack trace which enables efficient debugging. Almost all C/C++ compilers support tail call optimization.
Sometimes explicitly controlling the stack helps simplify things when multiple parameters are being used.
whereas, a recursive solution makes the size of the source code a lot smaller and more maintainable.
Conclusion
In the end, There’s no fixed answer. For a particular scenario, many factors need to be considered such as scalability, code maintainability, language/compiler being used, etc.
The best way would be to implement the solution using both ways, time the 2 solutions on an input set and analyze peak space utilization before deploying it on a production setup.
See Wikipedia - Recursion versus Iteration
I tried using the y-combinator (in both Lua and Clojure) as I thought that would allow me to exceed the size of default stack implementations when using recursion. It seems I was mistaken. Yes, it works, but in both of these systems, the stack blows at exactly the same point as it does using plain old recursion. A lowish ~3600 in Clojure and a highish ~333000 on my Android Lua implementation. It is also a bit slower than regular recursion.
So is there anything to be gained by using the y-combinator, or is it just an intellectual exercise to prove a point? Have I missed something?
===
PS. Sorry, I should have made it clearer that I am aware I can use TCO to exceed the stack. My question does not relate to that. I am interested in this
a) from the academic/intellectual point of view
b) whether there is anything that can be done about those function that cannot be written tail recursively.
The Y combinator allows a non-recursive function to be used recursively, but that recursion still consumes stack space through nested function invocations.
For functions that can't be made tail-recursive, you could try refactoring them using continuation passing style, which would consume heap space instead of stack.
Here's a good overview of the topic: https://www.cs.cmu.edu/~15150/previous-semesters/2012-spring/resources/lectures/11.pdf
A “tail call” will allow you to exceed any stack size limitations. See Programming in Lua, section 6.3: Proper Tail Calls:
...after the tail call, the program does not need to keep any information about the calling function in the stack. Some language implementations, such as the Lua interpreter, take advantage of this fact and actually do not use any extra stack space when doing a tail call. We say that those implementations support proper tail calls.
If you haven't seen it yet, there is a good explanation here: What is a Y-combinator?
In summary, it helps to prove that the lambda calculus is Turing complete, but is useless for normal programming tasks.
As you are probably aware, in Clojure you would just use loop/recur to implement a loop that does not consume the stack.
Why does Clojure have the "recur" special form?
When I replace the "recur" with the function itself I get the same result:
(defn print-down-from [x]
(when (pos? x)
(println x)
(recur (dec x))
)
)
(print-down-from 5)
Has the same result as
(defn print-down-from [x]
(when (pos? x)
(println x)
(print-down-from (dec x))
)
)
(print-down-from 5)
I was wondering if "recur" is just a safety measure so that the compiler will throw an error if a developer happens to use a non-tail recursion. Or maybe it's necessary for the compiler to optimize the tail recursion?
But what I'm wondering most is about any other reasons other than stack consumption.
As explained in the clojure.org page on functional programming:
In the absence of mutable local variables, looping and iteration must take a different form than in languages with built-in for or while constructs that are controlled by changing state. In functional languages looping and iteration are replaced/implemented via recursive function calls. Many such languages guarantee that function calls made in tail position do not consume stack space, and thus recursive loops utilize constant space. Since Clojure uses the Java calling conventions, it cannot, and does not, make the same tail call optimization guarantees. Instead, it provides the recur special operator, which does constant-space recursive looping by rebinding and jumping to the nearest enclosing loop or function frame. While not as general as tail-call-optimization, it allows most of the same elegant constructs, and offers the advantage of checking that calls to recur can only happen in a tail position.
When you don't use recur (or trampoline), your function calls consume stack: Thus, if you ran (print-down-from 100000), you would quickly observe a crash.
Providing this language facility, then, offers several benefits:
Contrary to conventional recursive calls (as given in the question's example), using recur does not consume stack.
The author knows that TCO is in use (and stack is not consumed), as use in a position where TCO is not possible would cause a compile-time failure. As such, the stack-consumptions characteristics are obvious both to the code's author and its readers, unlike languages with only automatic TCO (where one would need to read carefully -- taking macro expansions into account -- to determine whether a call is genuinely in tail position before knowing whether it is optimized).
Compatibility with conventional JVM calling conventions, and thus native interoperability with code written in other JVM-centric languages is preserved.
Finally, for background, see one of several other questions on the topic on StackOverflow:
Automatic TCO in Clojure (asking whether an automatic facility to make explicit invocation of recur unnecessary is available).
Why does TCO require support from the VM? (asking for the basis behind the claim made in the above-quoted clojure.org documentation that automatic TCO cannot be implemented without either JVM support or breaking use of JVM calling conventions and thus harming interoperability).
Why can't tail calls be optimized in JVM-based Lisps? (likewise, more generally)
As I understand it, the loop .. recur uses tail-end recursion, so your program doesn't blow the stack, and regular recursion does not. Some solutions to problems on 4clojure wind up not using loop .. recur, because -- I'm taking an educated guess -- the solution can only be derived using a direct, recursive function call, instead of loop .. recur.
From what I have read, in some of the Clojure books from a few years ago, you should feel free to use loop .. recur.
However, at least from the discussion in those books I have read and from answers I have received to my Clojure questions here in SO, there is some general consensus to try to solve your problem first using constructs like map. If that is not possible, then by all means go ahead and use loop .. recur, and if you do not think there is a possibility of blowing your stack, a direct recursion call.
I was reading the book, Structure and Interpretation of Computer Programs, where in it tells about the distinction between a recursive procedure and recursive process, and similarly between iterative procedure and iterative process. So, a recursive procedure could still generate an iterative process.
My question is: given a procedure which generates a recursive process, can you always write another procedure that achieves the same result but generates an iterative process?
The specific problem that I was trying to solve was to write a procedure which does an in-order traversal of a binary search tree but generates an iterative process. I know how you can use a stack to get an iterative procedure for this problem. However, that still generates a recursive process (correct me if I am wrong here).
Thanks,
Abhinav.
Some tasks are truly impossible to solve with linear iterative processes (e.g. tree recursion, which is impossible to convert to tail recursion). You either have to use the stack built into your platform, or re-create it yourself within the language (usually a much less efficient and uglier solution).
So if you define 'recursion' as 'using a stack to store different invocations of the same code', then yes, recursion sometimes is absolutely required.
If you define 'recursion' as 'a function in my language (eventually) calling itself', then you can get by without explicit recursion by re-implementing recursiveness yourself, as describes above. This is only useful if your language doesn't provide recursive procedures, or not enough stack space, or has similar limitations. (For instance, early Fortran's didn't have recursive procedures. Of course, they also didn't have the dynamic data structures that you would need to simulate them! Personally, I have never come across an actual example where implementing pseudo-recursion was the right solution.)
Read this former SO post:
Design patterns for converting recursive algorithms to iterative ones
there are a lot of good answers there which may help you further.
Any tail recursive process can be transformed into an iterative one.
But not all recursive processes can be transformed into an iterative one.
With respect to using recursion over non-recursive methods in sorting algorithms or, for that matter, any algorithm what are its pros and cons?
For the most part recursion is slower, and takes up more of the stack as well. The main advantage of recursion is that for problems like tree traversal it make the algorithm a little easier or more "elegant".
Check out some of the comparisons:
link
Recursion means a function calls repeatedly
It uses system stack to accomplish its task. As stack uses LIFO approach
and when a function is called the controlled is moved to where function is defined which has it is stored in memory with some address, this address is stored in stack
Secondly, it reduces a time complexity of a program.
Though bit off-topic,a bit related. Must read. : Recursion vs Iteration
All algorithms can be defined recursively. That makes it much, much easier to visualize and prove.
Some algorithms (e.g., the Ackermann Function) cannot (easily) be specified iteratively.
A recursive implementation will use more memory than a loop if tail call optimization can't be performed. While iteration may use less memory than a recursive function that can't be optimized, it has some limitations in its expressive power.
Any algorithm implemented using recursion can also be implemented using iteration.
Why not to use recursion
It is usually slower due to the overhead of maintaining the stack.
It usually uses more memory for the stack.
Why to use recursion
Recursion adds clarity and (sometimes) reduces the time needed to write and debug code (but doesn't necessarily reduce space requirements or speed of execution).
Reduces time complexity.
Performs better in solving problems based on tree structures.
For example, the Tower of Hanoi problem is more easily solved using recursion as opposed to iteration.
I personally prefer using Iterative over recursive function. Especially if you function has complex/heavy logic and number of iterations are large. This because with every recursive call call stack increases. It could potentially crash the stack if you operations are too large and also slow up process.
To start:
Pros:
It is the unique way of implementing a variable number of nested loops (and the only elegant way of implementing a big constant number of nested loops).
Cons:
Recursive methods will often throw a StackOverflowException when processing big sets. Recursive loops don't have this problem though.
To start :
Recursion:
A function that calls itself is called as recursive function and this technique is called as recursion.
Pros:
1. Reduce unnecessary calling of functions.
2. Through Recursion one can solve problems in easy way while its iterative solution is very big and complex.
3. Extremely useful when applying the same solution.
Cons:
1. Recursive solution is always logical and it is very difficult to trace.
2. In recursive we must have an if statement somewhere to force the function to return without the recursive call being executed, otherwise the function will never return.
3. Recursion uses more processor time.
Expressiveness
Most problems are naturally expressed by recursion such as Fibonacci, Merge sorting and quick sorting. In this respect, the code is written for humans, not machines.
Immutability
Iterative solutions often rely on varying temporary variables which makes the code hard to read. This can be avoided with recursion.
Performance
Recursion is not stack friendly. Stack can overflow when the recursion is not well designed or tail optimization is not supported.
Some situation would arise where you would have to abandon recursion in a problem where recursion appears to be to your advantage, this is because for problems where your recursion would have to occur thousand of times this would result in a stackoverflow error even though your code did not get stuck in an infinite recursion. Most programming languages limits you to a number of stack calls, so if your recursion goes beyond this limit, then you might consider not using recursion.
We should use recursion in following scenarios:
when we don't know the finite number of iteration for example our fuction exit condition is based on dynamic programming (memoization)
when we need to perform operations on reverse order of the elements. Meaning we want to process last element first and then n-1, n-2 and so on till first element
Recursion will save multiple traversals. And it will be useful, if we can divide the stack allocation like:
int N = 10;
int output = process(N) + process(N/2);
public void process(int n) {
if (n==N/2 + 1 || n==1) {
return 1;
}
return process(n-1) + process(n-2);
}
In this case only half stacks will be allocated at any given time.
Recursion gets a bad rep, I'm always surprised by the number of developers that wont even touch recursion because someone told them it was evil incarnate.
I've learned through trial and error that when done properly recursion can be one of the fastest ways to iterate over something, it is not a steadfast rule and each language/ compiler/ engine has it's own quirks so mileage will vary.
In javascript I can reliably speed up almost any iterative process by introducing recursion with the added benefit of reducing side effects and making the code more clear concise and reusable. Also pro tip its possible to get around the stack overflow issue (and no you dont disable the warning).
My personal Pros & Cons:
Pros:
- Reduces side effects.
- Makes code more concise and easier to reason about.
- Reduces system resource usage and performs better than the traditional for loop.
Cons:
- Can lead to stack overflow.
- More complicated to setup than a traditional for loop.
Mileage will vary depending on language/ complier/ engine.