We can print a linked list in reverse order with a stack as well as using recursion. My teacher said that using explicit stack is better since recursion also uses stack but has to maintain a lot of other parameters. Even if we use std::stack from stack, doesn't referring to an external library also take up time? How does using an explicit stack save time/space compared to using a recursive solution?
Recursion involves the use of implicit stacks. This is implemented in the background by the compiler being used to compile your code.
This background stack created by the compiler is known as a ‘Call stack’. Call Stack can be implemented using stack data structure which stores information about the active subroutines of a computer program.
Each subroutine call uses a frame inside the call stack called the stack frame. When a function returns a value, it’s stack frame is popped off the call stack.
Recursion's Call Stack vs Explicit Call Stack?
Stack overflow
The fundamental difference between the 2 stacks is that the space allocated by the compiler for the call stack of a program is fixed. This means there will be a stack overflow if you’re not sure about the maximum no. of recursive function calls expected and there are way too many calls than the space allocated to the stack can handle at a given point of time.
On the other hand, if you define an explicit stack, it’s implemented on the heap space allocated to the program by the compiler at run time. And guess what, the heap size is not fixed and can increase dynamically during run-time when required. You don’t really have to worry about the explicit stack overflowing.
Space and Time
Which one will be faster for a given situation?
Iterating on an explicit stack can be faster than recursion in languages that don’t support recursion related optimizations such as tail call optimization for tail recursion.
What’s Tail recursion?
Tail recursion is a special case of recursion where the recursive function doesn’t do any more computation after the recursive function call i.e. the last step of the function is a call to the recursive function.
What’s Tail-call optimization (TCO)?
Tail-call optimization is where you are able to avoid allocating a new stack frame for a function because the calling function will simply return the value that it gets from the called function.
So, compilers/languages that support tail-call optimizations implement the call to the recursive function with only a single stack frame in the call stack. If your compiler/language doesn’t support this, then using an explicit stack will save you a LOT of space and time.
Python doesn’t support tail call optimization. The main reason for this is to have a complete and clear stack trace which enables efficient debugging. Almost all C/C++ compilers support tail call optimization.
Sometimes explicitly controlling the stack helps simplify things when multiple parameters are being used.
whereas, a recursive solution makes the size of the source code a lot smaller and more maintainable.
Conclusion
In the end, There’s no fixed answer. For a particular scenario, many factors need to be considered such as scalability, code maintainability, language/compiler being used, etc.
The best way would be to implement the solution using both ways, time the 2 solutions on an input set and analyze peak space utilization before deploying it on a production setup.
See Wikipedia - Recursion versus Iteration
Related
I tried using the y-combinator (in both Lua and Clojure) as I thought that would allow me to exceed the size of default stack implementations when using recursion. It seems I was mistaken. Yes, it works, but in both of these systems, the stack blows at exactly the same point as it does using plain old recursion. A lowish ~3600 in Clojure and a highish ~333000 on my Android Lua implementation. It is also a bit slower than regular recursion.
So is there anything to be gained by using the y-combinator, or is it just an intellectual exercise to prove a point? Have I missed something?
===
PS. Sorry, I should have made it clearer that I am aware I can use TCO to exceed the stack. My question does not relate to that. I am interested in this
a) from the academic/intellectual point of view
b) whether there is anything that can be done about those function that cannot be written tail recursively.
The Y combinator allows a non-recursive function to be used recursively, but that recursion still consumes stack space through nested function invocations.
For functions that can't be made tail-recursive, you could try refactoring them using continuation passing style, which would consume heap space instead of stack.
Here's a good overview of the topic: https://www.cs.cmu.edu/~15150/previous-semesters/2012-spring/resources/lectures/11.pdf
A “tail call” will allow you to exceed any stack size limitations. See Programming in Lua, section 6.3: Proper Tail Calls:
...after the tail call, the program does not need to keep any information about the calling function in the stack. Some language implementations, such as the Lua interpreter, take advantage of this fact and actually do not use any extra stack space when doing a tail call. We say that those implementations support proper tail calls.
If you haven't seen it yet, there is a good explanation here: What is a Y-combinator?
In summary, it helps to prove that the lambda calculus is Turing complete, but is useless for normal programming tasks.
As you are probably aware, in Clojure you would just use loop/recur to implement a loop that does not consume the stack.
Usually functional recursion is simulated using the call stack ,but is there any way to simulate recursion without using the stack ADT?
Yes: one of the well-known approaches to implementing functional languages with first-class continuations is to heap allocate activation records, deallocation being handled by garbage collection. In this scheme call information is arranged in an immutable DAG, and a continuation has the particularly straightforward implementation of a pointer to an activation record.
Without the motivation of first-class continuations this arrangement isn't all that attractive for performance reasons.
You can simulate a stack using an array, for example (not using the stack ADT) ... But if you want an implementation of recursion without using stacks at all - implicit, explicit, or self-defined, go through this link:
http://home.olympus.net/~7seas/recurse.html
Any recursion can be modified to be a iterative function with stack structure, then why do I ever want to do that? If the answer is to avoid stackoverflow, then how come computer can overflow by recursion at all? why not compiler automatically put a recursion function on a stack in heap by default or with an additional keyword? I understand heap is also limited, but it is much bigger than stack allocated to a program.
Activation records can indeed be heap allocated, but this is usually considered too expensive. Despite that there are a few implementations which take this approach: Stackless Python and SML/NJ, for instance.
In those cases the motivation is probably aiding the implementation of continuations, not "fixing" stack overflow.
According to C/C++ maximum stack size of program and MSDN, the maximum stack size on Windows is 1 MiB. Your question is interesting. A keyword would certainly be possible and the compiler could do it. You need to ask compiler makers about it, but I guess it is not a very interesting feature.
Caching would be difficult because of other local variables lying on the stack and recursion book-keeping variables on the heap. Information is scattered around in memory.
Some recursive algorithms can be turned into truly iterative ones (like the calculation of the Factorial) or even into a closed-form expression (see for example the Fibonacci numbers). Then, there is no need for book-keeping of recursive calls.
In an iterative process you keep track of recursion state yourself.
It is possible that you can do this more efficient then the compiler can with the very general procedural mechanism (whose efficiency can be hampered by all kinds of ABI concerns).
Stack checking is possible, but not easy on all systems, since not all systems employ a fixed stack or linear address space. It also has an overhead of course, since the check is added to each recursion.
Exactly what parts of a recursive method call contributes to the stack--say, the returned object, arguments, local variables, etc.?
I'm trying to optimize the levels of recursion that an Android application can do on limited memory before running into a StackOverflowException.
Thanks in advance.
If you run out of stack space, don't optimize your stack usage. Doing that just means the same problem will come back later, with a slightly larger input set or called from somewhere else. And at some point you have reached the theoretical or practical minimum of space you can consume for the problem you're solving. Instead, convert the offending code to use a collection other than the machine stack (e.g. a heap-allocated stack or queue). Doing so sometimes results in very ugly code, but at least it won't crash.
But to answer the question: Generally all the things you name can take stack space, and temporary values take space too (so nesting expressions like crazy just to save local variables won't help). Some of these will be stored in registers, depending on the calling convention, but may have to be spilled(*) anyway. But regardless of the calling convention, this only saves you a few bytes, and everything will have to be spilled for calls as the callee is given usually free reign over registers during the call. So at the time your stack overflows, the stack is indeed crowded with parameters, local variables, and temporaries of earlier calls. Some may be optimized away altogether or share a stack slot if they aren't needed at the same time. Ultimately this is up to the JIT compiler.
(*) Spilling: Moving a value from a register to memory (i.e., the stack) because the register is needed for something else.
Each method has two stack frame sizes associated with it, the stack required for arguments and local variables, and the stack required for expression evaluation. The return value only counts as part of the stack required for expression evaluation. The JVM is able to verify that the method does not exceed these sizes as it executes.
Exactly how much stack is required for variables and expression evaluation is down to the bytecode compiler. For instance it is often able to share local variable slots among variables with non-overlapping lifetimes.
With respect to using recursion over non-recursive methods in sorting algorithms or, for that matter, any algorithm what are its pros and cons?
For the most part recursion is slower, and takes up more of the stack as well. The main advantage of recursion is that for problems like tree traversal it make the algorithm a little easier or more "elegant".
Check out some of the comparisons:
link
Recursion means a function calls repeatedly
It uses system stack to accomplish its task. As stack uses LIFO approach
and when a function is called the controlled is moved to where function is defined which has it is stored in memory with some address, this address is stored in stack
Secondly, it reduces a time complexity of a program.
Though bit off-topic,a bit related. Must read. : Recursion vs Iteration
All algorithms can be defined recursively. That makes it much, much easier to visualize and prove.
Some algorithms (e.g., the Ackermann Function) cannot (easily) be specified iteratively.
A recursive implementation will use more memory than a loop if tail call optimization can't be performed. While iteration may use less memory than a recursive function that can't be optimized, it has some limitations in its expressive power.
Any algorithm implemented using recursion can also be implemented using iteration.
Why not to use recursion
It is usually slower due to the overhead of maintaining the stack.
It usually uses more memory for the stack.
Why to use recursion
Recursion adds clarity and (sometimes) reduces the time needed to write and debug code (but doesn't necessarily reduce space requirements or speed of execution).
Reduces time complexity.
Performs better in solving problems based on tree structures.
For example, the Tower of Hanoi problem is more easily solved using recursion as opposed to iteration.
I personally prefer using Iterative over recursive function. Especially if you function has complex/heavy logic and number of iterations are large. This because with every recursive call call stack increases. It could potentially crash the stack if you operations are too large and also slow up process.
To start:
Pros:
It is the unique way of implementing a variable number of nested loops (and the only elegant way of implementing a big constant number of nested loops).
Cons:
Recursive methods will often throw a StackOverflowException when processing big sets. Recursive loops don't have this problem though.
To start :
Recursion:
A function that calls itself is called as recursive function and this technique is called as recursion.
Pros:
1. Reduce unnecessary calling of functions.
2. Through Recursion one can solve problems in easy way while its iterative solution is very big and complex.
3. Extremely useful when applying the same solution.
Cons:
1. Recursive solution is always logical and it is very difficult to trace.
2. In recursive we must have an if statement somewhere to force the function to return without the recursive call being executed, otherwise the function will never return.
3. Recursion uses more processor time.
Expressiveness
Most problems are naturally expressed by recursion such as Fibonacci, Merge sorting and quick sorting. In this respect, the code is written for humans, not machines.
Immutability
Iterative solutions often rely on varying temporary variables which makes the code hard to read. This can be avoided with recursion.
Performance
Recursion is not stack friendly. Stack can overflow when the recursion is not well designed or tail optimization is not supported.
Some situation would arise where you would have to abandon recursion in a problem where recursion appears to be to your advantage, this is because for problems where your recursion would have to occur thousand of times this would result in a stackoverflow error even though your code did not get stuck in an infinite recursion. Most programming languages limits you to a number of stack calls, so if your recursion goes beyond this limit, then you might consider not using recursion.
We should use recursion in following scenarios:
when we don't know the finite number of iteration for example our fuction exit condition is based on dynamic programming (memoization)
when we need to perform operations on reverse order of the elements. Meaning we want to process last element first and then n-1, n-2 and so on till first element
Recursion will save multiple traversals. And it will be useful, if we can divide the stack allocation like:
int N = 10;
int output = process(N) + process(N/2);
public void process(int n) {
if (n==N/2 + 1 || n==1) {
return 1;
}
return process(n-1) + process(n-2);
}
In this case only half stacks will be allocated at any given time.
Recursion gets a bad rep, I'm always surprised by the number of developers that wont even touch recursion because someone told them it was evil incarnate.
I've learned through trial and error that when done properly recursion can be one of the fastest ways to iterate over something, it is not a steadfast rule and each language/ compiler/ engine has it's own quirks so mileage will vary.
In javascript I can reliably speed up almost any iterative process by introducing recursion with the added benefit of reducing side effects and making the code more clear concise and reusable. Also pro tip its possible to get around the stack overflow issue (and no you dont disable the warning).
My personal Pros & Cons:
Pros:
- Reduces side effects.
- Makes code more concise and easier to reason about.
- Reduces system resource usage and performs better than the traditional for loop.
Cons:
- Can lead to stack overflow.
- More complicated to setup than a traditional for loop.
Mileage will vary depending on language/ complier/ engine.