I'm learning Data Structures & Algorithms now.
My lecture notes have an implementation of a binary search tree which is implemented using a recursive method. That is an elegant way, but my question is in real life code, should I implement a binary search tree recursively, will it generate a lot of calling stack if the tree has large height/depth number.
I understand that recursion is a key concept to understand lots of data structure concepts, but would you choose to use recursion in real life code?
A tree is recursive by nature. Each node of a tree represents a subtree, and each child of each note represents a subtree of that subtree, so recursion is the best bet, especially in practice where other people people might have to edit and maintain your code.
Now, IF depth becomes a problem for your call stack, than I'm afraid that there are deeper problems with your data structure (either it's monstrously huge, or it's very unbalanced)
"I understand that recursive is a key concept to understand lots of
data structure, but will you choose to use recursive in real life
code?"
After first learning about recursion I felt the same way. However, having been working in the Software industry for over a year now, I can say that I have used the concept of recursion to solve several problems. There are often times that recursion is cleaner, easier to understand/read, and just downright better. And to emphasize a point in the previous answer, a tree is a recursive data structure. IMO, there is no other way to traverse a BST :)
Many times, the compiler can optimize your code, to avoid creating a new stack frame for each recursive call (look up tail recursion, for example). Of course, it all depends on the algorithm and on your data structure. If the tree is reasonably balanced, I don't think a recursive algorithm should cause any problems.
its true that recursion is intutive and elegent and it produces code that is clear and concise. its also correct that some methods such as quick sort, DFS etc. are really hard to implement iterativelly.
but in practice recursive implementations are almost always going to be slow when compared to iterative counterparts because of all the function calls (To really understand the performance hit I suggest you learn how much book keeping stuff assembler has to do for a single function call).
the optimizations that we talk about are not applicable to every recursive method in general and manny compilers and interpreters dont even support them.
so in summary if you are writing something which is performance critical such as a data strucute then stay away from recursion (or use it if you are sure that your compiler/interpreter got you covered)
PS: CLRS (introduction to algorithms, page 290, last line) suggests that iterative search procedure for a BST is faster compared to recursive one.
Related
I am learning data structure and algorithms. I found it especially difficult to understand recursions.
So I have the following questions. But they are not related to any specific code.
When I implement methods, when/where should I consider recursion?
In general coding convention, should I prefer recursion over simple iteration if they are both feasible?
How to actually comprehend most possible forms of recursion so I can think of them when I need? What is the best way to learn it? (Any related book or website?) Is there any pattern?
I know the question may sounds unconstructive if you find recursion simple and natural.
But for me it doesn't align with my intuition well. I do appreciate any help.
1
Very often recursive solutions to problems are smaller when data can be seen as similar. Eg. If you have a binary tree and you want to get the sum of all the leaf nodes you define sum-tree as If it's a leaf node, it's sum is it's value, if it's not a leaf node it the addition of the sum of both sub-trees.
Here's a Scheme implementation of my text
(define (sum-tree tree)
(if (leaf? tree)
(node-value tree)
(+ (sum-tree (node-left tree))
(sum-tree (node-right tree)))))
Or the same in Java, defined as a method in the Node class.
public int sum()
{
if ( isLeaf() )
return value;
else
return left.sum() + right.sum();
}
An iterative solution to this would be longer and harder to read. In this case you should prefer recursion.
2
It depends. If you are programming in Python or Java you should not since they donæt have tail recursion. With Scheme however, it's the only way to go. If your language supports tail recursion you should pick recursion when it makes clearer code.
3
Learn by doing. You need to write some algorithms that uses recursion as a tool. Use paper to follow the flow of the stack if you are unsure of the flow. Learning some Scheme or a similar functional language might help you a lot.
Recursion can be used when you are repeating the same thing over and over. For example, you are traversing a tree, you can use a recursion method to go to the left or right child.
I would go for the one that is easier to read. Generally, simple iteration will be faster as it does not have any overhead (recursion has some overhead, and can cause stack overflow if the levels are to deep, while simple iteration won't). But for some case, writing a recursive function is a lot easier than writing the equivalent in the simple iteration.
I would rather see the problem first and then decide whether I need recursion to solve it, not vice versa. Any algorithm book should be good enough. Perhaps you can start over reading http://en.wikipedia.org/wiki/Recursion to begin with. There is a simple example there about recursion, which I think you will be able to implement too using simple iteration.
At first, wrapping my head around recursion was hard as well. When I was learning recursion it was during school with Java. I found it more often I would use recursion over iterators as they were annoying to write in Java. However, I learned Ruby and I found myself writing recursive methods less and less. Then, I learned Elixir and Erlang and found myself writing a lot of recursive functions. My point? Some tools will give themselves for writing with certain style.
Now to answer your questions, since you're just starting to learn recursion, I would suggest diving deep into them and trying to get comfortable with them writing them as much as you can.
Certain tasks are much better off with recursion (e.g. Fibonacci sequence, traversing trees, etc..). Some other's you're better off writing a simple loop. However, note that you can write any recursive method with a loop. It might get tricky on certain occasions though.
All in all, recursion is actually a pretty cool concept once you get the hang of it.
Take a look at this question that relates to recursion: Erlang exercise, creating lists
I'd go for a study of some well known recursive algorithms. For instance, you could try to implement a factorial computation, or to get all the paths lengths in a tree.
By doing that you'll (hopefully) see how the recursive approach helps to simplify the code, and why it is a good approach in these particular cases. This could give you some ideas for future applications :)
Here seems to be the two biggest things I can take from the How to Design Programs (simplified Racket) course I just finished, straight from the lecture notes of the course:
1) Tail call optimization, and the lack thereof in non-functional languages:
Sadly, most other languages do not support TAIL CALL
OPTIMIZATION. Put another way, they do build up a stack
even for tail calls.
Tail call optimization was invented in the mid 70s, long
after the main elements of most languages were developed.
Because they do not have tail call optimization, these
languages provide a fixed set of LOOPING CONSTRUCTS that
make it possible to traverse arbitrary sized data.
a) What are the equivalents to this type of optimization in procedural languages that don't feature it?
b) Do using those equivalents mean we avoid building up a stack in similar situations in languages that don't have it?
2) Mutation and multicore processors
This mechanism is fundamental in almost any other language you
program in. We have delayed introducing it until now for
several reasons:
despite being fundamental, it is surprisingly complex
overuse of it leads to programs that are not amenable
to parallelization (running on multiple processors).
Since multi-core computers are now common, the ability
to use mutation only when needed is becoming more and
more important
overuse of mutation can also make it difficult to
understand programs, and difficult to test them well
But mutable variables are important, and learning this mechanism
will give you more preparation to work with Java, Python and many
other languages. Even in such languages, you want to use a style
called "mostly functional programming".
I learned some Java, Python and C++ before taking this course, so came to take mutation for granted. Now that has been all thrown in the air by the above statement. My questions are:
a) where could I find more detailed information regarding what is suggested in the 2nd bullet, and what to do about it, and
b) what kind of patterns would emerge from a "mostly functional programming" style, as opposed to a more careless style I probably would have had had I continued on with those other languages instead of taking this course?
As Leppie points out, looping constructs manage to recover the space savings of proper tail calling, for the particular kinds of loops that they support. The only problem with looping constructs is that the ones you have are never enough, unless you just hurl the ball into the user's court and force them to model the stack explicitly.
To take an example, suppose you're traversing a binary tree using a loop. It works... but you need to explicitly keep track of the "ones to come back to." A recursive traversal in a tail-calling language allows you to have your cake and eat it too, by not wasting space when not required, and not forcing you to keep track of the stack yourself.
Your question on parallelism and concurrency is much more wide-open, and the best pointers are probably to areas of research, rather than existing solutions. I think that most would agree that there's a crisis going on in the computing world; how do we adapt our mutation-heavy programming skills to the new multi-core world?
Simply switching to a functional paradigm isn't a silver bullet here, either; we still don't know how to write high-level code and generate blazing fast non-mutating run-concurrently code. Lots of folks are working on this, though!
To expand on the "mutability makes parallelism hard" concept, when you have multiple cores going, you have to use synchronisation if you want to modify something from one core and have it be seen consistently by all the other cores.
Getting synchronisation right is hard. If you over-synchronise, you have deadlocks, slow (serial rather than parallel) performance, etc. If you under-synchronise, you have partially-observed changes (where another core sees only a portion of the changes you made from a different core), leaving your objects observed in an invalid "halfway changed" state.
It is for that reason that many functional programming languages encourage a message-queue concept instead of a shared state concept. In that case, the only shared state is the message queue, and managing synchronisation in a message queue is a solved problem.
a) What are the equivalents to this type of optimization in procedural languages that don't feature it? b) Do using those equivalents mean we avoid building up a stack in similar situations in languages that don't have it?
Well, the significance of a tail call is that it can evaluate another function without adding to the call stack, so anything that builds up the stack can't really be called an equivalent.
A tail call behaves essentially like a jump to the new code, using the language trappings of a function call and all the appropriate detail management. So in languages without this optimization, you'd use a jump within a single function. Loops, conditional blocks, or even arbitrary goto statements if nothing else works.
a) where could I find more detailed information regarding what is suggested in the 2nd bullet, and what to do about it
The second bullet sounds like an oversimplification. There are many ways to make parallelization more difficult than it needs to be, and overuse of mutation is just one.
However, note that parallelization (splitting a task into pieces that can be done simultaneously) is not entirely the same thing as concurrency (having multiple tasks executed simultaneously that may interact), though there's certainly overlap. Avoiding mutation is incredibly helpful in writing concurrent programs, since immutable data avoids a lot of race conditions and resource contention that would otherwise be possible.
b) what kind of patterns would emerge from a "mostly functional programming" style, as opposed to a more careless style I probably would have had had I continued on with those other languages instead of taking this course?
Have you looked at Haskell or Clojure? Both are heavily inclined to a very functional style emphasizing controlled mutation. Haskell is more rigorous about it but has a lot of tools for working with limited forms of mutability, while Clojure is a bit more informal and might be more familiar to you since it's another Lisp dialect.
I was reading the book, Structure and Interpretation of Computer Programs, where in it tells about the distinction between a recursive procedure and recursive process, and similarly between iterative procedure and iterative process. So, a recursive procedure could still generate an iterative process.
My question is: given a procedure which generates a recursive process, can you always write another procedure that achieves the same result but generates an iterative process?
The specific problem that I was trying to solve was to write a procedure which does an in-order traversal of a binary search tree but generates an iterative process. I know how you can use a stack to get an iterative procedure for this problem. However, that still generates a recursive process (correct me if I am wrong here).
Thanks,
Abhinav.
Some tasks are truly impossible to solve with linear iterative processes (e.g. tree recursion, which is impossible to convert to tail recursion). You either have to use the stack built into your platform, or re-create it yourself within the language (usually a much less efficient and uglier solution).
So if you define 'recursion' as 'using a stack to store different invocations of the same code', then yes, recursion sometimes is absolutely required.
If you define 'recursion' as 'a function in my language (eventually) calling itself', then you can get by without explicit recursion by re-implementing recursiveness yourself, as describes above. This is only useful if your language doesn't provide recursive procedures, or not enough stack space, or has similar limitations. (For instance, early Fortran's didn't have recursive procedures. Of course, they also didn't have the dynamic data structures that you would need to simulate them! Personally, I have never come across an actual example where implementing pseudo-recursion was the right solution.)
Read this former SO post:
Design patterns for converting recursive algorithms to iterative ones
there are a lot of good answers there which may help you further.
Any tail recursive process can be transformed into an iterative one.
But not all recursive processes can be transformed into an iterative one.
I am currently working in PHP, so this example will be in PHP, but the question applies to multiple languages.
I am working on this project with a fiend of mine, and as always we were held up by a big problem. Now we both went home, couldn't solve the problem. That night we both found the solution, only I used a loop to tackle the problem, and he used recursion.
Now I wanted to tell him the difference between the loop and recursion, but I couldn't come up with a solution where you need recursion over a normal loop.
I am going to make a simplified version of both, I hope someone can explain how one is different from the other.
Please forgive me for any coding errors
The loop:
printnumbers(1,10);
public function printnumbers($start,$stop)
{
for($i=$start;$i<=$stop;$i++)
{
echo $i;
}
}
Now the code above just simply prints out the numbers.
Now let's do this with recursion:
printnumbers(1,10);
public function printnumbers($start,$stop)
{
$i = $start;
if($i <= $stop)
{
echo $i;
printnumbers($start+1,$stop);
}
}
This method above will do the exact same thing as the loop, but then only with recursion.
Can anyone explain to me what there is different about using one of these methods.
Loops and recursions are in many ways equivalent. There are no programs the need one or the other, in principle you can always translate from loops to recursion or vice versa.
Recursions is more powerful in the sense that to translating recursion to a loop might need a stack that you have to manipulate yourself. (Try traversing a binary tree using a loop and you will feel the pain.)
On the other hand, many languages (and implementations), e.g., Java, don't implement tail recursion properly. Tail recursion is when the last thing you do in a function is to call yourself (like in your example). This kind of recursion does not have to consume any stack, but in many languages they do, which means you can't always use recursion.
Often, a problem is easier expressed using recursion. This is especially true when you talk about tree-like data structures (e.g. directories, decision trees...).
These data structures are finite in nature, so most of the time processing them is clearer with recursion.
When stack-depth is often limited, and every function call requires a piece of stack, and when talking about a possibly infinite data structure you will have to abandon recursion and translate it into iteration.
Especially functional languages are good at handling 'infinite' recursion. Imperative languages are focused on iteration-like loops.
In general, a recursive function will consume more stack space (since it's really a large set of function calls), while an iterative solution won't. This also means that an iterative solution, in general, will be faster because.
I am not sure if this applies to an interpreted language like PHP though, it is possible that the interpreter can handle this better.
A loop will be faster because there's always overhead in executing an extra function call.
A problem with learning about recursion is a lot of the examples given (say, factorials) are bad examples of using recursion.
Where possible, stick with a loop unless you need to do something different. A good example of using recursion is looping over each node in a Tree with multiple levels of child nodes.
Recursion is a bit slower (because function calls are slower than setting a variable), and uses more space on most languages' call stacks. If you tried to printnumbers(1, 1000000000), the recursive version would likely throw a PHP fatal error or even a 500 error.
There are some cases where recursion makes sense, like doing something to every part of a tree (getting all files in a directory and its subdirectories, or maybe messing with an XML document), but it has its price -- in speed, stack footprint, and the time spent to make sure it doesn't get stuck calling itself over and over til it crashes. If a loop makes more sense, it's definitely the way to go.
Well, I don't know about PHP but most languages generate a function call (at the machine level) for every recursion. So they have the potential to use a lot of stack space, unless the compiler produces tail-call optimizations (if your code allows it).
Loops are more 'efficient' in that sense because they don't grow the stack. Recursion has the advantage of being able to express some tasks more naturally though.
In this specific case, from a conceptual (rather than implementative) point of view, the two solutions are totally equivalent.
Compared to loops, a function call has its own overhead like allocating stack etc. And in most cases, loops are more understandable than their recursive counterparts.
Also, you will end up using more memory and can even run out of stack space if the difference between start and stop is high and there are too many instances of this code running simultaneously (which can happen as you get more traffic).
You don't really need recursion for a flat structure like that. The first code I ever used recursion in involved managing physical containers. Each container might contain stuff (a list of them, each with weights) and/or more containers, which have a weight. I needed the total weight of a container and all it held. (I was using it to predict the weight of large backpacks full of camping equipment without packing and weighing them.) This was easy to do with recursion and would have been a lot harder with loops. But many kinds of problems that naturally suit themselves to one approach can also be tackled with the other.
Stack overflow.
And no, I don't mean a website or something. I MEAN a "stack overflow".
I am trying to do work with examples on Trees as given here: http://cslibrary.stanford.edu/110/BinaryTrees.html
These examples all solve problems via recursion, I wonder if we can provide a iterative solution for each one of them, meaning, can we always be sure that a problem which can be solved by recursion will also have a iterative solution, in general. If not, what example can we give to show a problem which can be solved only by recursion/Iteration?
--
The only difference between iteration and recursion on a computer is whether you use the built-in stack or a user-defined stack. So they are equivalent.
In my experience, most recursive solution can indeed be solved iteratively.
It is also a good technique to have, as recursive solutions may have too large an overhead in memory and CPU consumptions.
Since recursion uses an implicit stack on which it stores information about each call, you can always implement that stack yourself and avoid the recursive calls. So yes, every recursive solution can be transformed into an iterative one.
Read this question for a proof.
Recursion and iteration are two tools that, at a very fundamental level, do the same thing: execute a repeated operation over a defined set of values. They are interchangeable in that there is no problem that cannot, in some way, be solved by only one of them. That does not mean, however, that one cannot be more suited than the other.
Recursion has the advantage where it will continue without a known end. A perfect example of this is a tuned and threaded Quick Sort.
You can't spawn additional loops, but you can spawn new threads via recursion.
As an "old guy," I fall back to my memory of learning that recursive descent parsers are easier to write, but that stack-based, iterative parsers perform better. Here's an article that seems to support that idea with metrics:
http://www.texttoolkit.com/index.php?option=com_content&view=article&catid=35%3Atechnology&id=60%3Abeyond-recursive-descent&Itemid=55
One thing to note is the author's mention of overrunning the call stack with recursive descent. An iterative, stack-based implementation can be much more efficient of resources.