Why does Clojure have the "recur" special form?
When I replace the "recur" with the function itself I get the same result:
(defn print-down-from [x]
(when (pos? x)
(println x)
(recur (dec x))
)
)
(print-down-from 5)
Has the same result as
(defn print-down-from [x]
(when (pos? x)
(println x)
(print-down-from (dec x))
)
)
(print-down-from 5)
I was wondering if "recur" is just a safety measure so that the compiler will throw an error if a developer happens to use a non-tail recursion. Or maybe it's necessary for the compiler to optimize the tail recursion?
But what I'm wondering most is about any other reasons other than stack consumption.
As explained in the clojure.org page on functional programming:
In the absence of mutable local variables, looping and iteration must take a different form than in languages with built-in for or while constructs that are controlled by changing state. In functional languages looping and iteration are replaced/implemented via recursive function calls. Many such languages guarantee that function calls made in tail position do not consume stack space, and thus recursive loops utilize constant space. Since Clojure uses the Java calling conventions, it cannot, and does not, make the same tail call optimization guarantees. Instead, it provides the recur special operator, which does constant-space recursive looping by rebinding and jumping to the nearest enclosing loop or function frame. While not as general as tail-call-optimization, it allows most of the same elegant constructs, and offers the advantage of checking that calls to recur can only happen in a tail position.
When you don't use recur (or trampoline), your function calls consume stack: Thus, if you ran (print-down-from 100000), you would quickly observe a crash.
Providing this language facility, then, offers several benefits:
Contrary to conventional recursive calls (as given in the question's example), using recur does not consume stack.
The author knows that TCO is in use (and stack is not consumed), as use in a position where TCO is not possible would cause a compile-time failure. As such, the stack-consumptions characteristics are obvious both to the code's author and its readers, unlike languages with only automatic TCO (where one would need to read carefully -- taking macro expansions into account -- to determine whether a call is genuinely in tail position before knowing whether it is optimized).
Compatibility with conventional JVM calling conventions, and thus native interoperability with code written in other JVM-centric languages is preserved.
Finally, for background, see one of several other questions on the topic on StackOverflow:
Automatic TCO in Clojure (asking whether an automatic facility to make explicit invocation of recur unnecessary is available).
Why does TCO require support from the VM? (asking for the basis behind the claim made in the above-quoted clojure.org documentation that automatic TCO cannot be implemented without either JVM support or breaking use of JVM calling conventions and thus harming interoperability).
Why can't tail calls be optimized in JVM-based Lisps? (likewise, more generally)
As I understand it, the loop .. recur uses tail-end recursion, so your program doesn't blow the stack, and regular recursion does not. Some solutions to problems on 4clojure wind up not using loop .. recur, because -- I'm taking an educated guess -- the solution can only be derived using a direct, recursive function call, instead of loop .. recur.
From what I have read, in some of the Clojure books from a few years ago, you should feel free to use loop .. recur.
However, at least from the discussion in those books I have read and from answers I have received to my Clojure questions here in SO, there is some general consensus to try to solve your problem first using constructs like map. If that is not possible, then by all means go ahead and use loop .. recur, and if you do not think there is a possibility of blowing your stack, a direct recursion call.
Related
I tried using the y-combinator (in both Lua and Clojure) as I thought that would allow me to exceed the size of default stack implementations when using recursion. It seems I was mistaken. Yes, it works, but in both of these systems, the stack blows at exactly the same point as it does using plain old recursion. A lowish ~3600 in Clojure and a highish ~333000 on my Android Lua implementation. It is also a bit slower than regular recursion.
So is there anything to be gained by using the y-combinator, or is it just an intellectual exercise to prove a point? Have I missed something?
===
PS. Sorry, I should have made it clearer that I am aware I can use TCO to exceed the stack. My question does not relate to that. I am interested in this
a) from the academic/intellectual point of view
b) whether there is anything that can be done about those function that cannot be written tail recursively.
The Y combinator allows a non-recursive function to be used recursively, but that recursion still consumes stack space through nested function invocations.
For functions that can't be made tail-recursive, you could try refactoring them using continuation passing style, which would consume heap space instead of stack.
Here's a good overview of the topic: https://www.cs.cmu.edu/~15150/previous-semesters/2012-spring/resources/lectures/11.pdf
A “tail call” will allow you to exceed any stack size limitations. See Programming in Lua, section 6.3: Proper Tail Calls:
...after the tail call, the program does not need to keep any information about the calling function in the stack. Some language implementations, such as the Lua interpreter, take advantage of this fact and actually do not use any extra stack space when doing a tail call. We say that those implementations support proper tail calls.
If you haven't seen it yet, there is a good explanation here: What is a Y-combinator?
In summary, it helps to prove that the lambda calculus is Turing complete, but is useless for normal programming tasks.
As you are probably aware, in Clojure you would just use loop/recur to implement a loop that does not consume the stack.
Can functions that take non-pure functions as arguments be considered pure if they do not change/store state directly, don't reference global variables etc?
Where and how do we draw the line on what is pure and what is not, is it solely on the function's in question code or do we take into account the effects of invoking the arguments?
E.g. imagine this scenario, where the pure function represents a stateless workflow and takes as arguments a few actions to be executed during this workflow. One of these actions changes some state somewhere. So if I'm looking strictly at the implementation of my workflow, it seems pure, but eventually it does modify state by invoking this parameter function that modifies state.
I'm tempted to speculate the workflow is also non-pure, but passing in a different argument that does not change state will make it pure, so I'm confused.
Any help will be much appreciated. Thank you.
(define (its-not-me launch-rockets)
(lambda ()
(launch-rockets)))
is indeed pure. It doesn't launch rockets, it just constructs a computation that will, if called. But no side effects are caused by merely constructing such computation.
(define (it-is-me launch-rockets)
(launch-rockets)
((its-not-me launch-rockets)))
does indeed launches the rockets, twice. Whether directly or indirectly, doesn't matter. If it causes side effects during its execution, it causes side effects during its execution.
And if we have
(define (launch-rockets)
(if (prime? (random-integer))
(do-actually-launch-rockets)
(list 'do-nothing)))
nothing changes. The first is still pure, constructing a computation which is potentially causing side-effects, is impure.
Even
(define (launch-rockets)
(if (prime? (random-integer))
(list 'i-am-only)
(list 'kidding)))
isn't a pure function, as it uses a non-pure effect, the randomness. Pure functions always return same result for the same arguments.
is Recursion is more general than Iteration?
for me Iteration means repetitive control using language constructs other than subroutine calls (like loop-constructs and/or explicit
goto’s), whereas recursion means as repetitive control obtained using subroutine calls). which is more general in these two?
Voted to close as likely to prompt opinion-based responses; my opinion-based response is: recursion is more general because:
it's simply a user-case of function calls, which a language will already have; and
recursion captures both a direct one-to-one pattern of repetition and more complicated patterns, such as tree traversal, divide and conquer, etc.
Conversely iteration tends to be a specific language-level construct allowing only a direct linear traversal.
In so far as it is not opinion-based, the most reasonable answer is that neither recursion nor iteration is more general than the other. A language can be Turing complete with recursion but no iteration (minimal Lisps are like that) and a language can also be Turing complete with iteration but no recursion (earlier versions of Fortran didn't support recursion). Both recursion and iteration are widely used. Iteration is probably more commonly used since for every person who learns programming with something like Lisp or Haskell there are probably a dozen who learn programming with things like Java or Visual Basic -- but I don't think that "most commonly used" is a good synonym for "general".
As far as I can tell, Clojure's recur is backed by the compiler whereas in other lisps it is implemented at a lower level.
As I read, this wouldn't be a "general" TCO. Aside from the obvious (a keyword + checking are needed), is in any way recur less powerful?
recur only supports tail-recursion optimization, which is a subclass of general TCO. Clojure also supports mutual or indirect recursion through trampoline.
EDIT
Also, I think general TCO was expected to land in JVM with Java 7 and recur was meant as a temporary solution. Then Oracle happened. I've mixed that with Project Lambda's (adding closures in Java) schedule
recur differs slightly from full TCO in that recur works with both loops and functions and does not do some of the things that a full implementation of TCO would. The philosophical backing for this is to make the special part look special as opposed to silently optimizing a uniform syntax.
With respect to using recursion over non-recursive methods in sorting algorithms or, for that matter, any algorithm what are its pros and cons?
For the most part recursion is slower, and takes up more of the stack as well. The main advantage of recursion is that for problems like tree traversal it make the algorithm a little easier or more "elegant".
Check out some of the comparisons:
link
Recursion means a function calls repeatedly
It uses system stack to accomplish its task. As stack uses LIFO approach
and when a function is called the controlled is moved to where function is defined which has it is stored in memory with some address, this address is stored in stack
Secondly, it reduces a time complexity of a program.
Though bit off-topic,a bit related. Must read. : Recursion vs Iteration
All algorithms can be defined recursively. That makes it much, much easier to visualize and prove.
Some algorithms (e.g., the Ackermann Function) cannot (easily) be specified iteratively.
A recursive implementation will use more memory than a loop if tail call optimization can't be performed. While iteration may use less memory than a recursive function that can't be optimized, it has some limitations in its expressive power.
Any algorithm implemented using recursion can also be implemented using iteration.
Why not to use recursion
It is usually slower due to the overhead of maintaining the stack.
It usually uses more memory for the stack.
Why to use recursion
Recursion adds clarity and (sometimes) reduces the time needed to write and debug code (but doesn't necessarily reduce space requirements or speed of execution).
Reduces time complexity.
Performs better in solving problems based on tree structures.
For example, the Tower of Hanoi problem is more easily solved using recursion as opposed to iteration.
I personally prefer using Iterative over recursive function. Especially if you function has complex/heavy logic and number of iterations are large. This because with every recursive call call stack increases. It could potentially crash the stack if you operations are too large and also slow up process.
To start:
Pros:
It is the unique way of implementing a variable number of nested loops (and the only elegant way of implementing a big constant number of nested loops).
Cons:
Recursive methods will often throw a StackOverflowException when processing big sets. Recursive loops don't have this problem though.
To start :
Recursion:
A function that calls itself is called as recursive function and this technique is called as recursion.
Pros:
1. Reduce unnecessary calling of functions.
2. Through Recursion one can solve problems in easy way while its iterative solution is very big and complex.
3. Extremely useful when applying the same solution.
Cons:
1. Recursive solution is always logical and it is very difficult to trace.
2. In recursive we must have an if statement somewhere to force the function to return without the recursive call being executed, otherwise the function will never return.
3. Recursion uses more processor time.
Expressiveness
Most problems are naturally expressed by recursion such as Fibonacci, Merge sorting and quick sorting. In this respect, the code is written for humans, not machines.
Immutability
Iterative solutions often rely on varying temporary variables which makes the code hard to read. This can be avoided with recursion.
Performance
Recursion is not stack friendly. Stack can overflow when the recursion is not well designed or tail optimization is not supported.
Some situation would arise where you would have to abandon recursion in a problem where recursion appears to be to your advantage, this is because for problems where your recursion would have to occur thousand of times this would result in a stackoverflow error even though your code did not get stuck in an infinite recursion. Most programming languages limits you to a number of stack calls, so if your recursion goes beyond this limit, then you might consider not using recursion.
We should use recursion in following scenarios:
when we don't know the finite number of iteration for example our fuction exit condition is based on dynamic programming (memoization)
when we need to perform operations on reverse order of the elements. Meaning we want to process last element first and then n-1, n-2 and so on till first element
Recursion will save multiple traversals. And it will be useful, if we can divide the stack allocation like:
int N = 10;
int output = process(N) + process(N/2);
public void process(int n) {
if (n==N/2 + 1 || n==1) {
return 1;
}
return process(n-1) + process(n-2);
}
In this case only half stacks will be allocated at any given time.
Recursion gets a bad rep, I'm always surprised by the number of developers that wont even touch recursion because someone told them it was evil incarnate.
I've learned through trial and error that when done properly recursion can be one of the fastest ways to iterate over something, it is not a steadfast rule and each language/ compiler/ engine has it's own quirks so mileage will vary.
In javascript I can reliably speed up almost any iterative process by introducing recursion with the added benefit of reducing side effects and making the code more clear concise and reusable. Also pro tip its possible to get around the stack overflow issue (and no you dont disable the warning).
My personal Pros & Cons:
Pros:
- Reduces side effects.
- Makes code more concise and easier to reason about.
- Reduces system resource usage and performs better than the traditional for loop.
Cons:
- Can lead to stack overflow.
- More complicated to setup than a traditional for loop.
Mileage will vary depending on language/ complier/ engine.