I have been experimenting with continuation-passing style, as I may need to deal with some non tail-recursive functions soon. Good technique to know, in any case! I wrote a test function, both in Lua and in Clojure; ran the Lua one on my little Android handheld.
The Lua version seems to have worked fine, Lua's stack already has a depth of about 300000, but with CPS, I was easily able to do over 7000000 iterations before the system blew up, probably out of lack of memory, rather than any limitation of the CPS/Lua combination.
The Clojure attempt fared less well. With little over 1000 iterations it was complaining of blown stack, it can do better just with plain iteration, which has a stack of about 1600, iirc.
Any ideas what might be the problem? Something inherent to the JVM, perhaps, or just some silly noob error? (Oh, BTW, the test function, sigma(log) was chosen because it grows slowly, and Lua does not support bignums on Android)
All ideas, hints, suggestions most welcome.
The Clojure code:
user=> (defn cps2 [op]
#_=> (fn [a b k] (k (op a b))))
#'user/cps2
user=> (defn cps-sigma [n k]
#_=> ((cps2 =) n 1 (fn [b]
#_=> (if b ; growing continuation
#_=> (k 0) ; in the recursive call
#_=> ((cps2 -) n 1 (fn [nm1]
#_=> (cps-sigma nm1 (fn [f]
#_=> ((cps2 +) (Math/log n) f k)))))))))
#'user/cps-sigma
user=> (cps-sigma 1000 identity)
5912.128178488171
user=> (cps-sigma 1500 identity)
StackOverflowError clojure.lang.Numbers.equal (Numbers.java:216)
user=>
===================
PS. After experimenting a bit, I tried the code I mention in my third comment, below
(defn mk-cps [accept? end-value kend kont]
(fn [n]
((fn [n k]
(let [cont (fn [v] (k (kont v n)))]
(if (accept? n)
(k end-value)
(recur (dec n) cont))))
n kend)))
(def sigmaln-cps (mk-cps zero? 0 identity #(+ %1 (Math/log %2))))
user=> (sigmaln-cps 11819) ;; #11819 iterations first try
StackOverflowError clojure.lang.RT.doubleCast (RT.java:1312)
That's obviously better, by an order, however I still think it's way too low. Technically it should be limited only by memory, yes?
I mean the toy Lua system, on a toy Android tablet did over 7000000...
Clojure has the trampoline function that can remove a lot of the confusing plumbing involved in this problem:
(defn sigma [n]
(letfn [(sig [curr n]
(if (<= n 1)
curr
#(sig (+ curr (Math/log n)) (dec n))))]
(trampoline sig 0 n)))
(sigma 1000)
=> 5912.128178488164
(sigma 1500)
=> 9474.406184917756
(sigma 1e7) ;; might take a few seconds
=> 1.511809654875759E8
The function you pass to trampoline can either return a new function, in which case the trampoline continues "bouncing", or a non-function value which would be a "final" value. This example doesn't involve mutually recursive functions, but those are also doable with trampoline.
Related
im currently writing a compiler in OCaml for a subset of scheme and am having trouble understanding how to compile with continuations. I found some great resources, namely:
The cps slides of the cmsu compiler course:
https://www.cs.umd.edu/class/fall2017/cmsc430/
This explanation of another cs course:
https://www.cs.utah.edu/~mflatt/past-courses/cs6520/public_html/s02/cps.pdf
Matt Mights posts on a-normal form and cps:
http://matt.might.net/articles/a-normalization/ and
http://matt.might.net/articles/cps-conversion/
Using the anormal transformation introduced in the anormal-paper, I now have code where function calls are either bound to a variable or returned.
Example:
(define (fib n)
(if (<= n 1)
n
(+ (fib (- n 1))
(fib (- n 2)))))
becomes:
(define (fib n)
(let ([c (<= n 1)])
(if c
n
(let ([n-1 (- n 1)])
(let ([v0 (fib n-1)])
(let ([n-2 (- n 2)])
(let ([v1 (fib n-2)])
(+ v0 v1)))))))
In order to cps-transform, I now have to:
add cont-parameters to all non-primitive functions
call the cont-parameter on tail-positions
transform all non-primitive function calls, so that they escape the let-binding and become an extra lambda with the previous let-bound variable as sole argument and the previous let-body
as the body
The result would look like:
(define (fib n k)
(let ([c (<= n 1)])
(if c
(k n)
(let ([n-1 (- n 1)])
(fib n-1
(lambda (v0)
(let ([n-2 (- n 2)])
(fib n-2
(lambda (v1)
(k (+ v0 v1))))))))))
Is this correct?
The csmu-course also talks about how Programs in CPS require no stack and never return. Is that because we don't need to to save the adresses to return to and closures as well as other datatypes are stored on the heap and references are kept alive by using the closures?
The csmu also talks about desugaring of call/cc:
(call/cc) => ((lambda (k f) (f k k)))
when using such desugaring, how does:
(+ 2 (call/cc (lambda (k) (k 2))))
in MIT-Scheme return 4, since the current continuation would probably be something like display?
is this correct?
(define (fib n k)
(let ([c (<= n 1)])
(if c
(k n)
(let ([n-1 (- n 1)])
(fib n-1
(lambda (v0)
(let ([n-2 (- n 2)])
(fib n-2
(lambda (v1)
(k (+ v0 v1))))))))))
you get an A+ đź’Ż
The csmu-course also talks about how Programs in CPS require no stack and never return. Is that because we don't need to to save the addresses to return to and closures as well as other datatypes are stored on the heap and references are kept alive by using the closures?
Exactly! See Chicken Complilation Process for an in-depth read about such a technique.
The csmu also talks about desugaring of call/cc:
(call/cc) => ((lambda (k f) (f k k)))
That doesn't look quite right. Here's a desugar of call/cc from Matt Might -
call/cc => (lambda (f cc) (f (lambda (x k) (cc x)) cc))
The essence of the idea of compiling with continuations is that you want to put an order on the evaluation of arguments passed to each function and after you evaluate that argument you send its value to the continuation passed.
It is required for the language in which you rewrite the code in CPS form to be tail recursive, otherwise it will stack empty frames, followed only by a return. If the implementation language does not impose tail-recursion you need to apply more sophisticated methods to get non-growing stack for cps code.
Take care, if you do it, you also need to change the signature of the primitives. The primitives will also be passed a continuation but they return immediately the answer in the passed continuation, they do not create other continuations.
The best reference about understanding how to compile with continuations remains the book of Andrew W. Appel and you need nothing more.
I have a problem in understanding the performance of a Common Lisp function (I am still a novice). I have two versions of this function, which simply computes the sum of all integers up to a given n.
Non-tail-recursive version:
(defun addup3 (n)
(if (= n 0)
0
(+ n (addup (- n 1)))))
Tail-recursive version:
(defun addup2 (n)
(labels ((f (acc k)
(if (= k 0)
acc
(f (+ acc k) (- k 1)))))
(f 0 n)))
I am trying to run these functions in CLISP with input n = 1000000. Here is the result
[2]> (addup3 1000000)
500000500000
[3]> (addup2 1000000)
*** - Program stack overflow. RESET
I can run both successfully in SBCL, but the non-tail-recursive one is faster (only by a little, but that seems strange to me). I've scoured Stackoverflow questions for answers but couldn't find something similar. Why do I get a stack overflow although the tail-recursive function is designed NOT to put all recursive function calls on the stack? Do I have to tell the interpreter/compiler to optimise tail calls? (I read something like (proclaim '(optimize (debug 1)) to set the debug level and optimize at the cost of tracing abilities, but I don't know what this does).
Maybe the answer is obvious and the code is bullshit, but I just can't figure it out.
Help is appreciated.
Edit: danlei pointed out the typo, it should be a call to addup3 in the first function, so it is recursive. If corrected, both versions overflow, but not his one
(defun addup (n)
"Adds up the first N integers"
(do ((i 0 (+ i 1))
(sum 0 (+ sum i)))
((> i n) sum)))
While it may be a more typical way to do it, I find it strange that tail recursion is not always optimised, considering my instructors like to tell me it's so much more efficient and stuff.
There is no requirement for a Common Lisp implementation to have tail call optimization. Most do, however (I think that ABCL does not, due to limitations of the Java virtual machine).
The documentation of the implementation should tell you what optimization settings should be chosen to have TCO (if available).
It is more idiomatic for Common Lisp code to use one of the looping constructs:
(loop :for i :upto n
:sum i)
(let ((sum 0))
(dotimes (i n)
(incf sum (1+ i))))
(do ((i 0 (1+ i))
(sum 0 (+ sum i)))
((> i n) sum))
In this case, of course, it is better to use the "little GauĂź":
(/ (* n (1+ n)) 2)
Well, your addup3 just isn't recursive at all.
(defun addup3 (n)
(if (= n 0)
0
(+ n (addup (- n 1))))) ; <--
It calls whatever addup is. Trying a corrected version in SBCL:
CL-USER> (defun addup3 (n)
(if (= n 0)
0
(+ n (addup3 (- n 1)))))
ADDUP3
CL-USER> (addup3 100000)
Control stack guard page temporarily disabled: proceed with caution
; ..
; Evaluation aborted on #<SB-SYS:MEMORY-FAULT-ERROR {C2F19B1}>.
As you'd expect.
Using GNU CommonLisp, GCL 2.6.12, compilation of addup2 will optimize tail calls, here is what I got:
>(compile 'addup2)
Compiling /tmp/gazonk_3012_0.lsp.
End of Pass 1.
;; Note: Tail-recursive call of F was replaced by iteration.
End of Pass 2.
OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3
Finished compiling /tmp/gazonk_3012_0.lsp.
Loading /tmp/gazonk_3012_0.o
start address -T 0x9556e8 Finished loading /tmp/gazonk_3012_0.o
#<compiled-function ADDUP2>
NIL
NIL
>>(addup2 1000000)
500000500000
>(addup3 1000000)
Error: ERROR "Invocation history stack overflow."
Fast links are on: do (si::use-fast-links nil) for debugging
Signalled by IF.
ERROR "Invocation history stack overflow."
Broken at +. Type :H for Help.
1 Return to top level.
>>(compile 'addup3)
Compiling /tmp/gazonk_3012_0.lsp.
End of Pass 1.
End of Pass 2.
OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3
Finished compiling /tmp/gazonk_3012_0.lsp.
Loading /tmp/gazonk_3012_0.o
start address -T 0x955a00 Finished loading /tmp/gazonk_3012_0.o
#<compiled-function ADDUP3>
NIL
NIL
>>(addup3 1000000)
Error: ERROR "Value stack overflow."
Hope it helps.
In SBCL User Manual:
The compiler is “properly tail recursive.” [...] The elimination of tail-recursive frames can be prevented by disabling tail-recursion optimization, which happens when the debug optimization quality is greater than 2.
And works as is in the REPL of a fresh image:
(defun sum-no-tail (n)
(if (zerop n)
0
(+ n (sum-no-tail (- n 1)))))
(defun sum-tail (n &key (acc 0))
(if (zerop n)
acc
(sum-tail (- n 1) :acc (+ n acc))))
CL-USER> (sum-no-tail 10000)
50005000 (26 bits, #x2FB0408)
CL-USER> (sum-no-tail 100000)
Control stack guard page temporarily disabled: proceed with caution
; Debugger entered on #<SB-KERNEL::CONTROL-STACK-EXHAUSTED {10026620A3}>
[1] CL-USER>
; Evaluation aborted on #<SB-KERNEL::CONTROL-STACK-EXHAUSTED {10026620A3}>
CL-USER> (sum-tail 100000)
5000050000 (33 bits, #x12A06B550)
CL-USER> (sum-tail 1000000)
500000500000 (39 bits, #x746A5A2920)
CL-USER> (sum-tail 10000000)
50000005000000 (46 bits, #x2D7988896B40)
Hope it helps in SBCL.
How do I do recursion in an anonymous function, without using tail recursion?
For example (from Vanderhart 2010, p 38):
(defn power
[number exponent]
(if (zero? exponent)
1
(* number (power number (- exponent 1)))))
Let's say I wanted to do this as an anonymous function. And for some reason I didn't want to use tail recursion. How would I do it? For example:
( (fn [number exponent] ......))))) 5 3)
125
Can I use loop for this, or can loop only be used with recur?
The fn special form gives you the option to provide a name that can be used internally for recursion.
(doc fn)
;=> (fn name? [params*] exprs*)
So, add "power" as the name to complete your example.
(fn power [n e]
(if (zero? e)
1
(* n (power n (dec e)))))
Even if the recursion happened in the tail position, it will not be optimized to replace the current stack frame. Clojure enforces you to be explicit about it with loop/recur and trampoline.
I know that in Clojure there's syntactic support for "naming" an anonymous function, as other answers have pointed out. However, I want to show a first-principles approach to solve the question, one that does not depend on the existence of special syntax on the programming language and that would work on any language with first-order procedures (lambdas).
In principle, if you want to do a recursive function call, you need to refer to the name of the function so "anonymous" (i.e. nameless functions) can not be used for performing a recursion ... unless you use the Y-Combinator. Here's an explanation of how it works in Clojure.
Let me show you how it's used with an example. First, a Y-Combinator that works for functions with a variable number of arguments:
(defn Y [f]
((fn [x] (x x))
(fn [x]
(f (fn [& args]
(apply (x x) args))))))
Now, the anonymous function that implements the power procedure as defined in the question. Clearly, it doesn't have a name, power is only a parameter to the outermost function:
(fn [power]
(fn [number exponent]
(if (zero? exponent)
1
(* number (power number (- exponent 1))))))
Finally, here's how to apply the Y-Combinator to the anonymous power procedure, passing as parameters number=5 and exponent=3 (it's not tail-recursive BTW):
((Y
(fn [power]
(fn [number exponent]
(if (zero? exponent)
1
(* number (power number (- exponent 1)))))))
5 3)
> 125
fn takes an optional name argument that can be used to call the function recursively.
E.g.:
user> ((fn fact [x]
(if (= x 0)
1
(* x (fact (dec x)))))
5)
;; ==> 120
Yes you can use loop for this. recur works in both loops and fns
user> (loop [result 5 x 1] (if (= x 3) result (recur (* result 5) (inc x))))
125
an idomatic clojure solution looks like this:
user> (reduce * (take 3 (repeat 5)))
125
or uses Math.pow() ;-)
user> (java.lang.Math/pow 5 3)
125.0
loop can be a recur target, so you could do it with that too.
For my prime numbers lazy seq, I am checking to see if an index value is divisible by all the primes below that current index (prime?). The problem is, when I call primes within itself (primes within shr-primes line), it only returns the initial value. Is it possible to keep the lazy-seq updated while building it lazily? It seems counter-intuitive to the lazy-seq concept.
(def primes
(cons 2 (for [x (range)
:let [y (-> x (* 2) (+ 3))
root (math/floor (math/sqrt y))
shr-primes (take-while (partial >= root) primes) ;; primes stuck at init value
prime? (every? #(not= % 0) (pmap #(rem y %) shr-primes))]
:when prime?]
y)))
If you're doing the Project Euler problems, I don't want to spoil the exercise for you, but here's how you would define a Fibonacci sequence so that the lazy-seq keeps "updating" itself as it goes:
(defn fib-maker
([] (concat [0 1] (fib 0 1)))
([a b] (lazy-seq (cons b (fib b (+ a b))))))
(def fib (fib-maker))
I've used the above approach to implement the prime number sequence you've outlined above, so if you want more details let me know. Meanwhile, this will hopefully be a helpful hint.
What am I doing wrong? Simple recursion a few thousand calls deep throws a StackOverflowError.
If the limit of Clojure recursions is so low, how can I rely on it?
(defn fact[x]
(if (<= x 1) 1 (* x (fact (- x 1)) )))
user=> (fact 2)
2
user=> (fact 4)
24
user=> (fact 4000)
java.lang.StackOverflowError (NO_SOURCE_FILE:0)
Here's another way:
(defn factorial [n]
(reduce * (range 1 (inc n))))
This won't blow the stack because range returns a lazy seq, and reduce walks across the seq without holding onto the head.
reduce makes use of chunked seqs if it can, so this can actually perform better than using recur yourself. Using Siddhartha Reddy's recur-based version and this reduce-based version:
user> (time (do (factorial-recur 20000) nil))
"Elapsed time: 2905.910426 msecs"
nil
user> (time (do (factorial-reduce 20000) nil))
"Elapsed time: 2647.277182 msecs"
nil
Just a slight difference. I like to leave my recurring to map and reduce and friends, which are more readable and explicit, and use recur internally a bit more elegantly than I'm likely to do by hand. There are times when you need to recur manually, but not that many in my experience.
The stack size, I understand, depends on the JVM you are using as well as the platform. If you are using the Sun JVM, you can use the -Xss and -XThreadStackSize parameters to set the stack size.
The preferred way to do recursion in Clojure though is to use loop/recur:
(defn fact [x]
(loop [n x f 1]
(if (= n 1)
f
(recur (dec n) (* f n)))))
Clojure will do tail-call optimization for this; that ensures that you’ll never run into StackOverflowErrors.
And due defn implies a loop binding, you could omit the loop expression, and use its arguments as the function argument. And to make it a 1 argument function, use the multiary caracteristic of functions:
(defn fact
([n] (fact n 1))
([n f]
(if (<= n 1)
f
(recur (dec n) (* f n)))))
Edit: For the record, here is a Clojure function that returns a lazy sequence of all the factorials:
(defn factorials []
(letfn [(factorial-seq [n fact]
(lazy-seq
(cons fact (factorial-seq (inc n) (* (inc n) fact)))))]
(factorial-seq 1 1)))
(take 5 (factorials)) ; will return (1 2 6 24 120)
Clojure has several ways of busting recursion
explicit tail calls with recur. (they must be truely tail calls so this wont work)
Lazy sequences as mentioned above.
trampolining where you return a function that does the work instead of doing it directly and then call a trampoline function that repeatedly calls its result until it turnes into a real value instead of a function.
(defn fact ([x] (trampoline (fact (dec x) x)))
([x a] (if (<= x 1) a #(fact (dec x) (*' x a)))))
(fact 42)
620448401733239439360000N
memoizing the the case of fact this can really shorten the stack depth, though it is not generally applicable.
ps: I dont have a repl on me so would someone kindly test-fix the trapoline fact function?
As I was about to post the following, I see that it's almost the same as the Scheme example posted by JasonTrue... Anyway, here's an implementation in Clojure:
user=> (defn fact[x]
((fn [n so_far]
(if (<= n 1)
so_far
(recur (dec n) (* so_far n)))) x 1))
#'user/fact
user=> (fact 0)
1
user=> (fact 1)
1
user=> (fact 2)
2
user=> (fact 3)
6
user=> (fact 4)
24
user=> (fact 5)
120
etc.
As l0st3d suggested, consider using recur or lazy-seq.
Also, try to make your sequence lazy by building it using the built-in sequence forms as a opposed to doing it directly.
Here's an example of using the built-in sequence forms to create a lazy Fibonacci sequence (from the Programming Clojure book):
(defn fibo []
(map first (iterate (fn [[a b]] [b (+ a b)]) [0 1])))
=> (take 5 (fibo))
(0 1 1 2 3)
The stack depth is a small annoyance (yet configurable), but even in a language with tail recursion like Scheme or F# you'd eventually run out of stack space with your code.
As far as I can tell, your code is unlikely to be tail recursion optimized even in an environment that supports tail recursion transparently. You would want to look at a continuation-passing style to minimize stack depth.
Here's a canonical example in Scheme from Wikipedia, which could be translated to Clojure, F# or another functional language without much trouble:
(define factorial
(lambda (n)
(let fact ([i n] [acc 1])
(if (zero? i)
acc
(fact (- i 1) (* acc i))))))
Another, simple recursive implementation simple could be this:
(defn fac [x]
"Returns the factorial of x"
(if-not (zero? x) (* x (fac (- x 1))) 1))
To add to Siddhartha Reddy's answer, you can also borrow the Factorial function form Structure And Interpretation of Computer Programs, with some Clojure-specific tweaks. This gave me pretty good performance even for very large factorial calculations.
(defn fac [n]
((fn [product counter max-count]
(if (> counter max-count)
product
(recur (apply *' [counter product])
(inc counter)
max-count)))
1 1 n))
Factorial numbers are by their nature very big. I'm not sure how Clojure deals with this (but I do see it works with java), but any implementation that does not use big numbers will overflow very fast.
Edit: This is without taking into consideration the fact that you are using recursion for this, which is also likely to use up resources.
Edit x2: If the implementation is using big numbers, which, as far as I know, are usually arrays, coupled with recursion (one big number copy per function entry, always saved on the stack due to the function calls) would explain a stack overflow. Try doing it in a for loop to see if that is the problem.