Yesterday i ran into this pipes library for common lisp. It looks to some extent quite like clojure's lazy sequences abstraction, so i decided using it to implement the classic (and classy) clojure example of recursive lazy Fibonacci sequence definition in Common Lisp (for purely educational purpose).
that is what it looks like in clojure:
(def fibs (lazy-cat [0 1] (map +' fibs (rest fibs))))
(nth fibs 100)
;;=> 354224848179261915075N
that is quite simple, but the problem is that it keeps possibly huge lazy sequence in a global scope forever, so with some hacks i rewrote it so it can be used inside let binding:
(let [f (memoize (fn [f]
(lazy-cat [0 1]
(let [data (f f)]
(map +' data (rest data))))))
fibs (f f)]
(nth fibs 100))
;;=> 354224848179261915075N
the whole memoize and (f f) thing is to emulate data recursion in let.
then i've implemented it using the same approach in CL.
first, some utilities:
;; analogue of `list*` for pipes
(defmacro make-pipe* (x1 &rest xs)
(if xs
`(pipes:make-pipe ,x1 (make-pipe* ,#xs))
x1))
;; wraps function so that it always returns the result of its first invocation
(defun once (f)
(let ((called (cons nil nil)))
(lambda (&rest args)
(if (car called)
(cdr called)
(let ((res (apply f args)))
(setf called (cons t res))
res)))))
;; map over two pipes
(defun pipe-map2 (fn pipe1 pipe2)
(if (or (eq pipe1 pipes:+empty-pipe+)
(eq pipe2 pipes:+empty-pipe+))
pipes:+empty-pipe+
(pipes:make-pipe (funcall fn (pipes:pipe-head pipe1) (pipes:pipe-head pipe2))
(pipe-map2 fn (pipes:pipe-tail pipe1) (pipes:pipe-tail pipe2)))))
and then here goes the actual implementation:
(let* ((f (once (lambda (f)
(make-pipe* 0 1
(let ((data (funcall f f)))
(pipe-map2 #'+ data (pipes:pipe-tail data)))))))
(fibs (funcall f f)))
(pipes:pipe-values fibs 10))
;;=> (0 1 1 2 3 5 8 13 21 34 55 . #<CLOSURE (LAMBDA () :IN PIPE-MAP2) {10096C6BBB}>)
ok. it works. But the question is: as common lisp provides much more metaprogramming and compilation control utilities than clojure has, are there any proper ones that could make "self recursive let" (as i call it) more elegant, eliminating the need for ugly hack with memoized function calls, preferably avoiding mutable state (though i'm not sure it is possible at all)?
after some meditation, i've got this solution:
(defmacro letr ((name val) &body body)
(let ((f-name (gensym)))
`(let ((,name (symbol-macrolet ((,name (funcall ,f-name ,f-name)))
(let* ((,f-name (once (lambda (,f-name) ,val))))
,name))))
,#body)))
which is in fact the rewrite of the initial solution by the means of symbol-macrolet
that one can be used this way:
CL-USER> (letr (fibs (make-pipe* 0 1 (pipe-map2 #'+ fibs (pipes:pipe-tail fibs))))
(pipes:pipe-values fibs 10))
;;=> (0 1 1 2 3 5 8 13 21 34 55 . #<CLOSURE (LAMBDA () :IN PIPE-MAP2) {1001D3FCBB}>)
which is expanded into this:
(LET ((FIBS
(SYMBOL-MACROLET ((FIBS (FUNCALL #:G596 #:G596)))
(LET* ((#:G596
(ONCE
(LAMBDA (#:G596)
(CONS 0
#'(LAMBDA ()
(CONS 1
#'(LAMBDA ()
(PIPE-MAP2 #'+ (FUNCALL #:G596 #:G596)
(PIPES:PIPE-TAIL
(FUNCALL #:G596
#:G596)))))))))))
(FUNCALL #:G596 #:G596)))))
(PIPES:PIPE-VALUES FIBS 10))
it is, of course, only usable in quite a narrow field of situations, where the recursive (funcall f f) is delayed, like in this case. otherwise it leads to infinite resursion causing stack overflow. (Though i'm pretty sure it can still be improved somehow)
If you have a recrusive function with 2 arguments then you have to have a singnature like [f arg1 arg2] then using your solution you have to recurse like this (f f arg1 arg2). You can make that thing shorter if you use a helper function and a volatile:
(defn memo [f]
(let [v (volatile! nil)]
(vreset! v (memoize (fn [& args] (apply f #v args))))))
So now you can do:
(let [f (memo (fn [this arg1 arg2] (this arg1 arg2)))] (f arg1 arg2))
So that makes the recurse call 1 argument shorter, that is, you don't have to call to go (f f), just (f).
Related
Consider this simple-minded recursive implementation of comp in Clojure:
(defn my-comp
([f]
(fn [& args]
(apply f args)))
([f & funcs]
(fn [& args]
(f (apply (apply my-comp funcs) args)))))
The right way to do this, I am told, is using recur, but I am unsure how recur works. In particular: is there a way to coax the code above into being recurable?
evaluation 1
First let's visualize the problem. my-comp as it is written in the question will create a deep stack of function calls, each waiting on the stack to resolve, blocked until the the deepest call returns -
((my-comp inc inc inc) 1)
((fn [& args]
(inc (apply (apply my-comp '(inc inc)) args))) 1)
(inc (apply (fn [& args]
(inc (apply (apply my-comp '(inc)) args))) '(1)))
(inc (inc (apply (apply my-comp '(inc)) '(1))))
(inc (inc (apply (fn [& args]
(apply inc args)) '(1))))
(inc (inc (apply inc '(1)))) ; ⚠️ deep in the hole we go...
(inc (inc 2))
(inc 3)
4
tail-recursive my-comp
Rather than creating a long sequence of functions, this my-comp is refactored to return a single function, which when called, runs a loop over the supplied input functions -
(defn my-comp [& fs]
(fn [init]
(loop [acc init [f & more] fs]
(if (nil? f)
acc
(recur (f acc) more))))) ; 🐍 tail recursion
((my-comp inc inc inc) 1)
;; 4
((apply my-comp (repeat 1000000 inc)) 1)
;; 1000001
evaluation 2
With my-comp rewritten to use loop and recur, we can see linear iterative evaluation of the composition -
((my-comp inc inc inc) 1)
(loop 1 (list inc inc inc))
(loop 2 (list inc inc))
(loop 3 (list inc))
(loop 4 nil)
4
multiple input args
Did you notice ten (10) apply calls at the beginning of this post? This is all in service to support multiple arguments for the first function in the my-comp sequence. It is a mistake to tangle this complexity with my-comp itself. The caller has control to do this if it is the desired behavior.
Without any additional changes to the refactored my-comp -
((my-comp #(apply * %) inc inc inc) '(3 4)) ; ✅ multiple input args
Which evaluates as -
(loop '(3 4) (list #(apply * %) inc inc inc))
(loop 12 (list inc inc inc))
(loop 13 (list inc inc))
(loop 14 (list inc))
(loop 15 nil)
15
right-to-left order
Above (my-comp a b c) will apply a first, then b, and finally c. If you want to reverse that order, a naive solution would be to call reverse at the loop call site -
(defn my-comp [& fs]
(fn [init]
(loop [acc init [f & more] (reverse fs)] ; ⚠️ naive
(if (nil? f)
acc
(recur (f acc) more)))))
Each time the returned function is called, (reverse fs) will be recomputed. To avoid this, use a let binding to compute the reversal just once -
(defn my-comp [& fs]
(let [fs (reverse fs)] ; ✅ reverse once
(fn [init]
(loop [acc init [f & more] fs]
(if (nil? f)
acc
(recur (f acc) more))))))
a way to do this, is to rearrange this code to pass some intermediate function back up to the definition with recur.
the model would be something like this:
(my-comp #(* 10 %) - +)
(my-comp (fn [& args] (#(* 10 %) (apply - args)))
+)
(my-comp (fn [& args]
((fn [& args] (#(* 10 %) (apply - args)))
(apply + args))))
the last my-comp would use the first my-comp overload (which is (my-comp [f])
here's how it could look like:
(defn my-comp
([f] f)
([f & funcs]
(if (seq funcs)
(recur (fn [& args]
(f (apply (first funcs) args)))
(rest funcs))
(my-comp f))))
notice that despite of not being the possible apply target, the recur form can still accept variadic params being passed as a sequence.
user> ((my-comp (partial repeat 3) #(* 10 %) - +) 1 2 3)
;;=> (-60 -60 -60)
notice, though, that in practice this implementation isn't really better than yours: while recur saves you from stack overflow on function creation, it would still overflow on application (somebody, correct me if i'm wrong):
(apply my-comp (repeat 1000000 inc)) ;; ok
((apply my-comp (repeat 1000000 inc)) 1) ;; stack overflow
so it would probably be better to use reduce or something else:
(defn my-comp-reduce [f & fs]
(let [[f & fs] (reverse (cons f fs))]
(fn [& args]
(reduce (fn [acc curr-f] (curr-f acc))
(apply f args)
fs))))
user> ((my-comp-reduce (partial repeat 3) #(* 10 %) - +) 1 2 3)
;;=> (-60 -60 -60)
user> ((apply my-comp-reduce (repeat 1000000 inc)) 1)
;;=> 1000001
There is already a good answer above, but I think the original suggestion to use recur may have been thinking of a more manual accumulation of the result. In case you haven't seen it, reduce is just a very specific usage of loop/recur:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn my-reduce
[step-fn init-val data-vec]
(loop [accum init-val
data data-vec]
(if (empty? data)
accum
(let [accum-next (step-fn accum (first data))
data-next (rest data)]
(recur accum-next data-next)))))
(dotest
(is= 10 (my-reduce + 0 (range 5))) ; 0..4
(is= 120 (my-reduce * 1 (range 1 6))) ; 1..5 )
In general, there can be any number of loop variables (not just 2 like for reduce). Using loop/recur gives you a more "functional" way of looping with accumulated state instead of using and atom and a doseq or something. As the name suggests, from the outside the effect is quite similar to a normal recursion w/o any stack size limits (i.e. tail-call optimization).
P.S. As this example shows, I like to use a let form to very explicitly name the values being generated for the next iteration.
P.P.S. While the compiler will allow you to type the following w/o confusion:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn my-reduce
[step-fn accum data]
(loop [accum accum
data data]
...))
it can be a bit confusing and/or sloppy to re-use variable names (esp. for people new to Clojure or your particular program).
Also
I would be remiss if I didn't point out that the function definition itself can be a recur target (i.e. you don't need to use loop). Consider this version of the factorial:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn fact-impl
[cum x]
(if (= x 1)
cum
(let [cum-next (* cum x)
x-next (dec x)]
(recur cum-next x-next))))
(defn fact [x] (fact-impl 1 x))
(dotest
(is= 6 (fact 3))
(is= 120 (fact 5)))
I am trying to learn Common Lisp with the book Common Lisp: A gentle introduction to Symbolic Computation. In addition, I am using SBCL, Emacs and Slime.
In chapter 7, the author suggests there are three styles of programming the book will cover: recursion, iteration and applicative programming.
I am interested on the last one. This style is famous for the applicative operator funcall which is the primitive responsible for other applicative operators such as mapcar.
Thus, with an educational purpose, I decided to implement my own version of mapcar using funcall:
(defun my-mapcar (fn xs)
(if (null xs)
nil
(cons (funcall fn (car xs))
(my-mapcar fn (cdr xs)))))
As you might see, I used recursion as a programming style to build an iconic applicative programming function.
It seems to work:
CL-USER> (my-mapcar (lambda (n) (+ n 1)) (list 1 2 3 4))
(2 3 4 5)
CL-USER> (my-mapcar (lambda (n) (+ n 1)) (list ))
NIL
;; comparing the results with the official one
CL-USER> (mapcar (lambda (n) (+ n 1)) (list ))
NIL
CL-USER> (mapcar (lambda (n) (+ n 1)) (list 1 2 3 4))
(2 3 4 5)
Is there a way to implement mapcar without using recursion or iteration? Using only applicative programming as a style?
Thanks.
Obs.: I tried to see how it was implemented. But it was not possible
CL-USER> (function-lambda-expression #'mapcar)
NIL
T
MAPCAR
I also used Emacs M-. to look for the documentation. However, the points below did not help me. I used this to find the files below:
/usr/share/sbcl-source/src/code/list.lisp
(DEFUN MAPCAR)
/usr/share/sbcl-source/src/compiler/seqtran.lisp
(:DEFINE-SOURCE-TRANSFORM MAPCAR)
/usr/share/sbcl-source/src/compiler/fndb.lisp
(DECLAIM MAPCAR SB-C:DEFKNOWN)
mapcar is by itself a primitive applicative operator (pag. 220 of Common Lisp: A gentle introduction to Symbolic Computation). So, if you want to rewrite it in an applicative way, you should use some other primitive applicative operator, for instance map or map-into. For instance, with map-into:
CL-USER> (defun my-mapcar (fn list &rest lists)
(apply #'map-into (make-list (length list)) fn list lists))
MY-MAPCAR
CL-USER> (my-mapcar #'1+ '(1 2 3))
(2 3 4)
CL-USER> (my-mapcar #'+ '(1 2 3) '(10 20 30) '(100 200 300))
(111 222 333)
Technically, recursion can be implemented as follows:
(defun fix (f)
(funcall (lambda (x) (funcall x x))
(lambda (x) (funcall f (lambda (&rest y) (apply (funcall x x) y))))))
Notice that fix does not use recursion in any way. In fact, we could have only used lambda in the definition of f as follows:
(defconstant fix-combinator
(lambda (g) (funcall
(lambda (x) (funcall x x))
(lambda (x) (funcall
g
(lambda (&rest y) (apply (funcall x x)
y)))))))
(defun fix-2 (f)
(funcall fix-combinator f))
The fix-combinator constant is more commonly known as the y combinator.
It turns out that fix has the following property:
Evaluating (apply (fix f) list) is equivalent to evaluating (apply (funcall f (fix f)) list). Informally, we have (fix f) = (funcall f (fix f)).
Thus, we can define map-car (I'm using a different name to avoid package lock) by
(defun map-car (func lst)
(funcall (fix (lambda (map-func) (lambda (lst) ; We want mapfunc to be (lambda (lst) (mapcar func lst))
(if (endp lst)
nil
(cons (funcall func (car lst))
(funcall map-func (cdr lst)))))))
lst))
Note the lack of recursion or iteration.
That being said, generally mapcar is just taken as a primitive notion when using the "applicative" style of programming.
Another way you can implement mapcar is by using the more general reduce function (a.k.a. fold). Let's name the user-provided function f and define my-mapcar.
The reduce function carries an accumulator value that builds up the resulting list, here it is going take a value v, a sublist rest, and call cons with (funcall f v) and rest, so as to build a list.
More precisely, here reduce is going to implement a right-fold, since cons is right-associative (e.g. the recursive list is the "right" hand side, ie. the second argument of cons, e.g. (cons a (cons b (cons nil)))).
In order to define a right-fold with reduce, you pass :from-end t, which indicates that it builds-up a value from the last element and the initial accumulator to obtain a new accumulator value, then the second to last element with that new accumulator to build a new accumulator, etc. This is how you ensure that the resulting elements are in the same order as the input list.
In that case, the reducing function takes its the current element as its first argument, and the accumulator as a second argument.
Since the type of the elements and the type of the accumulator are different, you need to pass an :initial-value for the accumulator (the default behavior where the initial-value is taken from the list is for functions like + or *, where the accumulator is in the same domain as the list elements).
With that in mind, you can write it as follows:
(defun my-map (f list)
(reduce (lambda (v rest) (cons (funcall f v) rest))
list
:from-end t
:initial-value nil))
For example:
(my-map #'prin1-to-string '(0 1 2 3))
; => ("0" "1" "2" "3")
I'm trying to solve this problem in a pure-functional way, without using set!.
I've written a function that calls a given lambda for each number in the fibonacci series, forever.
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(next b (+ a b)))))
(next 0 1)))
I think this is as succinct as it can be, but if I can shorten this, please enlighten me :)
With a definition like the above, is it possible to write another function that takes the first n numbers from the fibonacci series and gives me a list back, but without using variable mutation to track the state (which I understand is not really functional).
The function signature doesn't need to be the same as the following... any approach that will utilize each-fib without using set! is fine.
(take-n-fibs 7) ; (0 1 1 2 3 5 8)
I'm guessing there's some sort of continuations + currying trick I can use, but I keep coming back to wanting to use set!, which is what I'm trying to avoid (purely for learning purposes/shifting my thinking to purely functional).
Try this, implemented using lazy code by means of delayed evaluation:
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(delay (next b (+ a b))))))
(next 0 1)))
(define (take-n-fibs n fn)
(let loop ((i n)
(promise (each-fib fn)))
(when (positive? i)
(loop (sub1 i) (force promise)))))
As has been mentioned, each-fib can be further simplified by using a named let:
(define (each-fib fn)
(let next ((a 0) (b 1))
(fn a)
(delay (next b (+ a b)))))
Either way, it was necessary to modify each-fib a little for using the delay primitive, which creates a promise:
A promise encapsulates an expression to be evaluated on demand via force. After a promise has been forced, every later force of the promise produces the same result.
I can't think of a way to stop the original (unmodified) procedure from iterating indefinitely. But with the above change in place, take-n-fibs can keep forcing the lazy evaluation of as many values as needed, and no more.
Also, take-n-fibs now receives a function for printing or processing each value in turn, use it like this:
(take-n-fibs 10 (lambda (n) (printf "~a " n)))
> 0 1 1 2 3 5 8 13 21 34 55
You provide an iteration function over fibonacci elements. If you want, instead of iterating over each element, to accumulate a result, you should use a different primitive that would be a fold (or reduce) rather than an iter.
(It might be possible to use continuations to turn an iter into a fold, but that will probably be less readable and less efficient that a direct solution using either a fold or mutation.)
Note however that using an accumulator updated by mutation is also fine, as long as you understand what you are doing: you are using mutable state locally for convenience, but the function take-n-fibs is, seen from the outside, observationally pure, so you do not "contaminate" your program as a whole with side effects.
A quick prototype for fold-fib, adapted from your own code. I made an arbitrary choice as to "when stop folding": if the function returns null, we return the current accumulator instead of continuing folding.
(define (fold-fib init fn) (letrec ([next (lambda (acc a b)
(let ([acc2 (fn acc a)])
(if (null? acc2) acc
(next acc2 b (+ a b)))))])
(next init 0 1)))
(reverse (fold-fib '() (lambda (acc n) (if (> n 10) null (cons n acc)))))
It would be better to have a more robust convention to end folding.
I have written few variants. First you ask if
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(next b (+ a b)))))
(next 0 1)))
can be written any shorter. The pattern is used so often that special syntax called named let has been introduced. Your function looks like this using a named let:
(define (each-fib fn)
(let next ([a 0] [b 1])
(fn a)
(next b (+ a b))))
In order to get the control flowing from one function to another, one can in languages with supports TCO use continuation passing style. Each function gets an extra argument often called k (for continuation). The function k represents what-to-do-next.
Using this style, one can write your program as follows:
(define (generate-fibs k)
(let next ([a 0] [b 1] [k k])
(k a (lambda (k1)
(next b (+ a b) k1)))))
(define (count-down n k)
(let loop ([n n] [fibs '()] [next generate-fibs])
(if (zero? n)
(k fibs)
(next (λ (a next)
(loop (- n 1) (cons a fibs) next))))))
(count-down 5 values)
Now it is a bit annoying to write in style manually, so it could
be convenient to introduce the co-routines. Breaking your rule of not using set! I have chosen to use a shared variable fibs in which generate-fibs repeatedly conses new fibonacci numbers onto. The count-down routine merely read the values, when the count down is over.
(define (make-coroutine co-body)
(letrec ([state (lambda () (co-body resume))]
[resume (lambda (other)
(call/cc (lambda (here)
(set! state here)
(other))))])
(lambda ()
(state))))
(define fibs '())
(define generate-fib
(make-coroutine
(lambda (resume)
(let next ([a 0] [b 1])
(set! fibs (cons a fibs))
(resume count-down)
(next b (+ a b))))))
(define count-down
(make-coroutine
(lambda (resume)
(let loop ([n 10])
(if (zero? n)
fibs
(begin
(resume generate-fib)
(loop (- n 1))))))))
(count-down)
And a bonus you get a version with communicating threads:
#lang racket
(letrec ([result #f]
[count-down
(thread
(λ ()
(let loop ([n 10] [fibs '()])
(if (zero? n)
(set! result fibs)
(loop (- n 1) (cons (thread-receive) fibs))))))]
[produce-fibs
(thread
(λ ()
(let next ([a 0] [b 1])
(when (thread-running? count-down)
(thread-send count-down a)
(next b (+ a b))))))])
(thread-wait count-down)
result)
The thread version is Racket specific, the others ought to run anywhere.
Building a list would be hard. But displaying the results can still be done (in a very bad fashion)
#lang racket
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(next b (+ a b)))))
(next 0 1)))
(define (take-n-fibs n fn)
(let/cc k
(begin
(each-fib (lambda (x)
(if (= x (fib (+ n 1)))
(k (void))
(begin
(display (fn x))
(newline))))))))
(define fib
(lambda (n)
(letrec ((f
(lambda (i a b)
(if (<= n i)
a
(f (+ i 1) b (+ a b))))))
(f 1 0 1))))
Notice that i am using the regular fibonacci function as an escape (like i said, in a very bad fashion). I guess nobody will recommend programming like this.
Anyway
(take-n-fibs 7 (lambda (x) (* x x)))
0
1
1
4
9
25
64
i'm working through SICP - one exercise is to implement foreach (doseq). This is an academic exercise. In clojure, this is what I came up with:
(defn for-each [proc, items]
(if (empty? items) nil
(do
(proc (first items))
(recur proc (rest items)))))
but, i'm a little murky about if do is cheating, because do is a special form in clojure and i don't think anything like that has been introduced yet in SICP. is there a more minimalist answer?
Here's another attempt which only executes proc on the last element:
(defn for-each-2 [proc, items]
(let [f (first items)
r (rest items)]
(if (empty? r)
(proc f)
(recur proc r))))
Use doseq and you're all set. For example:
(doseq [e '(1 2 3)]
(prn e))
Will print:
1
2
3
nil
EDIT :
If you want to implement for-each by hand and using as few special forms as possible, here's another alternative, although it ends up being almost as short as yours:
(defn for-each [f l]
(cond (empty? l) nil
:else (do (f (first l))
(recur f (rest l)))))
Interestingly, the same procedure could have been written more succinctly in Scheme, the Lisp dialect used in SICP:
(define (for-each f l)
(cond ((null? l) null)
(else (f (first l))
(for-each f (rest l)))))
Here is my attempt. It just carries function execution in an inner loop.
(defn for-each [fun, xs]
(loop [fun fun
xs xs
action nil]
(if (first xs)
(recur fun (rest xs) (fun (first xs)))
xs)))
To set some context, I'm in the process of learning Clojure, and Lisp development more generally. On my path to Lisp, I'm currently working through the "Little" series in an effort to solidify a foundation in functional programming and recursive-based solution solving. In "The Little Schemer," I've worked through many of the exercises, however, I'm struggling a bit to convert some of them to Clojure. More specifically, I'm struggling to convert them to use "recur" so as to enable TCO. For example, here is a Clojure-based implementation to the "occurs*" function (from Little Schemer) which counts the number of occurrences of an atom appearing within a list of S-expressions:
(defn atom? [l]
(not (list? l)))
(defn occurs [a lst]
(cond
(empty? lst) 0
(atom? (first lst))
(cond
(= a (first lst)) (inc (occurs a (rest lst)))
true (occurs a (rest lst)))
true (+ (occurs a (first lst))
(occurs a (rest lst)))))
Basically, (occurs 'abc '(abc (def abc) (abc (abc def) (def (((((abc))))))))) will evaluate to 5. The obvious problem is that this definition consumes stack frames and will blow the stack if given a list of S-expressions too deep.
Now, I understand the option of refactoring recursive functions to use an accumulator parameter to enable putting the recursive call into the tail position (to allow for TCO), but I'm struggling if this option is even applicable to situations such as this one.
Here's how far I get if I try to refactor this using "recur" along with using an accumulator parameter:
(defn recur-occurs [a lst]
(letfn [(myoccurs [a lst count]
(cond
(empty? lst) 0
(atom? (first lst))
(cond
(= a (first lst)) (recur a (rest lst) (inc count))
true (recur a (rest lst) count))
true (+ (recur a (first lst) count)
(recur a (rest lst) count))))]
(myoccurs a lst 0)))
So, I feel like I'm almost there, but not quite. The obvious problem is my "else" clause in which the head of the list is not an atom. Conceptually, I want to sum the result of recurring over the first element in the list with the result of recurring over the rest of the list. I'm struggling in my head on how to refactor this such that the recurs can be moved to the tail position.
Are there additional techniques to the "accumulator" pattern to achieving getting your recursive calls put into the tail position that I should be applying here, or, is the issue simply more "fundamental" and that there isn't a clean Clojure-based solution due to the JVM's lack of TCO? If the latter, generally speaking, what should be the general pattern for Clojure programs to use that need to recur over a list of S-expressions? For what it's worth, I've seen the multi method w/lazy-seq technique used (page 151 of Halloway's "Programming Clojure" for reference) to "Replace Recursion with Laziness" - but I'm not sure how to apply that pattern to this example in which I'm not attempting to build a list, but to compute a single integer value.
Thank you in advance for any guidance on this.
Firstly, I must advise you to not worry much about implementation snags like stack overflows as you make your way through The Little Schemer. It is good to be conscientious of issues like the lack of tail call optimization when you're programming in anger, but the main point of the book is to teach you to think recursively. Converting the examples accumulator-passing style is certainly good practice, but it's essentially ditching recursion in favor of iteration.
However, and I must preface this with a spoiler warning, there is a way to keep the same recursive algorithm without being subject to the whims of the JVM stack. We can use continuation-passing style to make our own stack in the form of an extra anonymous function argument k:
(defn occurs-cps [a lst k]
(cond
(empty? lst) (k 0)
(atom? (first lst))
(cond
(= a (first lst)) (occurs-cps a (rest lst)
(fn [v] (k (inc v))))
:else (occurs-cps a (rest lst) k))
:else (occurs-cps a (first lst)
(fn [fst]
(occurs-cps a (rest lst)
(fn [rst] (k (+ fst rst))))))))
Instead of the stack being created implicitly by our non-tail function calls, we bundle up "what's left to do" after each call to occurs, and pass it along as the next continuation k. When we invoke it, we start off with a k that represents nothing left to do, the identity function:
scratch.core=> (occurs-cps 'abc
'(abc (def abc) (abc (abc def) (def (((((abc))))))))
(fn [v] v))
5
I won't go further into the details of how to do CPS, as that's for a later chapter of TLS. However, I will note that this of course doesn't yet work completely:
scratch.core=> (def ls (repeat 20000 'foo))
#'scratch.core/ls
scratch.core=> (occurs-cps 'foo ls (fn [v] v))
java.lang.StackOverflowError (NO_SOURCE_FILE:0)
CPS lets us move all of our non-trivial, stack-building calls to tail position, but in Clojure we need to take the extra step of replacing them with recur:
(defn occurs-cps-recur [a lst k]
(cond
(empty? lst) (k 0)
(atom? (first lst))
(cond
(= a (first lst)) (recur a (rest lst)
(fn [v] (k (inc v))))
:else (recur a (rest lst) k))
:else (recur a (first lst)
(fn [fst]
(recur a (rest lst) ;; Problem
(fn [rst] (k (+ fst rst))))))))
Alas, this goes wrong: java.lang.IllegalArgumentException: Mismatched argument count to recur, expected: 1 args, got: 3 (core.clj:39). The very last recur actually refers to the fn right above it, the one we're using to represent our continuations! We can get good behavior most of the time by changing just that recur to a call to occurs-cps-recur, but pathologically-nested input will still overflow the stack:
scratch.core=> (occurs-cps-recur 'foo ls (fn [v] v))
20000
scratch.core=> (def nested (reduce (fn [onion _] (list onion))
'foo (range 20000)))
#'scratch.core/nested
scratch.core=> (occurs-cps-recur 'foo nested (fn [v] v))
Java.lang.StackOverflowError (NO_SOURCE_FILE:0)
Instead of making the call to occurs-* and expecting it to give back an answer, we can have it return a thunk immediately. When we invoke that thunk, it'll go off and do some work right up until it does a recursive call, which in turn will return another thunk. This is trampolined style, and the function that "bounces" our thunks is trampoline. Returning a thunk each time we make a recursive call bounds our stack size to one call at a time, so our only limit is the heap:
(defn occurs-cps-tramp [a lst k]
(fn []
(cond
(empty? lst) (k 0)
(atom? (first lst))
(cond
(= a (first lst)) (occurs-cps-tramp a (rest lst)
(fn [v] (k (inc v))))
:else (occurs-cps-tramp a (rest lst) k))
:else (occurs-cps-tramp a (first lst)
(fn [fst]
(occurs-cps-tramp a (rest lst)
(fn [rst] (k (+ fst rst)))))))))
(declare done answer)
(defn my-trampoline [th]
(if done
answer
(recur (th))))
(defn empty-k [v]
(set! answer v)
(set! done true))
(defn run []
(binding [done false answer 'whocares]
(my-trampoline (occurs-cps-tramp 'foo nested empty-k))))
;; scratch.core=> (run)
;; 1
Note that Clojure has a built-in trampoline (with some limitations on the return type). Using that instead, we don't need a specialized empty-k:
scratch.core=> (trampoline (occurs-cps-tramp 'foo nested (fn [v] v)))
1
Trampolining is certainly a cool technique, but the prerequisite to trampoline a program is that it must contain only tail calls; CPS is the real star here. It lets you define your algorithm with the clarity of natural recursion, and through correctness-preserving transformations, express it efficiently on any host that has a single loop and a heap.
You can't do this with a fixed amount of memory. You can consume stack, or heap; that's the decision you get to make. If I were writing this in Clojure I would do it with map and reduce rather than with manual recursion:
(defn occurs [x coll]
(if (coll? coll)
(reduce + (map #(occurs x %) coll))
(if (= x coll)
1, 0)))
Note that shorter solutions exist if you use tree-seq or flatten, but at that point most of the problem is gone so there's not much to learn.
Edit
Here's a version that doesn't use any stack, instead letting its queue get larger and larger (using up heap).
(defn heap-occurs [item coll]
(loop [count 0, queue coll]
(if-let [[x & xs] (seq queue)]
(if (coll? x)
(recur count (concat x xs))
(recur (+ (if (= item x) 1, 0)
count)
xs))
count)))