I'm trying some lazy streams in Clojure.
If I do:
(defn ints-from [n]
(cons n (lazy-seq (ints-from (inc n)))))
and
(def nats (ints-from 0))
it's fine, I can do something like:
(take 5 nats)
Now I'm trying to encapsulate the 2 functions in 1:
(defn natz[]
( letfn [(aux [n]((cons n (lazy-seq (aux (inc n)))))) ] (aux 0) ))
This seems to compile, but does not do what I expect.
(take 4 natz)
gives:
(user=> IllegalArgumentException Don't know how to create ISeq from: user$natz
clojure.lang.RT.seqFrom (RT.java:494)
What am I missing?
One parenthesis less inside letfn definition and one parenthesis more to invoke natz function
(defn natz[]
(letfn [(aux [n] (cons n (lazy-seq (aux (inc n)))))]
(aux 0)))
Example usage:
(take 4 (natz))
=> (0 1 2 3)
Related
Consider this simple-minded recursive implementation of comp in Clojure:
(defn my-comp
([f]
(fn [& args]
(apply f args)))
([f & funcs]
(fn [& args]
(f (apply (apply my-comp funcs) args)))))
The right way to do this, I am told, is using recur, but I am unsure how recur works. In particular: is there a way to coax the code above into being recurable?
evaluation 1
First let's visualize the problem. my-comp as it is written in the question will create a deep stack of function calls, each waiting on the stack to resolve, blocked until the the deepest call returns -
((my-comp inc inc inc) 1)
((fn [& args]
(inc (apply (apply my-comp '(inc inc)) args))) 1)
(inc (apply (fn [& args]
(inc (apply (apply my-comp '(inc)) args))) '(1)))
(inc (inc (apply (apply my-comp '(inc)) '(1))))
(inc (inc (apply (fn [& args]
(apply inc args)) '(1))))
(inc (inc (apply inc '(1)))) ; ⚠️ deep in the hole we go...
(inc (inc 2))
(inc 3)
4
tail-recursive my-comp
Rather than creating a long sequence of functions, this my-comp is refactored to return a single function, which when called, runs a loop over the supplied input functions -
(defn my-comp [& fs]
(fn [init]
(loop [acc init [f & more] fs]
(if (nil? f)
acc
(recur (f acc) more))))) ; 🐍 tail recursion
((my-comp inc inc inc) 1)
;; 4
((apply my-comp (repeat 1000000 inc)) 1)
;; 1000001
evaluation 2
With my-comp rewritten to use loop and recur, we can see linear iterative evaluation of the composition -
((my-comp inc inc inc) 1)
(loop 1 (list inc inc inc))
(loop 2 (list inc inc))
(loop 3 (list inc))
(loop 4 nil)
4
multiple input args
Did you notice ten (10) apply calls at the beginning of this post? This is all in service to support multiple arguments for the first function in the my-comp sequence. It is a mistake to tangle this complexity with my-comp itself. The caller has control to do this if it is the desired behavior.
Without any additional changes to the refactored my-comp -
((my-comp #(apply * %) inc inc inc) '(3 4)) ; ✅ multiple input args
Which evaluates as -
(loop '(3 4) (list #(apply * %) inc inc inc))
(loop 12 (list inc inc inc))
(loop 13 (list inc inc))
(loop 14 (list inc))
(loop 15 nil)
15
right-to-left order
Above (my-comp a b c) will apply a first, then b, and finally c. If you want to reverse that order, a naive solution would be to call reverse at the loop call site -
(defn my-comp [& fs]
(fn [init]
(loop [acc init [f & more] (reverse fs)] ; ⚠️ naive
(if (nil? f)
acc
(recur (f acc) more)))))
Each time the returned function is called, (reverse fs) will be recomputed. To avoid this, use a let binding to compute the reversal just once -
(defn my-comp [& fs]
(let [fs (reverse fs)] ; ✅ reverse once
(fn [init]
(loop [acc init [f & more] fs]
(if (nil? f)
acc
(recur (f acc) more))))))
a way to do this, is to rearrange this code to pass some intermediate function back up to the definition with recur.
the model would be something like this:
(my-comp #(* 10 %) - +)
(my-comp (fn [& args] (#(* 10 %) (apply - args)))
+)
(my-comp (fn [& args]
((fn [& args] (#(* 10 %) (apply - args)))
(apply + args))))
the last my-comp would use the first my-comp overload (which is (my-comp [f])
here's how it could look like:
(defn my-comp
([f] f)
([f & funcs]
(if (seq funcs)
(recur (fn [& args]
(f (apply (first funcs) args)))
(rest funcs))
(my-comp f))))
notice that despite of not being the possible apply target, the recur form can still accept variadic params being passed as a sequence.
user> ((my-comp (partial repeat 3) #(* 10 %) - +) 1 2 3)
;;=> (-60 -60 -60)
notice, though, that in practice this implementation isn't really better than yours: while recur saves you from stack overflow on function creation, it would still overflow on application (somebody, correct me if i'm wrong):
(apply my-comp (repeat 1000000 inc)) ;; ok
((apply my-comp (repeat 1000000 inc)) 1) ;; stack overflow
so it would probably be better to use reduce or something else:
(defn my-comp-reduce [f & fs]
(let [[f & fs] (reverse (cons f fs))]
(fn [& args]
(reduce (fn [acc curr-f] (curr-f acc))
(apply f args)
fs))))
user> ((my-comp-reduce (partial repeat 3) #(* 10 %) - +) 1 2 3)
;;=> (-60 -60 -60)
user> ((apply my-comp-reduce (repeat 1000000 inc)) 1)
;;=> 1000001
There is already a good answer above, but I think the original suggestion to use recur may have been thinking of a more manual accumulation of the result. In case you haven't seen it, reduce is just a very specific usage of loop/recur:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn my-reduce
[step-fn init-val data-vec]
(loop [accum init-val
data data-vec]
(if (empty? data)
accum
(let [accum-next (step-fn accum (first data))
data-next (rest data)]
(recur accum-next data-next)))))
(dotest
(is= 10 (my-reduce + 0 (range 5))) ; 0..4
(is= 120 (my-reduce * 1 (range 1 6))) ; 1..5 )
In general, there can be any number of loop variables (not just 2 like for reduce). Using loop/recur gives you a more "functional" way of looping with accumulated state instead of using and atom and a doseq or something. As the name suggests, from the outside the effect is quite similar to a normal recursion w/o any stack size limits (i.e. tail-call optimization).
P.S. As this example shows, I like to use a let form to very explicitly name the values being generated for the next iteration.
P.P.S. While the compiler will allow you to type the following w/o confusion:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn my-reduce
[step-fn accum data]
(loop [accum accum
data data]
...))
it can be a bit confusing and/or sloppy to re-use variable names (esp. for people new to Clojure or your particular program).
Also
I would be remiss if I didn't point out that the function definition itself can be a recur target (i.e. you don't need to use loop). Consider this version of the factorial:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn fact-impl
[cum x]
(if (= x 1)
cum
(let [cum-next (* cum x)
x-next (dec x)]
(recur cum-next x-next))))
(defn fact [x] (fact-impl 1 x))
(dotest
(is= 6 (fact 3))
(is= 120 (fact 5)))
I am trying to rewrite this piece of code from https://github.com/lspector/gp/blob/master/src/gp/evolvefn_zip.clj
to use recur:
(defn random-code [depth]
(if (or (zero? depth)
(zero? (rand-int 2)))
(random-terminal)
(let [f (random-function)]
(cons f (repeatedly (get function-table f)
#(random-code (dec depth)))))))
The problem is, I have absolutely no idea how to do that.
The only thing I can think of is something like this:
(defn random-code [depth]
(loop [d depth t 0 c []]
(if (or (zero? depth)
(zero? (rand-int 2)))
(if (= t 0)
(conj c random-terminal)
(recur depth (dec t) (conj c (random-terminal))))
(let [f (random-function)]
(if (= t 0)
(recur (dec depth) (function-table f) (conj c f))
(recur depth (dec t) (conj c f)))))))
It's not a working piece of code, it's just to show the way I would try to solve it, it would only get more and more convoluted.
Is there a better way to convert normal recursion to tail recursion in clojure?
Here are 2 examples comparing a recursive algorithm and loop-recur:
(defn fact-recursion [n]
(if (zero? n)
1
(* n (fact-recursion (dec n)))))
(defn fact-recur [n]
(loop [count n
result 1]
(if (pos? count)
(recur (dec count) (* result count))
result )))
(fact-recursion 5) => 120
(fact-recur 5) => 120
(defn rangy-recursive [n]
(if (pos? n)
(cons n (rangy-recursive (dec n)))
[n]))
(defn rangy-recur [n]
(loop [result []
count n]
(if (pos? count)
(recur (conj result count) (dec count))
result)))
(rangy-recursive 5) => (5 4 3 2 1 0)
(rangy-recur 5) => [5 4 3 2 1]
The basic difference is that for loop-recur you need a 2nd loop "variable" (here named result) to accumulate the output of the algorithm. For plain recursion, the call stack accumulates the intermediate result.
So I am trying to re-implement the reduce method, so it can add a couple of numbers that normally can be done using reduce, like:
(reduce + [1 2 3]) ;; 6
(newRd + [1 2 3]) ;; 6
So I thought maybe it can be done using a recursive function that adds the first element of the vector every time it is called and do it again for the rest of the vector. Something like this:
(defn newRd [list]
(let [sum 0]
(if (not= 0 (count list))
(+ sum (first list))
(newRd (rest list))
)
)
)
I think I am not doing local storing correctly. Any suggestion or maybe a better approach?
there are two mistakes here in your code:
1) you don't add your current sum to the recursive call result
2) you should return zero when the list is empty
corrected variant:
(defn newRd [list]
(let [sum 0]
(if (not= 0 (count list))
(+ sum (first list)
(newRd (rest list)))
sum)))
in repl:
user> (newRd [1 2 3 4])
10
next, you can update it a bit:
first you don't really need the sum in let statement, since the sum always = 0
second, there is a lib function empty? to check if list is empty.
(defn newRd [list]
(if-not (empty? list)
(+ (first list)
(newRd (rest list)))
0))
but remember: clojure doesnt'do tail call optimization, so it is easy to cause stack owerflow with a long list:
user> (newRd (repeat 1000000 1))
StackOverflowError user/newRd (form-init289434844644272272.clj:73)
so it's preferable to use loop/recur
(defn sum-list [list]
(loop [list list sum 0]
(if (empty? list)
sum
(recur (rest list) (+ sum (first list))))))
in repl:
user> (sum-list (repeat 1000000 1))
1000000
the other option is to make the function itself tail recursive:
(defn newRd [list sum]
(if-not (empty? list)
(recur (rest list) (+ sum (first list)))
sum))
user> (newRd (repeat 1000000 1) 0)
1000000
then you can add the additoinal arity, for not to pass the second parameter in every call:
(defn newRd
([list] (newRd list 0))
([list sum]
(if-not (empty? list)
(recur (rest list) (+ sum (first list)))
sum)))
Further to leetwinksi's answer ...
You might as well implement new-reduce (camel case is not idiomatic) in general:
(defn new-reduce
([f init coll]
(if (seq coll)
(recur f (f init (f init (first coll))) (rest coll))
init))
([f coll]
(if (seq coll)
(reduce f (first coll) (rest coll))
(f))))
Then
(new-reduce + [1 2 3]) ;; 6
This is more or less what the source code for reduce looked like until recently, if you stripped chunking out.
The two-argument version that you use leans on the three-argument version, which you can recur on directly, without an explicit loop. This entails passing f each time, but that's what it used to do. Presumably it's faster to carry an extra argument than to work in a local scope.
I want to reverse a sequence in Clojure without using the reverse function, and do so recursively.
Here is what I came up with:
(defn reverse-recursively [coll]
(loop [r (rest coll)
acc (conj () (first coll))]
(if (= (count r) 0)
acc
(recur (rest r) (conj acc (first r))))))
Sample output:
user> (reverse-recursively '(1 2 3 4 5 6))
(6 5 4 3 2 1)
user> (reverse-recursively [1 2 3 4 5 6])
(6 5 4 3 2 1)
user> (reverse-recursively {:a 1 :b 2 :c 3})
([:c 3] [:b 2] [:a 1])
Questions:
Is there a more concise way of doing this, i.e. without loop/recur?
Is there a way to do this without using an "accumulator" parameter in the loop?
References:
Whats the best way to recursively reverse a string in Java?
http://groups.google.com/group/clojure/browse_thread/thread/4e7a4bfb0d71a508?pli=1
You don't need to count. Just stop when the remaining sequence is empty.
You shouldn't pre-populate the acc, since the original input may be empty (and it's more code).
Destructuring is cool.
(defn reverse-recursively [coll]
(loop [[r & more :as all] (seq coll)
acc '()]
(if all
(recur more (cons r acc))
acc)))
As for loop/recur and the acc, you need some way of passing around the working reversed list. It's either loop, or add another param to the function (which is really what loop is doing anyway).
Or use a higher-order function:
user=> (reduce conj '() [1 2 3 4])
(4 3 2 1)
For the sake of exhaustivenes, there is one more method using into. Since into internally uses conj it can be used as follows :
(defn reverse-list
"Reverse the element of alist."
[lst]
(into '() lst))
Yes to question 1, this is what I came up with for my answer to the recursion koan (I couldn't tell you whether it was good clojure practice or not).
(defn recursive-reverse [coll]
(if (empty? coll)
[]
(conj (recursive-reverse (rest coll)) (first coll) )))
In current version of Clojure there's a built-in function called rseq. For anyone who passes by.
(defn my-rev [col]
(loop [ col col
result []]
(if (empty? col)
result
(recur (rest col) (cons (first col) result)))))
Q1.
The JVM can not optimize the recursion, a recursive function that would directly and stack overflow. Therefore, in Clojure, which uses the loop/recur. So, without using a function that recur deep recursion can not be defined. (which is also used internally to recur as a function trampoline.)
Q2.
a recursive function by recur, must be tail-recursive. If the normal recursive function change to tail-recursive function, so there is a need to carry about the value of a variable is required as the accumulator.
(defn reverse-seq [sss]
(if (not (empty? sss))
(conj (reverse-seq (rest sss)) (first sss))
)
)
(defn recursive-reverse [coll]
(if (empty? coll)
()
(concat (vector (peek coll)) (recursive-reverse (pop coll )))
)
)
and test:
user=> (recursive-reverse [1])
(1)
user=> (recursive-reverse [1 2 3 4 5])
(5 4 3 2 1)
I'm trying to write a function that returns a memoized recursive function in Clojure, but I'm having trouble making the recursive function see its own memoized bindings. Is this because there is no var created? Also, why can't I use memoize on the local binding created with let?
This slightly unusual Fibonacci sequence maker that starts at a particular number is an example of what I wish I could do:
(defn make-fibo [y]
(memoize (fn fib [x] (if (< x 2)
y
(+ (fib (- x 1))
(fib (- x 2)))))))
(let [f (make-fibo 1)]
(f 35)) ;; SLOW, not actually memoized
Using with-local-vars seems like the right approach, but it doesn't work for me either. I guess I can't close over vars?
(defn make-fibo [y]
(with-local-vars [fib (fn [x] (if (< x 2)
y
(+ (#fib (- x 1))
(#fib (- x 2)))))]
(memoize fib)))
(let [f (make-fibo 1)]
(f 35)) ;; Var null/null is unbound!?!
I could of course manually write a macro that creates a closed-over atom and manage the memoization myself, but I was hoping to do this without such hackery.
There is an interesting way to do it that does rely neither on rebinding nor the behavior of def. The main trick is to go around the limitations of recursion by passing a function as an argument to itself:
(defn make-fibo [y]
(let
[fib
(fn [mem-fib x]
(let [fib (fn [a] (mem-fib mem-fib a))]
(if (<= x 2)
y
(+ (fib (- x 1)) (fib (- x 2))))))
mem-fib (memoize fib)]
(partial mem-fib mem-fib)))
Then:
> ((make-fibo 1) 50)
12586269025
What happens here:
The fib recursive function got a new argument mem-fib. This will be the memoized version of fib itself, once it gets defined.
The fib body is wrapped in a let form that redefines calls to fib so that they pass the mem-fib down to next levels of recursion.
mem-fib is defined as memoized fib
... and will be passed by partial as the first argument to itself to start the above mechanism.
This trick is similar to the one used by the Y combinator to calculate function's fix point in absence of a built-in recursion mechanism.
Given that def "sees" the symbol being defined, there is little practical reason to go this way, except maybe for creating anonymous in-place recursive memoized functions.
This seems to work:
(defn make-fibo [y]
(with-local-vars
[fib (memoize
(fn [x]
(if (< x 2)
y
(+ (fib (- x 2)) (fib (dec x))))))]
(.bindRoot fib #fib)
#fib))
with-local-vars only provides thread-local bindings for the newly created Vars, which are popped once execution leaves the with-local-vars form; hence the need for .bindRoot.
(def fib (memoize (fn [x] (if (< x 2)
x
(+ (fib (- x 1))
(fib (- x 2)))))))
(time (fib 35))
Here is the simplest solution:
(def fibo
(memoize (fn [n]
(if (< n 2)
n
(+ (fibo (dec n))
(fibo (dec (dec n))))))))
You can encapsulate the recursive memoized function pattern in a macro if you plan to use it several times.
(defmacro defmemo
[name & fdecl]
`(def ~name
(memoize (fn ~fdecl))))
Here's a cross between the Y-combinator and Clojure's memoize:
(defn Y-mem [f]
(let [mem (atom {})]
(#(% %)
(fn [x]
(f #(if-let [e (find #mem %&)]
(val e)
(let [ret (apply (x x) %&)]
(swap! mem assoc %& ret)
ret))))))))
You can macrosugar this up:
(defmacro defrecfn [name args & body]
`(def ~name
(Y-mem (fn [foo#]
(fn ~args (let [~name foo#] ~#body))))))
Now for using it:
(defrecfn fib [n]
(if (<= n 1)
n
(+' (fib (- n 1))
(fib (- n 2)))))
user=> (time (fib 200))
"Elapsed time: 0.839868 msecs"
280571172992510140037611932413038677189525N
Or the Levenshtein distance:
(defrecfn edit-dist [s1 s2]
(cond (empty? s1) (count s2)
(empty? s2) (count s1)
:else (min (inc (edit-dist s1 (butlast s2)))
(inc (edit-dist (butlast s1) s2))
((if (= (last s1) (last s2)) identity inc)
(edit-dist (butlast s1) (butlast s2))))))
Your first version actually works, but you're not getting all the benefits of memoization because you're only running through the algorithm once.
Try this:
user> (time (let [f (make-fibo 1)]
(f 35)))
"Elapsed time: 1317.64842 msecs"
14930352
user> (time (let [f (make-fibo 1)]
[(f 35) (f 35)]))
"Elapsed time: 1345.585041 msecs"
[14930352 14930352]
You can generate memoized recursive functions in Clojure with a variant of the Y combinator. For instance, the code for factorial would be:
(def Ywrap
(fn [wrapper-func f]
((fn [x]
(x x))
(fn [x]
(f (wrapper-func (fn [y]
((x x) y))))))))
(defn memo-wrapper-generator []
(let [hist (atom {})]
(fn [f]
(fn [y]
(if (find #hist y)
(#hist y)
(let [res (f y)]
(swap! hist assoc y res)
res))))))
(def Ymemo
(fn [f]
(Ywrap (memo-wrapper-generator) f)))
(def factorial-gen
(fn [func]
(fn [n]
(println n)
(if (zero? n)
1
(* n (func (dec n)))))))
(def factorial-memo (Ymemo factorial-gen))
This is explained in details in this article about Y combinator real life application: recursive memoization in clojure.