I am trying to write up a simple Markovian state space models, that, as the name suggests iteratively looks back one step to predict the next state.
Here is what is supposed to be a MWE, though it is not because I cannot quite figure out how I am supposed to place (recur ... ) in the below code.
;; helper function
(defn dur-call
[S D]
(if (< 1 D)
(- D 1)
(rand-int S)))
;; helper function
(defn trans-call
[S D]
(if (< 1 D)
S
(rand-int 3)))
;; state space model
(defn test-func
[t]
(loop
[S (rand-int 3)]
(if (<= t 0)
[S (rand-int (+ S 1))]
(let [pastS (first (test-func (- t 1)))
pastD (second (test-func (- t 1)))
S (trans-call pastS pastD)]
(recur ...?)
[S (dur-call S pastD)]))))
My target is to calculate some a state at say time t=5 say, in which case the model needs to look back and calculate states t=[0 1 2 3 4] as well. This should, in my mind, be done well with loop/recur but could also be done with reduce perhaps (not sure how, still new to Clojure). My problem is really that it would seemt have to use recur inside let but that should not work given how loop/recur are designed.
your task is really to generate the next item based on the previous one, starting with some seed. In clojure it can be fulfilled by using iterate function:
user> (take 10 (iterate #(+ 2 %) 1))
(1 3 5 7 9 11 13 15 17 19)
you just have to define the function to produce the next value. It could look like this (not sure about the correctness of the computation algorithm, just based on what is in the question):
(defn next-item [[prev-s prev-d :as prev-item]]
(let [s (trans-call prev-s prev-d)]
[s (dur-call s prev-d)]))
and now let's iterate with it, starting from some value:
user> (take 5 (iterate next-item [3 4]))
([3 4] [3 3] [3 2] [3 1] [0 0])
now your test function could be implemented this way:
(defn test-fn [t]
(when (not (neg? t))
(nth (iterate next-item
(let [s (rand-int 3)]
[s (rand-int (inc s))]))
t)))
you can also do it with loop (but it is still less idiomatic):
(defn test-fn-2 [t]
(when (not (neg? t))
(let [s (rand-int 3)
d (rand-int (inc s))]
(loop [results [[s d]]]
(if (< t (count results))
(peek results)
(recur (conj results (next-item (peek results)))))))))
here we pass all the accumulated results to the next iteration of the loop.
also you can introduce the loop's iteration index and just pass around the last result together with it:
(defn test-fn-3 [t]
(when (not (neg? t))
(let [s (rand-int 3)
d (rand-int (inc s))]
(loop [result [s d] i 0]
(if (= i t)
result
(recur (next-item result) (inc i)))))))
and one more example with reduce:
(defn test-fn-4 [t]
(when (not (neg? t))
(reduce (fn [prev _] (next-item prev))
(let [s (rand-int 3)
d (rand-int (inc s))]
[s d])
(range t))))
Related
Consider this simple-minded recursive implementation of comp in Clojure:
(defn my-comp
([f]
(fn [& args]
(apply f args)))
([f & funcs]
(fn [& args]
(f (apply (apply my-comp funcs) args)))))
The right way to do this, I am told, is using recur, but I am unsure how recur works. In particular: is there a way to coax the code above into being recurable?
evaluation 1
First let's visualize the problem. my-comp as it is written in the question will create a deep stack of function calls, each waiting on the stack to resolve, blocked until the the deepest call returns -
((my-comp inc inc inc) 1)
((fn [& args]
(inc (apply (apply my-comp '(inc inc)) args))) 1)
(inc (apply (fn [& args]
(inc (apply (apply my-comp '(inc)) args))) '(1)))
(inc (inc (apply (apply my-comp '(inc)) '(1))))
(inc (inc (apply (fn [& args]
(apply inc args)) '(1))))
(inc (inc (apply inc '(1)))) ; ⚠️ deep in the hole we go...
(inc (inc 2))
(inc 3)
4
tail-recursive my-comp
Rather than creating a long sequence of functions, this my-comp is refactored to return a single function, which when called, runs a loop over the supplied input functions -
(defn my-comp [& fs]
(fn [init]
(loop [acc init [f & more] fs]
(if (nil? f)
acc
(recur (f acc) more))))) ; 🐍 tail recursion
((my-comp inc inc inc) 1)
;; 4
((apply my-comp (repeat 1000000 inc)) 1)
;; 1000001
evaluation 2
With my-comp rewritten to use loop and recur, we can see linear iterative evaluation of the composition -
((my-comp inc inc inc) 1)
(loop 1 (list inc inc inc))
(loop 2 (list inc inc))
(loop 3 (list inc))
(loop 4 nil)
4
multiple input args
Did you notice ten (10) apply calls at the beginning of this post? This is all in service to support multiple arguments for the first function in the my-comp sequence. It is a mistake to tangle this complexity with my-comp itself. The caller has control to do this if it is the desired behavior.
Without any additional changes to the refactored my-comp -
((my-comp #(apply * %) inc inc inc) '(3 4)) ; ✅ multiple input args
Which evaluates as -
(loop '(3 4) (list #(apply * %) inc inc inc))
(loop 12 (list inc inc inc))
(loop 13 (list inc inc))
(loop 14 (list inc))
(loop 15 nil)
15
right-to-left order
Above (my-comp a b c) will apply a first, then b, and finally c. If you want to reverse that order, a naive solution would be to call reverse at the loop call site -
(defn my-comp [& fs]
(fn [init]
(loop [acc init [f & more] (reverse fs)] ; ⚠️ naive
(if (nil? f)
acc
(recur (f acc) more)))))
Each time the returned function is called, (reverse fs) will be recomputed. To avoid this, use a let binding to compute the reversal just once -
(defn my-comp [& fs]
(let [fs (reverse fs)] ; ✅ reverse once
(fn [init]
(loop [acc init [f & more] fs]
(if (nil? f)
acc
(recur (f acc) more))))))
a way to do this, is to rearrange this code to pass some intermediate function back up to the definition with recur.
the model would be something like this:
(my-comp #(* 10 %) - +)
(my-comp (fn [& args] (#(* 10 %) (apply - args)))
+)
(my-comp (fn [& args]
((fn [& args] (#(* 10 %) (apply - args)))
(apply + args))))
the last my-comp would use the first my-comp overload (which is (my-comp [f])
here's how it could look like:
(defn my-comp
([f] f)
([f & funcs]
(if (seq funcs)
(recur (fn [& args]
(f (apply (first funcs) args)))
(rest funcs))
(my-comp f))))
notice that despite of not being the possible apply target, the recur form can still accept variadic params being passed as a sequence.
user> ((my-comp (partial repeat 3) #(* 10 %) - +) 1 2 3)
;;=> (-60 -60 -60)
notice, though, that in practice this implementation isn't really better than yours: while recur saves you from stack overflow on function creation, it would still overflow on application (somebody, correct me if i'm wrong):
(apply my-comp (repeat 1000000 inc)) ;; ok
((apply my-comp (repeat 1000000 inc)) 1) ;; stack overflow
so it would probably be better to use reduce or something else:
(defn my-comp-reduce [f & fs]
(let [[f & fs] (reverse (cons f fs))]
(fn [& args]
(reduce (fn [acc curr-f] (curr-f acc))
(apply f args)
fs))))
user> ((my-comp-reduce (partial repeat 3) #(* 10 %) - +) 1 2 3)
;;=> (-60 -60 -60)
user> ((apply my-comp-reduce (repeat 1000000 inc)) 1)
;;=> 1000001
There is already a good answer above, but I think the original suggestion to use recur may have been thinking of a more manual accumulation of the result. In case you haven't seen it, reduce is just a very specific usage of loop/recur:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn my-reduce
[step-fn init-val data-vec]
(loop [accum init-val
data data-vec]
(if (empty? data)
accum
(let [accum-next (step-fn accum (first data))
data-next (rest data)]
(recur accum-next data-next)))))
(dotest
(is= 10 (my-reduce + 0 (range 5))) ; 0..4
(is= 120 (my-reduce * 1 (range 1 6))) ; 1..5 )
In general, there can be any number of loop variables (not just 2 like for reduce). Using loop/recur gives you a more "functional" way of looping with accumulated state instead of using and atom and a doseq or something. As the name suggests, from the outside the effect is quite similar to a normal recursion w/o any stack size limits (i.e. tail-call optimization).
P.S. As this example shows, I like to use a let form to very explicitly name the values being generated for the next iteration.
P.P.S. While the compiler will allow you to type the following w/o confusion:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn my-reduce
[step-fn accum data]
(loop [accum accum
data data]
...))
it can be a bit confusing and/or sloppy to re-use variable names (esp. for people new to Clojure or your particular program).
Also
I would be remiss if I didn't point out that the function definition itself can be a recur target (i.e. you don't need to use loop). Consider this version of the factorial:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test))
(defn fact-impl
[cum x]
(if (= x 1)
cum
(let [cum-next (* cum x)
x-next (dec x)]
(recur cum-next x-next))))
(defn fact [x] (fact-impl 1 x))
(dotest
(is= 6 (fact 3))
(is= 120 (fact 5)))
I am trying to rewrite this piece of code from https://github.com/lspector/gp/blob/master/src/gp/evolvefn_zip.clj
to use recur:
(defn random-code [depth]
(if (or (zero? depth)
(zero? (rand-int 2)))
(random-terminal)
(let [f (random-function)]
(cons f (repeatedly (get function-table f)
#(random-code (dec depth)))))))
The problem is, I have absolutely no idea how to do that.
The only thing I can think of is something like this:
(defn random-code [depth]
(loop [d depth t 0 c []]
(if (or (zero? depth)
(zero? (rand-int 2)))
(if (= t 0)
(conj c random-terminal)
(recur depth (dec t) (conj c (random-terminal))))
(let [f (random-function)]
(if (= t 0)
(recur (dec depth) (function-table f) (conj c f))
(recur depth (dec t) (conj c f)))))))
It's not a working piece of code, it's just to show the way I would try to solve it, it would only get more and more convoluted.
Is there a better way to convert normal recursion to tail recursion in clojure?
Here are 2 examples comparing a recursive algorithm and loop-recur:
(defn fact-recursion [n]
(if (zero? n)
1
(* n (fact-recursion (dec n)))))
(defn fact-recur [n]
(loop [count n
result 1]
(if (pos? count)
(recur (dec count) (* result count))
result )))
(fact-recursion 5) => 120
(fact-recur 5) => 120
(defn rangy-recursive [n]
(if (pos? n)
(cons n (rangy-recursive (dec n)))
[n]))
(defn rangy-recur [n]
(loop [result []
count n]
(if (pos? count)
(recur (conj result count) (dec count))
result)))
(rangy-recursive 5) => (5 4 3 2 1 0)
(rangy-recur 5) => [5 4 3 2 1]
The basic difference is that for loop-recur you need a 2nd loop "variable" (here named result) to accumulate the output of the algorithm. For plain recursion, the call stack accumulates the intermediate result.
So I am trying to re-implement the reduce method, so it can add a couple of numbers that normally can be done using reduce, like:
(reduce + [1 2 3]) ;; 6
(newRd + [1 2 3]) ;; 6
So I thought maybe it can be done using a recursive function that adds the first element of the vector every time it is called and do it again for the rest of the vector. Something like this:
(defn newRd [list]
(let [sum 0]
(if (not= 0 (count list))
(+ sum (first list))
(newRd (rest list))
)
)
)
I think I am not doing local storing correctly. Any suggestion or maybe a better approach?
there are two mistakes here in your code:
1) you don't add your current sum to the recursive call result
2) you should return zero when the list is empty
corrected variant:
(defn newRd [list]
(let [sum 0]
(if (not= 0 (count list))
(+ sum (first list)
(newRd (rest list)))
sum)))
in repl:
user> (newRd [1 2 3 4])
10
next, you can update it a bit:
first you don't really need the sum in let statement, since the sum always = 0
second, there is a lib function empty? to check if list is empty.
(defn newRd [list]
(if-not (empty? list)
(+ (first list)
(newRd (rest list)))
0))
but remember: clojure doesnt'do tail call optimization, so it is easy to cause stack owerflow with a long list:
user> (newRd (repeat 1000000 1))
StackOverflowError user/newRd (form-init289434844644272272.clj:73)
so it's preferable to use loop/recur
(defn sum-list [list]
(loop [list list sum 0]
(if (empty? list)
sum
(recur (rest list) (+ sum (first list))))))
in repl:
user> (sum-list (repeat 1000000 1))
1000000
the other option is to make the function itself tail recursive:
(defn newRd [list sum]
(if-not (empty? list)
(recur (rest list) (+ sum (first list)))
sum))
user> (newRd (repeat 1000000 1) 0)
1000000
then you can add the additoinal arity, for not to pass the second parameter in every call:
(defn newRd
([list] (newRd list 0))
([list sum]
(if-not (empty? list)
(recur (rest list) (+ sum (first list)))
sum)))
Further to leetwinksi's answer ...
You might as well implement new-reduce (camel case is not idiomatic) in general:
(defn new-reduce
([f init coll]
(if (seq coll)
(recur f (f init (f init (first coll))) (rest coll))
init))
([f coll]
(if (seq coll)
(reduce f (first coll) (rest coll))
(f))))
Then
(new-reduce + [1 2 3]) ;; 6
This is more or less what the source code for reduce looked like until recently, if you stripped chunking out.
The two-argument version that you use leans on the three-argument version, which you can recur on directly, without an explicit loop. This entails passing f each time, but that's what it used to do. Presumably it's faster to carry an extra argument than to work in a local scope.
I'm trying to solve this problem in a pure-functional way, without using set!.
I've written a function that calls a given lambda for each number in the fibonacci series, forever.
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(next b (+ a b)))))
(next 0 1)))
I think this is as succinct as it can be, but if I can shorten this, please enlighten me :)
With a definition like the above, is it possible to write another function that takes the first n numbers from the fibonacci series and gives me a list back, but without using variable mutation to track the state (which I understand is not really functional).
The function signature doesn't need to be the same as the following... any approach that will utilize each-fib without using set! is fine.
(take-n-fibs 7) ; (0 1 1 2 3 5 8)
I'm guessing there's some sort of continuations + currying trick I can use, but I keep coming back to wanting to use set!, which is what I'm trying to avoid (purely for learning purposes/shifting my thinking to purely functional).
Try this, implemented using lazy code by means of delayed evaluation:
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(delay (next b (+ a b))))))
(next 0 1)))
(define (take-n-fibs n fn)
(let loop ((i n)
(promise (each-fib fn)))
(when (positive? i)
(loop (sub1 i) (force promise)))))
As has been mentioned, each-fib can be further simplified by using a named let:
(define (each-fib fn)
(let next ((a 0) (b 1))
(fn a)
(delay (next b (+ a b)))))
Either way, it was necessary to modify each-fib a little for using the delay primitive, which creates a promise:
A promise encapsulates an expression to be evaluated on demand via force. After a promise has been forced, every later force of the promise produces the same result.
I can't think of a way to stop the original (unmodified) procedure from iterating indefinitely. But with the above change in place, take-n-fibs can keep forcing the lazy evaluation of as many values as needed, and no more.
Also, take-n-fibs now receives a function for printing or processing each value in turn, use it like this:
(take-n-fibs 10 (lambda (n) (printf "~a " n)))
> 0 1 1 2 3 5 8 13 21 34 55
You provide an iteration function over fibonacci elements. If you want, instead of iterating over each element, to accumulate a result, you should use a different primitive that would be a fold (or reduce) rather than an iter.
(It might be possible to use continuations to turn an iter into a fold, but that will probably be less readable and less efficient that a direct solution using either a fold or mutation.)
Note however that using an accumulator updated by mutation is also fine, as long as you understand what you are doing: you are using mutable state locally for convenience, but the function take-n-fibs is, seen from the outside, observationally pure, so you do not "contaminate" your program as a whole with side effects.
A quick prototype for fold-fib, adapted from your own code. I made an arbitrary choice as to "when stop folding": if the function returns null, we return the current accumulator instead of continuing folding.
(define (fold-fib init fn) (letrec ([next (lambda (acc a b)
(let ([acc2 (fn acc a)])
(if (null? acc2) acc
(next acc2 b (+ a b)))))])
(next init 0 1)))
(reverse (fold-fib '() (lambda (acc n) (if (> n 10) null (cons n acc)))))
It would be better to have a more robust convention to end folding.
I have written few variants. First you ask if
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(next b (+ a b)))))
(next 0 1)))
can be written any shorter. The pattern is used so often that special syntax called named let has been introduced. Your function looks like this using a named let:
(define (each-fib fn)
(let next ([a 0] [b 1])
(fn a)
(next b (+ a b))))
In order to get the control flowing from one function to another, one can in languages with supports TCO use continuation passing style. Each function gets an extra argument often called k (for continuation). The function k represents what-to-do-next.
Using this style, one can write your program as follows:
(define (generate-fibs k)
(let next ([a 0] [b 1] [k k])
(k a (lambda (k1)
(next b (+ a b) k1)))))
(define (count-down n k)
(let loop ([n n] [fibs '()] [next generate-fibs])
(if (zero? n)
(k fibs)
(next (λ (a next)
(loop (- n 1) (cons a fibs) next))))))
(count-down 5 values)
Now it is a bit annoying to write in style manually, so it could
be convenient to introduce the co-routines. Breaking your rule of not using set! I have chosen to use a shared variable fibs in which generate-fibs repeatedly conses new fibonacci numbers onto. The count-down routine merely read the values, when the count down is over.
(define (make-coroutine co-body)
(letrec ([state (lambda () (co-body resume))]
[resume (lambda (other)
(call/cc (lambda (here)
(set! state here)
(other))))])
(lambda ()
(state))))
(define fibs '())
(define generate-fib
(make-coroutine
(lambda (resume)
(let next ([a 0] [b 1])
(set! fibs (cons a fibs))
(resume count-down)
(next b (+ a b))))))
(define count-down
(make-coroutine
(lambda (resume)
(let loop ([n 10])
(if (zero? n)
fibs
(begin
(resume generate-fib)
(loop (- n 1))))))))
(count-down)
And a bonus you get a version with communicating threads:
#lang racket
(letrec ([result #f]
[count-down
(thread
(λ ()
(let loop ([n 10] [fibs '()])
(if (zero? n)
(set! result fibs)
(loop (- n 1) (cons (thread-receive) fibs))))))]
[produce-fibs
(thread
(λ ()
(let next ([a 0] [b 1])
(when (thread-running? count-down)
(thread-send count-down a)
(next b (+ a b))))))])
(thread-wait count-down)
result)
The thread version is Racket specific, the others ought to run anywhere.
Building a list would be hard. But displaying the results can still be done (in a very bad fashion)
#lang racket
(define (each-fib fn)
(letrec
((next (lambda (a b)
(fn a)
(next b (+ a b)))))
(next 0 1)))
(define (take-n-fibs n fn)
(let/cc k
(begin
(each-fib (lambda (x)
(if (= x (fib (+ n 1)))
(k (void))
(begin
(display (fn x))
(newline))))))))
(define fib
(lambda (n)
(letrec ((f
(lambda (i a b)
(if (<= n i)
a
(f (+ i 1) b (+ a b))))))
(f 1 0 1))))
Notice that i am using the regular fibonacci function as an escape (like i said, in a very bad fashion). I guess nobody will recommend programming like this.
Anyway
(take-n-fibs 7 (lambda (x) (* x x)))
0
1
1
4
9
25
64
I'm a newcomer to clojure who wanted to see what all the fuss is about. Figuring the best way to get a feel for it is to write some simple code, I thought I'd start with a Fibonacci function.
My first effort was:
(defn fib [x, n]
(if (< (count x) n)
(fib (conj x (+ (last x) (nth x (- (count x) 2)))) n)
x))
To use this I need to seed x with [0 1] when calling the function. My question is, without wrapping it in a separate function, is it possible to write a single function that only takes the number of elements to return?
Doing some reading around led me to some better ways of achieving the same funcionality:
(defn fib2 [n]
(loop [ x [0 1]]
(if (< (count x) n)
(recur (conj x (+ (last x) (nth x (- (count x) 2)))))
x)))
and
(defn fib3 [n]
(take n
(map first (iterate (fn [[a b]] [b (+ a b)]) [0 1]))))
Anyway, more for the sake of the exercise than anything else, can anyone help me with a better version of a purely recursive Fibonacci function? Or perhaps share a better/different function?
To answer you first question:
(defn fib
([n]
(fib [0 1] n))
([x, n]
(if (< (count x) n)
(fib (conj x (+ (last x) (nth x (- (count x) 2)))) n)
x)))
This type of function definition is called multi-arity function definition. You can learn more about it here: http://clojure.org/functional_programming
As for a better Fib function, I think your fib3 function is quite awesome and shows off a lot of functional programming concepts.
This is fast and cool:
(def fib (lazy-cat [0 1] (map + fib (rest fib))))
from:
http://squirrel.pl/blog/2010/07/26/corecursion-in-clojure/
In Clojure it's actually advisable to avoid recursion and instead use the loop and recur special forms. This turns what looks like a recursive process into an iterative one, avoiding stack overflows and improving performance.
Here's an example of how you'd implement a Fibonacci sequence with this technique:
(defn fib [n]
(loop [fib-nums [0 1]]
(if (>= (count fib-nums) n)
(subvec fib-nums 0 n)
(let [[n1 n2] (reverse fib-nums)]
(recur (conj fib-nums (+ n1 n2)))))))
The loop construct takes a series of bindings, which provide initial values, and one or more body forms. In any of these body forms, a call to recur will cause the loop to be called recursively with the provided arguments.
You can use the thrush operator to clean up #3 a bit (depending on who you ask; some people love this style, some hate it; I'm just pointing out it's an option):
(defn fib [n]
(->> [0 1]
(iterate (fn [[a b]] [b (+ a b)]))
(map first)
(take n)))
That said, I'd probably extract the (take n) and just have the fib function be a lazy infinite sequence.
(def fib
(->> [0 1]
(iterate (fn [[a b]] [b (+ a b)]))
(map first)))
;;usage
(take 10 fib)
;;output (0 1 1 2 3 5 8 13 21 34)
(nth fib 9)
;; output 34
A good recursive definition is:
(def fib
(memoize
(fn [x]
(if (< x 2) 1
(+ (fib (dec (dec x))) (fib (dec x)))))))
This will return a specific term. Expanding this to return first n terms is trivial:
(take n (map fib (iterate inc 0)))
Here is the shortest recursive function I've come up with for computing the nth Fibonacci number:
(defn fib-nth [n] (if (< n 2)
n
(+ (fib-nth (- n 1)) (fib-nth (- n 2)))))
However, the solution with loop/recursion should be faster for all but the first few values of 'n' since Clojure does tail-end optimization on loop/recur.
this is my approach
(defn fibonacci-seq [n]
(cond
(= n 0) 0
(= n 1) 1
:else (+ (fibonacci-seq (- n 1)) (fibonacci-seq (- n 2)))
)
)
For latecomers. Accepted answer is a slightly complicated expression of this:
(defn fib
([n]
(fib [0 1] n))
([x, n]
(if (< (count x) n)
(recur (conj x (apply + (take-last 2 x))) n)
x)))
For what it's worth, lo these years hence, here's my solution to 4Closure Problem #26: Fibonacci Sequence
(fn [x]
(loop [i '(1 1)]
(if (= x (count i))
(reverse i)
(recur
(conj i (apply + (take 2 i)))))))
I don't, by any means, think this is the optimal or most idiomatic approach. The whole reason I'm going through the exercises at 4Clojure ... and mulling over code examples from Rosetta Code is to learn clojure.
Incidentally I'm well aware that the Fibonacci sequence formally includes 0 ... that this example should loop [i '(1 0)] ... but that wouldn't match their spec. nor pass their unit tests despite how they've labelled this exercise. It is written as an anonymous recursive function in order to conform to the requirements for the 4Clojure exercises ... where you have to "fill in the blank" within a given expression. (I'm finding the whole notion of anonymous recursion to be a bit of a mind bender; I get that the (loop ... (recur ... special form is constrained to tail-recursion ... but it's still a weird syntax to me).
I'll take #[Arthur Ulfeldt]'s comment, regarding fib3 in the original posting, under consideration as well. I've only used Clojure's iterate once, so far.