I am writing some signal processing software, and I am starting off by writing out a discrete convolution function.
This works fine for the first ten thousand or so list of values, but as they get larger (say, 100k), I begin to get StackOverflow errors, of course.
Unfortunately, I am having a lot of trouble converting the imperative convolution algorithm I have to a recursive & lazy version that is actually fast enough to use (having at least a modicum of elegance would be nice as well).
I am also not 100% sure I have this function completely right, yet – please let me know if I'm missing something/doing something wrong. I think it's correct.
(defn convolve
"
Convolves xs with is.
This is a discrete convolution.
'xs :: list of numbers
'is :: list of numbers
"
[xs is]
(loop [xs xs finalacc () acc ()]
(if (empty? xs)
(concat finalacc acc)
(recur (rest xs)
(if (empty? acc)
()
(concat finalacc [(first acc)]))
(if (empty? acc)
(map #(* (first xs) %) is)
(vec-add
(map #(* (first xs) %) is)
(rest acc)))))))
I'd be much obliged for any sort of help: I'm still getting my bearings in Clojure, and making this elegant and lazy and/or recursive would be wonderful.
I'm a little surprised how difficult it is to express an algorithm which is quite easy to express in an imperative language in a Lisp. But perhaps I'm doing it wrong!
EDIT:
Just to show how easy it is to express in an imperative language, and to give people the algorithm that works nicely and is easy to read, here is the Python version. Aside from being shorter, more concise and far easier to reason about, it executes orders of magnitude faster than the Clojure code: even my imperative Clojure code using Java arrays.
from itertools import repeat
def convolve(ns, ms):
y = [i for i in repeat(0, len(ns)+len(ms)-1)]
for n in range(len(ns)):
for m in range(len(ms)):
y[n+m] = y[n+m] + ns[n]*ms[m]
return y
Here, on the other hand, is the imperative Clojure code. It also drops the last, non fully-immersed, values from the convolution. So aside from being slow and ugly, it's not 100% functional. Nor functional.
(defn imp-convolve-1
[xs is]
(let [ys (into-array Double (repeat (dec (+ (count xs) (count is))) 0.0))
xs (vec xs)
is (vec is)]
(map #(first %)
(for [i (range (count xs))]
(for [j (range (count is))]
(aset ys (+ i j)
(+ (* (nth xs i) (nth is j))
(nth ys (+ i j)))))))))
This is so disheartening. Please, someone show me I've just missed something obvious.
EDIT 3:
Here's another version I thought up yesterday, showing how I'd like to be able express it (though other solutions are quite elegant; I'm just putting another one out there!)
(defn convolve-2
[xs is]
(reduce #(vec-add %1 (pad-l %2 (inc (count %1))))
(for [x xs]
(for [i is]
(* x i)))))
It uses this utility function vec-add:
(defn vec-add
([xs] (vec-add xs []))
([xs ys]
(let [lxs (count xs)
lys (count ys)
xs (pad-r xs lys)
ys (pad-r ys lxs)]
(vec (map #(+ %1 %2) xs ys))))
([xs ys & more]
(vec (reduce vec-add (vec-add xs ys) more))))
(vec (reduce vec-add (vec-add xs ys) more))))
(defn ^{:static true} convolve ^doubles [^doubles xs ^doubles is]
(let [xlen (count xs)
ilen (count is)
ys (double-array (dec (+ xlen ilen)))]
(dotimes [p xlen]
(dotimes [q ilen]
(let [n (+ p q), x (aget xs p), i (aget is q), y (aget ys n)]
(aset ys n (+ (* x i) y)))))
ys))
Riffing on j-g-faustus's version if I was doing this in the Clojure equiv branch. Works for me. ~400ms for 1,000,000 points, ~25ms for 100,000 on a i7 Mackbook Pro.
The likely cause of the stack overflow errors is that the lazy thunks are getting too deep. (concat and map are lazy). Try wrapping those calls in doall to force evaluation of their return values.
As for a more functional solution, try something like this:
(defn circular-convolve
"Perform a circular convolution of vectors f and g"
[f g]
(letfn [(point-mul [m n]
(* (f m) (g (mod (- n m) (count g)))))
(value-at [n]
(reduce + (map #(point-mul % n) (range (count g)))))]
(map value-at (range (count g)))))
Use can use reduce to perform summation easily, and since map produces a lazy sequence, this function is also lazy.
Can't help with a high-performance functional version, but you can get a 100-fold speedup for the imperative version by foregoing laziness and adding type hints:
(defn imp-convolve-2 [xs is]
(let [^doubles xs (into-array Double/TYPE xs)
^doubles is (into-array Double/TYPE is)
ys (double-array (dec (+ (count xs) (count is)))) ]
(dotimes [i (alength xs)]
(dotimes [j (alength is)]
(aset ys (+ i j)
(+ (* (aget xs i) (aget is j))
(aget ys (+ i j))))))
ys))
With xs size 100k and is size 2, your imp-convolve-1 takes ~6,000ms on my machine when wrapped in a doall, while this one takes ~35ms.
Update
Here is a lazy functional version:
(defn convolve
([xs is] (convolve xs is []))
([xs is parts]
(cond
(and (empty? xs) (empty? parts)) nil
(empty? xs) (cons
(reduce + (map first parts))
(convolve xs is
(remove empty? (map rest parts))))
:else (cons
(+ (* (first xs) (first is))
(reduce + (map first parts)))
(lazy-seq
(convolve (rest xs) is
(cons
(map (partial * (first xs)) (rest is))
(remove empty? (map rest parts)))))))))
On sizes 100k and 2, it clocks in at ~600ms (varying 450-750ms) vs ~6,000ms for imp-convolve-1 and ~35ms for imp-convolve-2.
So it's functional, lazy and has tolerable performance. Still, it's twice as much code as the imperative version and took me 1-2 additional hours to find, so I'm not sure that I see the point.
I'm all for pure functions when they make the code shorter or simpler, or have some other benefit over an imperative version. When they don't, I have no objection to switch to imperative mode.
Which is one of the reasons I think Clojure is great, since you can use either approach as you see fit.
Update 2:
I'll amend my "what's the point of doing this functionally" by saying that I like this functional implementation (the second one, further down the page) by David Cabana.
It's brief, readable and times to ~140ms with the same input sizes as above (100,000 and 2), making it by far the best-performing functional implementation of those I tried.
Considering that it is functional (but not lazy), uses no type hints and works for all numeric types (not just doubles), that's quite impressive.
(defn convolve [xs is]
(if (> (count xs) (count is))
(convolve is xs)
(let [os (dec (+ (count xs) (count is)))
lxs (count xs)
lis (count is)]
(for [i (range os)]
(let [[start-x end-x] [(- lxs (min lxs (- os i))) (min i (dec lxs))]
[start-i end-i] [(- lis (min lis (- os i))) (min i (dec lis))]]
(reduce + (map *
(rseq (subvec xs start-x (inc end-x)))
(subvec is start-i (inc end-i)))))))))
It is possible to express a lazy, functional solution in concise terms. Alas, the performance for > 2k is impractical. I'm interested to see if there are ways to speed it up without sacrificing readability.
Edit:
After reading drcabana's informative post on the topic (http://erl.nfshost.com/2010/07/17/discrete-convolution-of-finite-vectors/), I've updated my code to accept different sized vectors. His implementation is better performing:
for xs size 3, is size 1000000, ~2s vs ~3s
Edit 2:
Taking drcabana's ideas of simply reversing xs and padding is, I arrived at:
(defn convolve [xs is]
(if (> (count xs) (count is))
(convolve is xs)
(let [is (concat (repeat (dec (count xs)) 0) is)]
(for [s (take-while not-empty (iterate rest is))]
(reduce + (map * (rseq xs) s))))))
This is probably as concise as it's going to get, but it is still slower overall, likely due to take-while. Kudos to the blog author for a well considered approach. The only advantage here is that the above is truly lazy in that if I ask (nth res 10000), it will only need the first 10k calculations to arrive at a result.
Not really an answer to any of the many questions you asked, but I have several comments on the ones you didn't ask.
You probably shouldn't use nth on vectors. Yes, it's O(1), but because nth works on other sequences in O(n), it (a) doesn't make it clear that you expect the input to be a vector, and (b) means if you make a mistake, your code will mysteriously get really slow instead of failing immediately.
for and map are both lazy, and aset is side-effects-only. This combination is a recipe for disaster: for side-effecting for-like behavior, use doseq.
for and doseq allow multiple bindings, so you don't need to pile up loads of them like you (apparently) do in Python.
(doseq [i (range (count cs))
j (range (count is))]
...)
will do what you want.
#(first %) is more concisely written as first; likewise #(+ %1 %2) is +.
Calling vec on a bunch of intermediate results that don't need to be vectors will slow you down. Specifically in vec-add it's sufficient to only call vec when you make a final return value: in (vec (reduce foo bar)) there's no reason for foo to turn its intermediate results into vectors if it never uses them for random access.
Related
Hi i am looking for a bit of help with some Clojure code. I have written a function that will take in a list and calculate the qty*price for a list eg. '(pid3 6 9)
What i am looking for is to expand my current function so that it recursively does the qty*price calculation until it reaches the end of the list.
My current function is written like this:
(defn pid-calc [list] (* (nth list 1) (nth list 2)))
I have tried implementing it into a recursive function but have had no luck at all, i want to be able to call something like this:
(pid-calcc '( (pid1 8 5) (pid2 5 6))
return==> 70
Thats as close as i have came to an answer and cannot seem to find one. If anyone can help me find a solution i that would be great. As so far i am yet to find anything that will compile.
(defn pid-calc [list]
(if(empty? list)
nil
(* (nth list 1) (nth list 2)(+(pid-calc (rest list))))))
You don't need a recursive function. Just use + and map:
(defn pid-calc [list]
(letfn [(mul [[_ a b]] (* a b))]
(apply + (map mul list))))
#sloth's answer, suitably corrected, is a concise and fast enough way to solve your problem. It shows you a lot.
Your attempt at a recursive solution can be (a)mended to
(defn pid-calc [list]
(if (empty? list)
0
(let [x (first list)]
(+ (* (nth x 1) (nth x 2)) (pid-calc (next list))))))
This works on the example, but - being properly recursive - will run out of stack space on a long enough list. The limit is usually about 10K items.
We can get over this without being so concise as #sloth. You might find the following easier to understand:
(defn pid-calc [list]
(let [line-total (fn [item] (* (nth item 1) (nth item 2)))]
(apply + (map line-total list))))
reduce fits your scenario quite well:
(def your-list [[:x 1 2] [:x 1 3]])
(reduce #(+ %1 (* (nth %2 1) (nth %2 2))) 0 your-list)
(reduce #(+ %1 (let [[_ a b] %2] (* a b)) 0 your-list)
In clojure, I would like to write a tail-recursive function that memoizes its intermediate results for subsequent calls.
[EDIT: this question has been rewritten using gcd as an example instead of factorial.]
The memoized gcd (greatest common divisor) could be implemented like this:
(def gcd (memoize (fn [a b]
(if (zero? b)
a
(recur b (mod a b))))
In this implementation, intermediate results are not memoized for subsequent calls. For example, in order to calculate gcd(9,6), gcd(6,3) is called as an intermediate result. However, gcd(6,3) is not stored in the cache of the memoized function because the recursion point of recur is the anonymous function that is not memoized.
Therefore, if after having called gcd(9,6), we call gcd(6,3) we won't benefit from the memoization.
The only solution I can think about will be to use mundane recursion (explicitely call gcd instead of recur) but then we will not benefit from Tail Call Optimization.
Bottom Line
Is there a way to achieve both:
Tail call optimization
Memoization of intermediate results for subsequent calls
Remarks
This question is similar to Combine memoization and tail-recursion. But all the answers there are related to F#. Here, I am looking for an answer in clojure.
This question has been left as an exercise for the reader by The Joy of Clojure (chap 12.4). You can consult the relevant page of the book at http://bit.ly/HkQrio.
in your case it's hard to show memoize do anything with factorial because the intermediate calls are unique, so I'll rewrite a somewhat contrived example assuming the point is to explore ways to avoid blowing the stack:
(defn stack-popper [n i]
(if (< i n) (* i (stack-popper n (inc i))) 1))
which can then get something out of a memoize:
(def stack-popper
(memoize (fn [n i] (if (< i n) (* i (stack-popper n (inc i))) 1))))
the general approaches to not blowing the stack are:
use tail calls
(def stack-popper
(memoize (fn [n acc] (if (> n 1) (recur (dec n) (* acc (dec n))) acc))))
use trampolines
(def stack-popper
(memoize (fn [n acc]
(if (> n 1) #(stack-popper (dec n) (* acc (dec n))) acc))))
(trampoline (stack-popper 4 1))
use a lazy sequence
(reduce * (range 1 4))
None of these work all the time, though I have yet to hit a case where none of them work. I almost always go for the lazy ones first because I find them to be most clojure like, then I head for tail calling with recur or tramplines
(defmacro memofn
[name args & body]
`(let [cache# (atom {})]
(fn ~name [& args#]
(let [update-cache!# (fn update-cache!# [state# args#]
(if-not (contains? state# args#)
(assoc state# args#
(delay
(let [~args args#]
~#body)))
state#))]
(let [state# (swap! cache# update-cache!# args#)]
(-> state# (get args#) deref))))))
This will allow a recursive definition of a memoized function, which also caches intermediate results. Usage:
(def fib (memofn fib [n]
(case n
1 1
0 1
(+ (fib (dec n)) (fib (- n 2))))))
(def gcd
(let [cache (atom {})]
(fn [a b]
#(or (#cache [a b])
(let [p (promise)]
(deliver p
(loop [a a b b]
(if-let [p2 (#cache [a b])]
#p2
(do
(swap! cache assoc [a b] p)
(if (zero? b)
a
(recur b (mod a b))))))))))))
There is some concurrency issues (double evaluation, the same problem as with memoize, but worse because of the promises) which may be fixed using #kotarak's advice.
Turning the above code into a macro is left as an exercise to the reader. (Fogus's note was imo tongue-in-cheek.)
Turning this into a macro is really a simple exercise in macrology, please remark that the body (the 3 last lines) remain unchanged.
Using Clojure's recur you can write factorial using an accumulator that has no stack growth, and just memoize it:
(defn fact
([n]
(fact n 1))
([n acc]
(if (= 1 n)
acc
(recur (dec n)
(* n acc)))))
This is factorial function implemented with anonymous recursion with tail call and memoization of intermediate results. The memoization is integrated with the function and a reference to shared buffer (implemented using Atom reference type) is passed by a lexical closure.
Since the factorial function operates on natural numbers and the arguments for succesive results are incremental, Vector seems more tailored data structure to store buffered results.
Instead of passing the result of a previous computation as an argument (accumulator) we're getting it from the buffer.
(def ! ; global variable referring to a function
(let [m (atom [1 1 2 6 24])] ; buffer of results
(fn [n] ; factorial function definition
(let [m-count (count #m)] ; number of results in a buffer
(if (< n m-count) ; do we have buffered result for n?
(nth #m n) ; · yes: return it
(loop [cur m-count] ; · no: compute it recursively
(let [r (*' (nth #m (dec cur)) cur)] ; new result
(swap! m assoc cur r) ; store the result
(if (= n cur) ; termination condition:
r ; · base case
(recur (inc cur)))))))))) ; · recursive case
(time (do (! 8000) nil)) ; => "Elapsed time: 154.280516 msecs"
(time (do (! 8001) nil)) ; => "Elapsed time: 0.100222 msecs"
(time (do (! 7999) nil)) ; => "Elapsed time: 0.090444 msecs"
(time (do (! 7999) nil)) ; => "Elapsed time: 0.055873 msecs"
I'm trying to teach myself clojure and I'm using the principles of Prime Factors Kata and TDD to do so.
Via a series of Midje tests like this:
(fact (primefactors 1) => (list))
(fact (primefactors 2) => (list 2))
(fact (primefactors 3) => (list 3))
(fact (primefactors 4) => (list 2 2))
I was able to create the following function:
(defn primefactors
([n] (primefactors n 2))
([n candidate]
(cond (<= n 1) (list)
(= 0 (rem n candidate)) (conj (primefactors (/ n candidate)) candidate)
:else (primefactors n (inc candidate))
)
)
)
This works great until I throw the following edge case test at it:
(fact (primefactors 1000001) => (list 101 9901))
I end up with a stack overflow error. I know I need to turn this into a proper recur loops but all the examples I see seem to be too simplistic and only point to a counter or numerical variable as the focus. How do I make this recursive?
Thanks!
Here's a tail recursive implementation of the primefactors procedure, it should work without throwing a stack overflow error:
(defn primefactors
([n]
(primefactors n 2 '()))
([n candidate acc]
(cond (<= n 1) (reverse acc)
(zero? (rem n candidate)) (recur (/ n candidate) candidate (cons candidate acc))
:else (recur n (inc candidate) acc))))
The trick is using an accumulator parameter for storing the result. Notice that the reverse call at the end of the recursion is optional, as long as you don't care if the factors get listed in the reverse order they were found.
Your second recursive call already is in the tail positions, you can just replace it with recur.
(primefactors n (inc candidate))
becomes
(recur n (inc candidate))
Any function overload opens an implicit loop block, so you don't need to insert that manually. This should already improve the stack situation somewhat, as this branch will be more commonly taken.
The first recursion
(primefactors (/ n candidate))
isn't in the tail position as its result is passed to conj. To put it in the tail position, you'll need to collect the prime factors in an additional accumulator argument onto which you conj the result from the current recursion level and then pass to recur on each invocation. You'll need to adjust your termination condition to return that accumulator.
The typical way is to include an accumulator as one of the function arguments. Add a 3-arity version to your function definition:
(defn primefactors
([n] (primefactors n 2 '()))
([n candidate acc]
...)
Then modify the (conj ...) form to call (recur ...) and pass (conj acc candidate) as the third argument. Make sure you pass in three arguments to recur, i.e. (recur (/ n candidate) 2 (conj acc candidate)), so that you're calling the 3-arity version of primefactors.
And the (<= n 1) case need to return acc rather than an empty list.
I can go into more detail if you can't figure the solution out for yourself, but I thought I should give you a chance to try to work it out first.
This function really shouldn't be tail-recursive: it should build a lazy sequence instead. After all, wouldn't it be nice to know that 4611686018427387902 is non-prime (it's divisible by two), without having to crunch the numbers and find that its other prime factor is 2305843009213693951?
(defn prime-factors
([n] (prime-factors n 2))
([n candidate]
(cond (<= n 1) ()
(zero? (rem n candidate)) (cons candidate (lazy-seq (prime-factors (/ n candidate)
candidate)))
:else (recur n (inc candidate)))))
The above is a fairly unimaginative translation of the algorithm you posted; of course better algorithms exist, but this gets you correctness and laziness, and fixes the stack overflow.
A tail recursive, accumulator-free, lazy-sequence solution:
(defn prime-factors [n]
(letfn [(step [n div]
(when (< 1 n)
(let [q (quot n div)]
(cond
(< q div) (cons n nil)
(zero? (rem n div)) (cons div (lazy-step q div))
:else (recur n (inc div))))))
(lazy-step [n div]
(lazy-seq
(step n div)))]
(lazy-step n 2)))
Recursive calls embedded in lazy-seq are not evaluated before iteration upon the sequence, eliminating the risks of stack-overflow without resorting to an accumulator.
I'm a newcomer to clojure who wanted to see what all the fuss is about. Figuring the best way to get a feel for it is to write some simple code, I thought I'd start with a Fibonacci function.
My first effort was:
(defn fib [x, n]
(if (< (count x) n)
(fib (conj x (+ (last x) (nth x (- (count x) 2)))) n)
x))
To use this I need to seed x with [0 1] when calling the function. My question is, without wrapping it in a separate function, is it possible to write a single function that only takes the number of elements to return?
Doing some reading around led me to some better ways of achieving the same funcionality:
(defn fib2 [n]
(loop [ x [0 1]]
(if (< (count x) n)
(recur (conj x (+ (last x) (nth x (- (count x) 2)))))
x)))
and
(defn fib3 [n]
(take n
(map first (iterate (fn [[a b]] [b (+ a b)]) [0 1]))))
Anyway, more for the sake of the exercise than anything else, can anyone help me with a better version of a purely recursive Fibonacci function? Or perhaps share a better/different function?
To answer you first question:
(defn fib
([n]
(fib [0 1] n))
([x, n]
(if (< (count x) n)
(fib (conj x (+ (last x) (nth x (- (count x) 2)))) n)
x)))
This type of function definition is called multi-arity function definition. You can learn more about it here: http://clojure.org/functional_programming
As for a better Fib function, I think your fib3 function is quite awesome and shows off a lot of functional programming concepts.
This is fast and cool:
(def fib (lazy-cat [0 1] (map + fib (rest fib))))
from:
http://squirrel.pl/blog/2010/07/26/corecursion-in-clojure/
In Clojure it's actually advisable to avoid recursion and instead use the loop and recur special forms. This turns what looks like a recursive process into an iterative one, avoiding stack overflows and improving performance.
Here's an example of how you'd implement a Fibonacci sequence with this technique:
(defn fib [n]
(loop [fib-nums [0 1]]
(if (>= (count fib-nums) n)
(subvec fib-nums 0 n)
(let [[n1 n2] (reverse fib-nums)]
(recur (conj fib-nums (+ n1 n2)))))))
The loop construct takes a series of bindings, which provide initial values, and one or more body forms. In any of these body forms, a call to recur will cause the loop to be called recursively with the provided arguments.
You can use the thrush operator to clean up #3 a bit (depending on who you ask; some people love this style, some hate it; I'm just pointing out it's an option):
(defn fib [n]
(->> [0 1]
(iterate (fn [[a b]] [b (+ a b)]))
(map first)
(take n)))
That said, I'd probably extract the (take n) and just have the fib function be a lazy infinite sequence.
(def fib
(->> [0 1]
(iterate (fn [[a b]] [b (+ a b)]))
(map first)))
;;usage
(take 10 fib)
;;output (0 1 1 2 3 5 8 13 21 34)
(nth fib 9)
;; output 34
A good recursive definition is:
(def fib
(memoize
(fn [x]
(if (< x 2) 1
(+ (fib (dec (dec x))) (fib (dec x)))))))
This will return a specific term. Expanding this to return first n terms is trivial:
(take n (map fib (iterate inc 0)))
Here is the shortest recursive function I've come up with for computing the nth Fibonacci number:
(defn fib-nth [n] (if (< n 2)
n
(+ (fib-nth (- n 1)) (fib-nth (- n 2)))))
However, the solution with loop/recursion should be faster for all but the first few values of 'n' since Clojure does tail-end optimization on loop/recur.
this is my approach
(defn fibonacci-seq [n]
(cond
(= n 0) 0
(= n 1) 1
:else (+ (fibonacci-seq (- n 1)) (fibonacci-seq (- n 2)))
)
)
For latecomers. Accepted answer is a slightly complicated expression of this:
(defn fib
([n]
(fib [0 1] n))
([x, n]
(if (< (count x) n)
(recur (conj x (apply + (take-last 2 x))) n)
x)))
For what it's worth, lo these years hence, here's my solution to 4Closure Problem #26: Fibonacci Sequence
(fn [x]
(loop [i '(1 1)]
(if (= x (count i))
(reverse i)
(recur
(conj i (apply + (take 2 i)))))))
I don't, by any means, think this is the optimal or most idiomatic approach. The whole reason I'm going through the exercises at 4Clojure ... and mulling over code examples from Rosetta Code is to learn clojure.
Incidentally I'm well aware that the Fibonacci sequence formally includes 0 ... that this example should loop [i '(1 0)] ... but that wouldn't match their spec. nor pass their unit tests despite how they've labelled this exercise. It is written as an anonymous recursive function in order to conform to the requirements for the 4Clojure exercises ... where you have to "fill in the blank" within a given expression. (I'm finding the whole notion of anonymous recursion to be a bit of a mind bender; I get that the (loop ... (recur ... special form is constrained to tail-recursion ... but it's still a weird syntax to me).
I'll take #[Arthur Ulfeldt]'s comment, regarding fib3 in the original posting, under consideration as well. I've only used Clojure's iterate once, so far.
I'm new to functional languages and clojure, so please bear with me...
I'm trying to construct a list of functions, with either random parameters or constants. The function that constructs the list of functions is already working, though it doesn't return the function itself. I verified this using println.
(edit: Okay, it isn't working correctly yet after all)
(edit: Now it's working, but it cannot be "eval"-ed. it seems I need to recur at least two times, to ensure there are at least two children nodes. Is this possible?)
Here is the snippet:
(def operations (list #(- %1 %2) #(+ %1 %2) #(* %1 %2) #(/ %1 %2)))
(def parameters (list \u \v \w \x \y \z))
(def parameterlistcount 6)
(def paramcount 2)
(def opcount 4)
(defn generate-function
([] (generate-function 2 4 0.5 0.6 () parameters))
([pc maxdepth fp pp function-list params]
(if (and (pos? maxdepth) (< (rand) fp))
(let [function-list
(conj function-list
(nth operations
(rand-int (count operations))))]
(recur pc (dec maxdepth) fp pp function-list params))
(if (and (< (rand) pp) (pos? pc))
(let [ params (pop parameters)
function-list
(conj function-list
(nth params
(rand-int (count params))))]
(if (contains? (set operations) (last function-list) )
(recur (dec pc) maxdepth fp pp function-list params)
nil))
(let [function-list
(conj function-list
(rand-int 100))]
(if (or (pos? maxdepth) (pos? pc))
(if (contains? (set operations) (last function-list) )
(recur pc maxdepth fp pp function-list params)
nil)
function-list))))))
Any help will be appreciated, thanks!
Here's my shot at rewriting your function (see comments below):
(defn generate-function
([] (generate-function 2 4 0.5 0.6 ()))
([pc maxdepth fp pp function-list]
(if (and (pos? maxdepth) (< (rand) fp))
(let [function-list
(conj function-list
{:op
(nth operations
(rand-int (count operations)))})]
(recur pc (dec maxdepth) fp pp function-list))
(if (and (< (rand) pp) (pos? pc))
(let [function-list
(conj function-list
{:param
(nth parameters
(rand-int (count parameters)))})]
(recur (dec pc) maxdepth fp pp function-list))
(let [function-list
(conj function-list
{:const
(rand-int 100)})]
(if (or (pos? maxdepth) (pos? pc))
(recur pc maxdepth fp pp function-list)
function-list))))))
And some examples of use from my REPL...
user> (generate-function)
({:const 63} {:op #<user$fn__4557 user$fn__4557#6cbb2d>} {:const 77} {:param \w} {:op #<user$fn__4559 user$fn__4559#8e68bd>} {:const 3} {:param \v} {:const 1} {:const 8} {:op #<user$fn__4559 user$fn__4559#8e68bd>} {:op #<user$fn__4555 user$fn__4555#6f0962>})
user> (generate-function)
({:const 27} {:param \y} {:param \v} {:op #<user$fn__4561 user$fn__4561#10463c3>} {:op #<user$fn__4561 user$fn__4561#10463c3>} {:op #<user$fn__4561 user$fn__4561#10463c3>} {:op #<user$fn__4561 user$fn__4561#10463c3>} {:const 61})
A couple of things to keep in mind, in pretty random order:
I used recur in the above to avoid consuming stack in the recursive self-calls. However, you have this dotimes statement which makes me wonder if you might be interested in constructing a bunch of function-lists in parallel with one generate-function call. If so, tail-recursion with recur might not be an option with simplistic code like this, so you could either settle for the regular self-calls (but do consider the possibility of hitting the recursion limit; if you're positive that you'll only generate smallish functions and this won't be a problem, go ahead with the self-calls) or investigate continuation-passing style and rewrite your function in that style.
The (do (dec pc) ...) thing in your code does nothing to the value of pc in the next recursive call, or indeed to its current value. Local variables (or locals, as they are most often called in the community) in Clojure are immutable; this includes function parameters. If you want to pass along a decremented pc to some function, you'll have to do just that, like you did with maxdepth in an earlier branch of your code.
I renamed your function to generate-function, because camel case in function names is quite unusual in Clojure land. Also, I renamed the parameter which you called function to function-list (so maybe I should have used a name like generate-function-list for the function... hm), because that's what it is for now.
Note that there's no point to keeping a separate opcount Var around; Clojure's persistent lists (as created by the list function) carry their count around, so (count some-list) is a constant-time operation (and very fast). Also, it would be idiomatic to use vectors for operations and parameters (and you can switch to vectors without changing anything in the rest of the code!). E.g. [\u \v \w \x \y \z].
In Clojure 1.2, you'll be able to use (rand-nth coll) for (nth coll (rand-int (count coll))).
If you want to generate actual Clojure functions from trees of items representing ops, params and constants, you'll want to use eval. That's discouraged in most scenarios, but not for evolutionary programming and similar stuff where it's the only way to go.
Personally, I'd use a different format of the op/param/constant maps: something like {:category foo, :content bar} where foo is :op, :param or :const and bar is something appropriate in connection to any given foo.
In general it is a better idea in Clojure to use (recur ...) for your recursive functions. From the docs: "Note that recur is the only non-stack-consuming looping construct in Clojure." link
One other thing to note is that you might want to call the randomizer outside the recursive function, so you can define the stop-condition inside the function.
So like this:
(let [n (rand-int 10)]
(println "Let's define f with n =" n)
(defn f [x]
(if (> x n)
"result"
(do (println x)
(recur (inc x))))))
It prints:
Let's define f with n = 4
user> (f 0)
0
1
2
3
4
"result"
where 4 is of course a random number between 0 (inclusive) and 10 (exclusive).
So okay, I discovered I was going about this the wrong way.
A recursive definition of a tree is non other than defining vertices, and trying to tie everything with it. So, I came up with this, in less than 15 minutes. >_<
(defn generate-branch
"Generate branches for tree"
([] (generate-branch 0.6 () (list \x \y \z)))
([pp branch params]
(loop [branch
(conj branch (nth operations (rand-int (count operations))))]
(if (= (count branch) 3)
branch
(if (and (< (rand) pp))
(recur (conj branch (nth params (rand-int (count params)))))
(recur (conj branch (rand-int 100))))))))
(defn build-vertex
"Generates a vertex from branches"
[]
(let [vertex (list (nth operations (rand-int (count operations))))]
(conj vertex (take 5 (repeatedly generate-branch)))))
THANKS EVERYONE!