Is there a way to remove an item from a vector based on index as of now i am using subvec to split the vector and recreate it again. I am looking for the reverse of assoc for vectors?
subvec is probably the best way. The Clojure docs say subvec is "O(1) and very fast, as the resulting vector shares structure with the original and no trimming is done". The alternative would be walking the vector and building a new one while skipping certain elements, which would be slower.
Removing elements from the middle of a vector isn't something vectors are necessarily good at. If you have to do this often, consider using a hash-map so you can use dissoc.
See:
subvec at clojuredocs.org
subvec at clojure.github.io, where the official website points to.
(defn vec-remove
"remove elem in coll"
[pos coll]
(into (subvec coll 0 pos) (subvec coll (inc pos))))
user=> (def a [1 2 3 4 5])
user=> (time (dotimes [n 100000] (vec (concat (take 2 a) (drop 3 a)))))
"Elapsed time: 1185.539413 msecs"
user=> (time (dotimes [n 100000] (vec (concat (subvec a 0 2) (subvec a 3 5)))))
"Elapsed time: 760.072048 msecs"
Yup - subvec is fastest
The vector library clojure.core.rrb-vector provides logarithmic time concatenation and slicing. Assuming you need persistence, and considering what you're asking for, a logarithmic time solution is as fast as theoretically possible. In particular, it is much faster than any solution using clojure's native subvec, as the concat step puts any such solution into linear time.
(require '[clojure.core.rrb-vector :as fv])
(let [s (vec [0 1 2 3 4])]
(fv/catvec (fv/subvec s 0 2) (fv/subvec s 3 5)))
; => [0 1 3 4]
Here is a solution iv found to be nice:
(defn index-exclude [r ex]
"Take all indices execpted ex"
(filter #(not (ex %)) (range r)))
(defn dissoc-idx [v & ds]
(map v (index-exclude (count v) (into #{} ds))))
(dissoc-idx [1 2 3] 1 2)
'(1)
subvec is fast ; combined with transients it gives even better results.
Using criterium to benchmark:
user=> (def len 5)
user=> (def v (vec (range 0 5))
user=> (def i (quot len 2))
user=> (def j (inc i))
; using take/drop
user=> (bench
(vec (concat (take i v) (drop j v))))
; Execution time mean : 817,618757 ns
; Execution time std-deviation : 9,371922 ns
; using subvec
user=> (bench
(vec (concat (subvec v 0 i) (subvec v j len))))
; Execution time mean : 604,501041 ns
; Execution time std-deviation : 8,163552 ns
; using subvec and transients
user=> (bench
(persistent!
(reduce conj! (transient (vec (subvec v 0 i))) (subvec v j len))))
; Execution time mean : 307,819500 ns
; Execution time std-deviation : 4,359432 ns
The speedup is even greater at greater lengths ; the same bench with a len equal to 10000 gives means: 1,368250 ms, 953,565863 µs, 314,387437 µs.
Yet another possibility which ought to work with any sequence and not bomb if the index was out of range...
(defn drop-index [col idx]
(filter identity (map-indexed #(if (not= %1 idx) %2) col)))
It may be faster to get the indexes you want.
(def a [1 2 3 4 5])
(def indexes [0 1 3 4])
(time (dotimes [n 100000] (vec (concat (subvec a 0 2) (subvec a 3 5)))))
"Elapsed time: 69.401787 msecs"
(time (dotimes [n 100000] (mapv #(a %) indexes)))
"Elapsed time: 28.18766 msecs"
Related
I am trying to write up a simple Markovian state space models, that, as the name suggests iteratively looks back one step to predict the next state.
Here is what is supposed to be a MWE, though it is not because I cannot quite figure out how I am supposed to place (recur ... ) in the below code.
;; helper function
(defn dur-call
[S D]
(if (< 1 D)
(- D 1)
(rand-int S)))
;; helper function
(defn trans-call
[S D]
(if (< 1 D)
S
(rand-int 3)))
;; state space model
(defn test-func
[t]
(loop
[S (rand-int 3)]
(if (<= t 0)
[S (rand-int (+ S 1))]
(let [pastS (first (test-func (- t 1)))
pastD (second (test-func (- t 1)))
S (trans-call pastS pastD)]
(recur ...?)
[S (dur-call S pastD)]))))
My target is to calculate some a state at say time t=5 say, in which case the model needs to look back and calculate states t=[0 1 2 3 4] as well. This should, in my mind, be done well with loop/recur but could also be done with reduce perhaps (not sure how, still new to Clojure). My problem is really that it would seemt have to use recur inside let but that should not work given how loop/recur are designed.
your task is really to generate the next item based on the previous one, starting with some seed. In clojure it can be fulfilled by using iterate function:
user> (take 10 (iterate #(+ 2 %) 1))
(1 3 5 7 9 11 13 15 17 19)
you just have to define the function to produce the next value. It could look like this (not sure about the correctness of the computation algorithm, just based on what is in the question):
(defn next-item [[prev-s prev-d :as prev-item]]
(let [s (trans-call prev-s prev-d)]
[s (dur-call s prev-d)]))
and now let's iterate with it, starting from some value:
user> (take 5 (iterate next-item [3 4]))
([3 4] [3 3] [3 2] [3 1] [0 0])
now your test function could be implemented this way:
(defn test-fn [t]
(when (not (neg? t))
(nth (iterate next-item
(let [s (rand-int 3)]
[s (rand-int (inc s))]))
t)))
you can also do it with loop (but it is still less idiomatic):
(defn test-fn-2 [t]
(when (not (neg? t))
(let [s (rand-int 3)
d (rand-int (inc s))]
(loop [results [[s d]]]
(if (< t (count results))
(peek results)
(recur (conj results (next-item (peek results)))))))))
here we pass all the accumulated results to the next iteration of the loop.
also you can introduce the loop's iteration index and just pass around the last result together with it:
(defn test-fn-3 [t]
(when (not (neg? t))
(let [s (rand-int 3)
d (rand-int (inc s))]
(loop [result [s d] i 0]
(if (= i t)
result
(recur (next-item result) (inc i)))))))
and one more example with reduce:
(defn test-fn-4 [t]
(when (not (neg? t))
(reduce (fn [prev _] (next-item prev))
(let [s (rand-int 3)
d (rand-int (inc s))]
[s d])
(range t))))
In clojure, I would like to write a tail-recursive function that memoizes its intermediate results for subsequent calls.
[EDIT: this question has been rewritten using gcd as an example instead of factorial.]
The memoized gcd (greatest common divisor) could be implemented like this:
(def gcd (memoize (fn [a b]
(if (zero? b)
a
(recur b (mod a b))))
In this implementation, intermediate results are not memoized for subsequent calls. For example, in order to calculate gcd(9,6), gcd(6,3) is called as an intermediate result. However, gcd(6,3) is not stored in the cache of the memoized function because the recursion point of recur is the anonymous function that is not memoized.
Therefore, if after having called gcd(9,6), we call gcd(6,3) we won't benefit from the memoization.
The only solution I can think about will be to use mundane recursion (explicitely call gcd instead of recur) but then we will not benefit from Tail Call Optimization.
Bottom Line
Is there a way to achieve both:
Tail call optimization
Memoization of intermediate results for subsequent calls
Remarks
This question is similar to Combine memoization and tail-recursion. But all the answers there are related to F#. Here, I am looking for an answer in clojure.
This question has been left as an exercise for the reader by The Joy of Clojure (chap 12.4). You can consult the relevant page of the book at http://bit.ly/HkQrio.
in your case it's hard to show memoize do anything with factorial because the intermediate calls are unique, so I'll rewrite a somewhat contrived example assuming the point is to explore ways to avoid blowing the stack:
(defn stack-popper [n i]
(if (< i n) (* i (stack-popper n (inc i))) 1))
which can then get something out of a memoize:
(def stack-popper
(memoize (fn [n i] (if (< i n) (* i (stack-popper n (inc i))) 1))))
the general approaches to not blowing the stack are:
use tail calls
(def stack-popper
(memoize (fn [n acc] (if (> n 1) (recur (dec n) (* acc (dec n))) acc))))
use trampolines
(def stack-popper
(memoize (fn [n acc]
(if (> n 1) #(stack-popper (dec n) (* acc (dec n))) acc))))
(trampoline (stack-popper 4 1))
use a lazy sequence
(reduce * (range 1 4))
None of these work all the time, though I have yet to hit a case where none of them work. I almost always go for the lazy ones first because I find them to be most clojure like, then I head for tail calling with recur or tramplines
(defmacro memofn
[name args & body]
`(let [cache# (atom {})]
(fn ~name [& args#]
(let [update-cache!# (fn update-cache!# [state# args#]
(if-not (contains? state# args#)
(assoc state# args#
(delay
(let [~args args#]
~#body)))
state#))]
(let [state# (swap! cache# update-cache!# args#)]
(-> state# (get args#) deref))))))
This will allow a recursive definition of a memoized function, which also caches intermediate results. Usage:
(def fib (memofn fib [n]
(case n
1 1
0 1
(+ (fib (dec n)) (fib (- n 2))))))
(def gcd
(let [cache (atom {})]
(fn [a b]
#(or (#cache [a b])
(let [p (promise)]
(deliver p
(loop [a a b b]
(if-let [p2 (#cache [a b])]
#p2
(do
(swap! cache assoc [a b] p)
(if (zero? b)
a
(recur b (mod a b))))))))))))
There is some concurrency issues (double evaluation, the same problem as with memoize, but worse because of the promises) which may be fixed using #kotarak's advice.
Turning the above code into a macro is left as an exercise to the reader. (Fogus's note was imo tongue-in-cheek.)
Turning this into a macro is really a simple exercise in macrology, please remark that the body (the 3 last lines) remain unchanged.
Using Clojure's recur you can write factorial using an accumulator that has no stack growth, and just memoize it:
(defn fact
([n]
(fact n 1))
([n acc]
(if (= 1 n)
acc
(recur (dec n)
(* n acc)))))
This is factorial function implemented with anonymous recursion with tail call and memoization of intermediate results. The memoization is integrated with the function and a reference to shared buffer (implemented using Atom reference type) is passed by a lexical closure.
Since the factorial function operates on natural numbers and the arguments for succesive results are incremental, Vector seems more tailored data structure to store buffered results.
Instead of passing the result of a previous computation as an argument (accumulator) we're getting it from the buffer.
(def ! ; global variable referring to a function
(let [m (atom [1 1 2 6 24])] ; buffer of results
(fn [n] ; factorial function definition
(let [m-count (count #m)] ; number of results in a buffer
(if (< n m-count) ; do we have buffered result for n?
(nth #m n) ; · yes: return it
(loop [cur m-count] ; · no: compute it recursively
(let [r (*' (nth #m (dec cur)) cur)] ; new result
(swap! m assoc cur r) ; store the result
(if (= n cur) ; termination condition:
r ; · base case
(recur (inc cur)))))))))) ; · recursive case
(time (do (! 8000) nil)) ; => "Elapsed time: 154.280516 msecs"
(time (do (! 8001) nil)) ; => "Elapsed time: 0.100222 msecs"
(time (do (! 7999) nil)) ; => "Elapsed time: 0.090444 msecs"
(time (do (! 7999) nil)) ; => "Elapsed time: 0.055873 msecs"
I'm a newcomer to clojure who wanted to see what all the fuss is about. Figuring the best way to get a feel for it is to write some simple code, I thought I'd start with a Fibonacci function.
My first effort was:
(defn fib [x, n]
(if (< (count x) n)
(fib (conj x (+ (last x) (nth x (- (count x) 2)))) n)
x))
To use this I need to seed x with [0 1] when calling the function. My question is, without wrapping it in a separate function, is it possible to write a single function that only takes the number of elements to return?
Doing some reading around led me to some better ways of achieving the same funcionality:
(defn fib2 [n]
(loop [ x [0 1]]
(if (< (count x) n)
(recur (conj x (+ (last x) (nth x (- (count x) 2)))))
x)))
and
(defn fib3 [n]
(take n
(map first (iterate (fn [[a b]] [b (+ a b)]) [0 1]))))
Anyway, more for the sake of the exercise than anything else, can anyone help me with a better version of a purely recursive Fibonacci function? Or perhaps share a better/different function?
To answer you first question:
(defn fib
([n]
(fib [0 1] n))
([x, n]
(if (< (count x) n)
(fib (conj x (+ (last x) (nth x (- (count x) 2)))) n)
x)))
This type of function definition is called multi-arity function definition. You can learn more about it here: http://clojure.org/functional_programming
As for a better Fib function, I think your fib3 function is quite awesome and shows off a lot of functional programming concepts.
This is fast and cool:
(def fib (lazy-cat [0 1] (map + fib (rest fib))))
from:
http://squirrel.pl/blog/2010/07/26/corecursion-in-clojure/
In Clojure it's actually advisable to avoid recursion and instead use the loop and recur special forms. This turns what looks like a recursive process into an iterative one, avoiding stack overflows and improving performance.
Here's an example of how you'd implement a Fibonacci sequence with this technique:
(defn fib [n]
(loop [fib-nums [0 1]]
(if (>= (count fib-nums) n)
(subvec fib-nums 0 n)
(let [[n1 n2] (reverse fib-nums)]
(recur (conj fib-nums (+ n1 n2)))))))
The loop construct takes a series of bindings, which provide initial values, and one or more body forms. In any of these body forms, a call to recur will cause the loop to be called recursively with the provided arguments.
You can use the thrush operator to clean up #3 a bit (depending on who you ask; some people love this style, some hate it; I'm just pointing out it's an option):
(defn fib [n]
(->> [0 1]
(iterate (fn [[a b]] [b (+ a b)]))
(map first)
(take n)))
That said, I'd probably extract the (take n) and just have the fib function be a lazy infinite sequence.
(def fib
(->> [0 1]
(iterate (fn [[a b]] [b (+ a b)]))
(map first)))
;;usage
(take 10 fib)
;;output (0 1 1 2 3 5 8 13 21 34)
(nth fib 9)
;; output 34
A good recursive definition is:
(def fib
(memoize
(fn [x]
(if (< x 2) 1
(+ (fib (dec (dec x))) (fib (dec x)))))))
This will return a specific term. Expanding this to return first n terms is trivial:
(take n (map fib (iterate inc 0)))
Here is the shortest recursive function I've come up with for computing the nth Fibonacci number:
(defn fib-nth [n] (if (< n 2)
n
(+ (fib-nth (- n 1)) (fib-nth (- n 2)))))
However, the solution with loop/recursion should be faster for all but the first few values of 'n' since Clojure does tail-end optimization on loop/recur.
this is my approach
(defn fibonacci-seq [n]
(cond
(= n 0) 0
(= n 1) 1
:else (+ (fibonacci-seq (- n 1)) (fibonacci-seq (- n 2)))
)
)
For latecomers. Accepted answer is a slightly complicated expression of this:
(defn fib
([n]
(fib [0 1] n))
([x, n]
(if (< (count x) n)
(recur (conj x (apply + (take-last 2 x))) n)
x)))
For what it's worth, lo these years hence, here's my solution to 4Closure Problem #26: Fibonacci Sequence
(fn [x]
(loop [i '(1 1)]
(if (= x (count i))
(reverse i)
(recur
(conj i (apply + (take 2 i)))))))
I don't, by any means, think this is the optimal or most idiomatic approach. The whole reason I'm going through the exercises at 4Clojure ... and mulling over code examples from Rosetta Code is to learn clojure.
Incidentally I'm well aware that the Fibonacci sequence formally includes 0 ... that this example should loop [i '(1 0)] ... but that wouldn't match their spec. nor pass their unit tests despite how they've labelled this exercise. It is written as an anonymous recursive function in order to conform to the requirements for the 4Clojure exercises ... where you have to "fill in the blank" within a given expression. (I'm finding the whole notion of anonymous recursion to be a bit of a mind bender; I get that the (loop ... (recur ... special form is constrained to tail-recursion ... but it's still a weird syntax to me).
I'll take #[Arthur Ulfeldt]'s comment, regarding fib3 in the original posting, under consideration as well. I've only used Clojure's iterate once, so far.
Say I have a predicate that takes two items and returns true of false. I want to insert an item in between every consecutive pair in a sequence that returns true for the predicate. I've come up with a couple of solutions but I was wondering what would be idiomatic way to do it functionally in Clojure?
My first draft would be something like
(defn insert-between [pred inter coll]
(lazy-seq
(when-let [s (seq coll)]
(cons (first s)
(mapcat (fn [[left right]]
(if (pred left right)
[inter right]
[right]))
(partition 2 1 s))))))
user> (insert-between < :less [1 6 7 4 3])
(1 :less 6 :less 7 4 3)
Seems to work, but I'm special-casing the first element in an ugly way and I think you could get around that. The solution could definitely be improved, anyway.
This is my try at it:
(defn interpose-predicated [pred in coll]
(if (next coll)
(->> coll
(partition 2 1)
(mapcat (comp next #(if (apply pred %) (interpose in %) %)))
(cons (first coll)))
coll))
It's the usual SO [clojure] race to come up with the most concise solution. :-) I usually don't win, but I learn a lot in the process. Anyway, here is my solution:
(defn interpose-p [[a b & _ :as s] d p]
(when-not (empty? s)
(if (and (not (nil? b)) (p a b))
(cons a (cons d (interpose-p (rest s) d p)))
(cons a (interpose-p (rest s) d p)))))
(interpose-p [1 2 3 2 1 2 3] "," <)
(1 "," 2 "," 3 2 1 "," 2 "," 3)
Update: Even though the discussion is over, here is an updated solution taking into account everyone's comments. This time is should be fairly industrial strength assuming my understanding of lazy-seq is correct. It is templated off of the lazy-seq discussion here.
(defn interpose-p
[pred coll sep]
(let [f (fn [c]
(when-let [[a b & _ :as s] (seq c)]
(if (and b (pred a b))
(list* a sep (interpose-p pred (rest s) sep))
(list* a (interpose-p pred (rest s) sep)))))]
(lazy-seq (f coll))))
Say I have a vector:
(def data ["Hello" "World" "Test" "This"])
And I want to populate a table somewhere that has an api:
(defn setCell
[row col value]
(some code here))
Then what is the best way to get the following calls to happen:
(setCell 0 0 "Hello")
(setCell 0 1 "World")
(setCell 0 2 "Test")
(setCell 0 3 "This")
I found that the following will work:
(let [idv (map vector (iterate inc 0) data)]
(doseq [[index value] idv] (setCell 0 index value)))
But is there a faster way that does not require a new temporary datastructure idv?
You can get the same effect in a very clojure-idiomatic way by just mapping the indexes along with the data.
(map #(setCell 0 %1 %2) (iterate inc 0) data)
You may want to wrap this in a (doall or (doseq to make the calls happen now. It's just fine to map an infinite seq along with the finite one because map will stop when the shortest seq runs out.
A bit late in the game but for people accessing this page: there is now (since clojure 1.2) a map-indexed function available in clojure.core.
One issue (unless I'm mistaken): there's no "pmap" equivalent, meaning that map-indexed computations cannot easily be parallelized. In that case, I'd refer to solutions offered above.
The way you're doing it is idiomatic (and identical to clojure.contrib.seq-utils/indexed in fact). If you really want to avoid the extra data structure, you can do this:
(loop [data data, index 0]
(when (seq data)
(setCell 0 index (first data))
(recur (rest data) (inc index))))
I'd use your version unless there was a good reason not to though.
The nicest way would be to use clojure.contrib.seq-utils/indexed, which will look like this (using destructuring):
(doseq [[idx val] (indexed ["Hello" "World" "Test" "This"])]
(setCell 0 idx val))
I did a short comparison of the performance of the options sofar:
; just some function that sums stuff
(defn testThis
[i value]
(def total (+ total i value)))
; our test dataset. Make it non-lazy with doall
(def testD (doall (range 100000)))
; time using Arthur's suggestion
(def total 0.0)
(time (doall (map #(testThis %1 %2) (iterate inc 0) testD)))
(println "Total: " total)
; time using Brian's recursive version
(def total 0.0)
(time (loop [d testD i 0]
(when (seq d)
(testThis i (first d))
(recur (rest d) (inc i)))))
(println "Total: " total)
; with the idiomatic indexed version
(def total 0.0)
(time (let [idv (map vector (iterate inc 0) testD)]
(doseq [[i value] idv] (testThis i value))))
(println "Total: " total)
Results on my 1 core laptop:
"Elapsed time: 598.224635 msecs"
Total: 9.9999E9
"Elapsed time: 241.573161 msecs"
Total: 9.9999E9
"Elapsed time: 959.050662 msecs"
Total: 9.9999E9
Preliminary Conclusion:
Use the loop/recur solution.