Learning clojure, trying to create a lazy infinite sequence of all prime numbers.
I'm aware that there are more efficient algorithms; I'm doing the following more as a POC/lesson than as the ideal solution.
I have a function that, given a sequence of primes, tells me what the next prime is:
(next-prime [2 3 5]) ; result: 7
My lazy sequence must therefore pass itself to this function, then take the result and add that to itself.
My first attempt:
(def lazy-primes
(lazy-cat [2 3 5] (find-next-prime lazy-primes)))
..which results in an IllegalArgumentException: Don't know how to create ISeq from: java.lang.Integer
My second attempt:
(def lazy-primes
(lazy-cat [2 3 5] [(find-next-prime lazy-primes)]))
..which gives me [2 3 5 7] when asked for 10 elements.
Attempt 3:
(def lazy-primes
(lazy-cat [2 3 5]
(conj lazy-primes (find-next-prime lazy-primes))))
(take 10 lazy-primes) ; result: (2 3 5 7 2 3 5 7 2 3)
All of these seem like they should work (or at least, should work given that the preceding didn't work). Why am I getting the bogus output for each case?
Reasons why your initial attempts don't work:
(find-next-prime lazy-primes) returns an integer but lazy-cat needs a sequence
[(find-next-prime lazy-primes)] creates a vector (and is hence seqable) but it only gets evaluated once when it is first accessed
conj is adding new primes to the start of the sequence (since lazy-cat and hence lazy-primes returns a sequence)... which is probably not what you want! It's also possibly confusing find-next-prime depending on how that is implemented, and there might be a few subtle issues around chunked sequences as well.....
You might instead want to use something like:
(defn seq-fn [builder-fn num ss]
(cons
(builder-fn (take num ss))
(lazy-seq (seq-fn builder-fn (inc num) ss))))
(def lazy-primes
(lazy-cat [2 3 5] (seq-fn next-prime 3 lazy-primes)))
A bit complicated, but basically what I'm doing is using the higher-order helper function to provide a closure over a set of parameters that includes the number of primes created so far, so that it can generate the next prime incrementally at each step.
p.s. as I'm sure you are aware there are faster algorithms for generating primes! I'm assuming that this is intended primarily as an exercise in Clojure and the use of lazy sequences, in which case all well and good! But if you really care about generating lots of primes I'd recommend taking a look at the Sieve of Atkin
Alternatively, you could use iterate: the built-in function that lazily takes the output of a function and applies that to the function again
clojure.core/iterate
([f x])
Returns a lazy sequence of x, (f x), (f (f x)) etc.
f must be free of side-effects
in order for you to make it work, the next-prime function should concatenate its result to its input, and return the concatenation.
Then you can just call (take 100 (iterate list-primes [1])) to get a list of the first 100
primes.
With your next-prime function you can generate a lazy sequence of all primes with the following snippet of code:
(def primes (map peek (iterate #(conj % (next-prime %)) [2])))
The combination you are looking for is concat + lazy-seq + local fn.
Take a look at the implementation of Erathostenes' Sieve in the Clojure Contrib libraries: https://github.com/richhickey/clojure-contrib/blob/78ee9b3e64c5ac6082fb223fc79292175e8e4f0c/src/main/clojure/clojure/contrib/lazy_seqs.clj#L66
One more word, though: this implementation uses a more sophisticated algorithm for the Sieve in a functional language.
Another implementation for Clojure can be found in Rosetta code. However, I don't like that one as it uses atoms, which you don't need for the solution of this algo in Clojure.
Related
I am experimenting with clojure's lazy sequences. In order to see when the evaluation of an item would occur, I created a function called square that prints the result before returning it. I then apply this function to a vector using map.
(defn square [x]
(let [result (* x x)]
(println "printing " result)
result))
(def s (map square [1 2 3 4 5])) ; outputs nothing
Here in my declaration of s, the REPL does not output anything. This signals that the computation has not started yet. This appears to be correct. I then do:
(first s)
The function "first" takes only the first item. So I expect that only 1 will be evaluated. My expectation is the REPL will output the following:
printing 1
1
However, the REPL outputted the following instead.
printing 1
printing 4
printing 9
printing 16
printing 25
1
So rather than evaluating only the first item, it seems it evaluates all items, even though I am accessing just the first item.
If the state of a lazy sequence can only be either all values computed and no values computed, then how can it gain the advantages of lazy evaluation? I came from a scheme background and I was expecting more like the behavior of streams. Looks like I am mistaken. Can anyone explain what is going on?
Laziness isn't all-or-nothing, but some implementations of seq operate on 'chunks' of the input sequence (see here for an explanation). This is the case for vector which you can test for with chunked-seq?:
(chunked-seq? (seq [1 2 3 4 5]))
When given a collection map checks to see if the underlying sequence is chunked, and if so evaluates the result on a per-chunk basis instead of an element at a time.
The chunk size is usually 32 so you can see this behaviour by comparing the result of
(first (map square (vec (range 35))))
This should only display a message for the first 32 items and not the entire sequence.
When would I use vector-of to create a vector, instead of the vector function. Is the guideline to use vector most of the time and only for performance reason switch to vector-of?
I could not find good info on when to use vector-of.
vector-of is used for creating a vector of a single primitive type, :int, :long, :float, :double, :byte, :short, :char, or :boolean. It doesn't allow other types as it stores the values unboxed internally. So, if your vector need to include other types than those primitive types, you cannot use vector-of. But if you are sure that the vector will have data of a single primitive type, you can use vector-of for better performance.
user=> (vector-of :int 1 2 3 4 5)
[1 2 3 4 5]
user=> (vector-of :double 1.0 2.0)
[1.0 2.0]
user=> (vector-of :string "hello" "world")
Execution error (IllegalArgumentException) at user/eval5 (REPL:1).
Unrecognized type :string
As you can see, you should specify primitive type as an argument.
vector can be used to create a vector of any type.
user=> (vector 1 2.0 "hello")
[1 2.0 "hello"]
You can put any type when you use vector.
Also, there's another function vec, which is used for creating a new vector containing the contents of coll.
user=> (vec '(1 2 3 4 5))
[1 2 3 4 5]
Usually, you can get the basic information of a function/macro from the repl, like the following.
user=> (doc vector-of)
-------------------------
clojure.core/vector-of
([t] [t & elements])
Creates a new vector of a single primitive type t, where t is one
of :int :long :float :double :byte :short :char or :boolean. The
resulting vector complies with the interface of vectors in general,
but stores the values unboxed internally.
Optionally takes one or more elements to populate the vector.
Reference:
https://clojuredocs.org/clojure.core/vector-of
https://clojuredocs.org/clojure.core/vector
https://clojuredocs.org/clojure.core/vec
Nobody really ever uses vector-of. If you don't super care about performance, vector is fine, and if you do super care about performance you usually want a primitive array or some other java type. Honestly I would expect occasional weird snags when passing a vector-of to anything that expects an ordinary vector or sequence - maybe it works fine, but it's just such a rare thing to see that it wouldn't surprise me if it caused issues.
In Clojure we see the following examples of using map:
(map inc [1 2 3 4 5])
;;=> (2 3 4 5 6)
We observe that map takes a function and applies it to each element in a collection.
In Clojure we see the following example of using juxt:
((juxt inc dec) 2)
;=>[3 1]
We observe that juxt takes a data structure and applies each function to it.
My question is: Is juxt the opposite of map?
(In that you map a data structure over a collection of functions instead of the other way around)
I don't think you can call juxt the opposite of map. Juxt composes a number of functions into one function that produces a sequence of the result of applying each of the functions to its argument(s).
I guess juxt could be (should be?) expressed using a map
(defn juxt [& funs] (fn [& args] (map #(apply % args)) funs))
If I understand correctly Clojure can return lists (as in other Lisps) but also vectors and sets.
What I don't really get is why there's not always a collection that is returned.
For example if I take the following code:
(loop [x 128]
(when (> x 1)
(println x)
(recur (/ x 2))))
It does print 128 64 32 16 8 4 2. But that's only because println is called and println has the side-effect (?) of printing something.
So I tried replacing it with this (removing the println):
(loop [x 128]
(when (> x 1)
x
(recur (/ x 2))))
And I was expecting to get some collecting (supposedly a list), like this:
(128 64 32 16 8 4 2)
but instead I'm getting nil.
I don't understand which determines what creates a collection and what doesn't and how you switch from one to the other. Also, seen that Clojure somehow encourages a "functional" way of programming, aren't you supposed to nearly always return collections?
Why are so many functions that apparently do not return any collection? And what would be an idiomatic way to make these return collections?
For example, how would I solve the above problem by first constructing a collection and then iterating (?) in an idiomatic way other the resulting list/vector?
First I don't know how to transform the loop so that it produces something else than nil and then I tried the following:
(reduce println '(1 2 3))
But it prints "1 2nil 3nil" instead of the "1 2 3nil" I was expecting.
I realize this is basic stuff but I'm just starting and I'm obviously missing basic stuff here.
(P.S.: retag appropriately, I don't know which terms I should use here)
A few other comments have pointed out that when doesn't really work like if - but I don't think that's really your question.
The loop and recur forms create an iteration - like a for loop in other languages. In this case, when you are printing, it is indeed just for the side effects. If you want to return a sequence, then you'll need to build one:
(loop [x 128
acc []]
(if (< x 1)
acc
(recur (/ x 2)
(cons x acc))))
=> (1 2 4 8 16 32 64 128)
In this case, I replaced the spot where you were calling printf with a recur and a form that adds x to the front of that accumulator. In the case that x is less than 1, the code returns the accumulator - and thus a sequence. If you want to add to the end of the vector instead of the front, change it to conj:
(loop [x 128
acc []]
(if (< x 1)
acc
(recur (/ x 2)
(conj acc x))))
=> [128 64 32 16 8 4 2 1]
You were getting nil because that was the result of your expression -- what the final println returned.
Does all this make sense?
reduce is not quite the same thing -- it is used to reduce a list by repeatedly applying a binary function (a function that takes 2 arguments) to either an initial value and the first element of a sequence, or the first two elements of the sequence for the first iteration, then subsequent iterations are passed the result of the previous iteration and the next value from the sequence. Some examples may help:
(reduce + [1 2 3 4])
10
This executes the following:
(+ 1 2) => 3
(+ 3 3) => 6
(+ 6 4) => 10
Reduce will result in whatever the final result is from the binary function being executed -- in this case we're reducing the numbers in the sequence into the sum of all the elements.
You can also supply an initial value:
(reduce + 5 [1 2 3 4])
15
Which executes the following:
(+ 5 1) => 6
(+ 6 2) => 8
(+ 8 3) => 11
(+ 11 4) => 15
HTH,
Kyle
The generalized abstraction over collection is called a sequence in Clojure and many data structure implement this abstraction so that you can use all sequence related operations on those data structure without thinking about which data structure is being passed to your function(s).
As far as the sample code is concerned - the loop, recur is for recursion - so basically any problem that you want to solve using recursion can be solved using it, classic example being factorial. Although you can create a vector/list using loop - by using the accumulator as a vector and keep appending items to it and in the exist condition of recursion returning the accumulated vector - but you can use reductions and take-while functions to do so as shown below. This will return a lazy sequence.
Ex:
(take-while #(> % 1) (reductions (fn [s _] (/ s 2)) 128 (range)))
Whilst learning Clojure, I've spent ages trying to make sense of monads - what they are and how we can use them.... with not too much success. However, I found an excellent 'Monads for Dummies' Video Series - http://vimeo.com/20717301 - by Brian Marik for Clojure
So far, my understanding of monads is that it is sort of like a macro in that it allows a set of statements to be written in a form that is easy to read - but monads are much more formalised. My observations are limited to two examples:
1. The Identity Monad (or the 'let' monad) taken from http://onclojure.com/2009/03/05/a-monad-tutorial-for-clojure-programmers-part-1/
The form that we wish to write is:
(let [a 1
b (inc a)]
(* a b))
and the corresponding monad is
(domonad identity-m
[a 1
b (inc a)]
(* a b))
2. The Sequence Monad (or the 'for' monad) taken from http://onclojure.com/2009/03/06/a-monad-tutorial-for-clojure-programmers-part-2/
The form we wish to write is:
(for [a (range 5)
b (range a)]
(* a b))
and the corresponding monad is
(domonad sequence-m
[a (range 5)
b (range a)]
(* a b))
Monad Definitions in Clojure
Looking at the source, using clojure monads library - https://github.com/clojure/algo.monads:
user=>(use 'clojure.algo.monads)
nil
indentity monad:
user=> (source identity-m)
(defmonad identity-m
[m-result identity
m-bind (fn m-result-id [mv f]
(f mv))
])
sequence monad:
user=> (source sequence-m)
(defmonad sequence-m
[m-result (fn m-result-sequence [v]
(list v))
m-bind (fn m-bind-sequence [mv f]
(flatten* (map f mv)))
m-zero (list)
m-plus (fn m-plus-sequence [& mvs]
(flatten* mvs))
])
So my conclusion is that a monad is some sort of a generalised higher-order function that takes in an input-function and input-values, adds its own control logic and spits out a 'thing' that can be used in a 'domonad' block.
Question 1
So finally, to the questions: I want to learn how to write a monad and say I want to write a 'map monad' that imitates the 'map' form in clojure:
(domonad map-m
[a [1 2 3 4 5]
b [5 6 7 8 9]]
(+ a b))
Should be equivalent to
(map + [1 2 3 4 5] [5 6 7 8 9])
and return the values
[6 8 10 12 14]
If I look at the source, it should give me something similar to identity-m and sequence-m:
user=> (source map-m)
(defmonad map-m
[m-result ...
m-bind ...
m-zero ...
m-plus ...
])
Question 2
I also want to be able to define 'reduce-m' such that I can write:
(domonad reduce-m
[a [1 2 3 4 5]]
(* a))
this could potentially give me 1 x 2 x 3 x 4 x 5 = 120 or
(domonad reduce-m
[a [1 2 3 4 5]
b [1 2 3 4 5]]
(+ a b))
will give me (1+2+3+4+5) + (1+2+3+4+5) = 30
Finally
Would I also be able to write a 'juxt monad' that imitates the juxt function but instead of passing in values for binding, I pass in a set of functions. :
(domonad juxt-m
[a #(+ % 1)
b #(* % 2)]
'([1 2 3 4 5] b a) )
gives
[ [2 2] [4 3] [6 4] [8 5] [9 6] ]
Potentially, I could do all of those things with macros so I don't really know how useful these 'monads' will be or if they are even considered 'monads'... With all the resources on the internet, It seems to me that if I wanted to learn Monads properly, I have to learn Haskell and right now, learning another syntactic form is just too hard. I think I found some links that maybe relevant but it is too cryptic for me
Please can someone shed some light!
Your examples are not monads. A monad represents composable computational steps. In the trivial identity monad, the computational step is just an expression evaluation.
In the maybe monad, a step is an expression that may succeed or fail.
In the sequence monad, a step is an expression that produces a variable number of results (the elements of the sequence).
In the writer monad, a computational step is a combination of expression evaluation and log output. In the state monad, a computational step involves accessing and/or modifying a piece of mutable state.
In all these cases, the monad plumbery takes care of correctly combining steps. The m-result function packages a "plain" value to fit into the monadic computation scheme, and the m-bind function feeds the result of one computational step into the next computational step.
In (map + a b), there are no computational steps to be combined. There is no notion of order. It's just nested expression evaluation. Same for reduce.
Your questions are not a type of monads. They seems more like a syntactic sugar and that you can accomplish using macros and I won't even suggest you do that because map, reduce etc are simple functions and there is no need to make their interface complex.
The reason these cases are not monads because moands are sort of amplified values which wrap normal values. In map and reduce case the vector that you use doesn't needs to be amplified to allow it to work in map or reduce.
It would be helpful for you to try macroexpand on the domoand expressions.
For ex: the sequence monad example should expand to something like:
(bind (result (range 5))
(fn [a]
(bind (result (range a))
(fn [b] (* a b))
)))
Where the bind and result are functions defined in the sequence monad m-bind and m-result.
So basically the vector expression(s) after the domand get nested and the expression(s) after the vector are used as it is in above case the (* a b) is called as it is (just that the a and b values are provided by the monad). In your example of map monad the vector expressions are supposed to be as it is and the last expression (+ a b) should somehow mean (map + a b) which is not what a monad is supposed to do.
I found some really good monads resources:
http://www.clojure.net/tags.html#monads-ref (Jim Duey's Monads Guide, which really goes into the nitty gritty monad definitions)
http://homepages.inf.ed.ac.uk/wadler/topics/monads.html#marktoberdorf (A whole load of papers on monads)
http://vimeo.com/17207564 (A talk on category theory, which I half followed)
So from Jim's Guide - http://www.clojure.net/2012/02/06/Legalities/ - he gives three axioms for definition of 'bind-m' and 'reduce-m' functions:
Identity
The first Monad Law can be written as
(m-bind (m-result x) f) is equal to (f x)
What this means is that whatever m-result does to x to make it into a monadic value, m-bind undoes before applying f to x. So with regards to m-bind, m-result is sort of like an identity function. Or in category theory terminology, its unit. Which is why sometimes you’ll see it named ‘unit’.
Reverse Identity The second Monad Law can be written as
(m-bind mv m-result) is equal to mv where mv is a monadic value.
This law is something like a complement to the first law. It basically ensures that m-result is a monadic function and that whatever m-bind does to a monadic value to extract a value, m-result undoes to create a monadic value.
Associativity
The third Monad Law can be written as
(m-bind (m-bind mv f) g) is equal to (m-bind mv (fn [x] (m-bind (f x) g)))
where f and g are monadic functions and mv is a monadic value.
What this law is saying is that it doesnt matter whether the f is applied to mv and then g is applied to the result, or whether a new monadic function is created out of a composition of f and g which is then applied to mv. In either order, the resulting value is the same monadic value. Stated another way, m-bind is left and right associative.
In http://www.clojure.net/2012/02/04/Sets-not-lists/ he gives an monad that takes sets as inputs instead of taking lists. Will work through all the examples...