Background
I'm working through Ullmans Elements of ML programming in my spare-time. End goal is to self-study Andrew Appels Modern Compiler Implementation in ML.
In Elements of ML, Ullman describes the difference list:
There is a trick known to LISP programmers as difference lists, in which one
manipulates lists more efficiently by keeping, as an extra parameter of your
function, a list that represents in some way what you have already accomplished.
The idea comes up in a number of different applications;
Ullman uses reverse as an example of the difference list technique. Here is a slow function that runs in O(n^2).
fun reverse nil = nil
| reverse (x::xs) = reverse(xs) # [x]
And the faster one using a difference list
fun rev1(nil, M) = M
| rev1(x::xs, ys) = rev1(xs, x::ys)
fun reverse L = rev1(L, nil)
My problem
I have this Binary Search Tree (BST) data type.
datatype 'a btree = Empty
| Node of 'a * 'a btree * 'a btree
A naive solution for collecting a list of the elements in pre-order would be
fun preOrder Empty = nil
| preOrder (Node(x, left, right)) = [x] # preOrder left # preOrder right
But Ullman points out that the # operator is slow and suggests in exercise 6.3.5 that I implement preOrder using a difference list.
After some head scratching I came up with this function:
fun preOrder tree = let
fun pre (Empty, L) = L
| pre (Node(x, left, right), L) = let
val L = pre(right, L)
val L = pre(left, L)
in
x::L
end
in
pre (tree, nil)
end
It outputs the elements in pre-order. BUT it evaluates the tree in post-order! And the code is uglier than the naive preOrder one.
> val t = Node(5,
Node(3,
Node(1, Empty, Empty),
Node(4, Empty, Empty)),
Node(9, Empty, Empty))
> preOrder t
val it = [5,3,1,4,9] : int list
Prior Art
I tried searching for references to difference lists in ML programming, and found John Hughes original article describing how to use difference lists for reverse.
I also found Matthew Brecknells difference list blog post with examples in Haskell. He makes a distinction between using an accumulator, like Ullmans reverse example and creating a new type for difference lists. He also presents a tree flattener. But I have a hard time understanding the Haskell code and would appreciate a similar expose but in Standard ML.
abc
Question
How implement a function that actually evaluate the tree in pre-order and collects the elements in pre-order? Do I have to reverse the list after my traversal? Or is there some other trick?
How can I generalize this technique to work for in-order and post-order traversal?
What is the idiomatic way for using a difference list for a BST algorithm?
Your eventual method of doing this is is the best it reasonably gets. The nice way to do this turns out to be
fun preOrderHelper (Empty, lst) = lst
| preOrderHelper (Node(x, left, right), lst) =
x :: preOrderHelper(left, preOrderHelper(right, lst))
fun preOrder tree = preOrderHelper(tree, Nil)
Note that the run time of preOrderHelper(tree, list) is only a function of tree. Call r(t) the run time of preOrderHelper on tree t. Then we have r(Empty) = O(1) and r(Node(x, left, right)) = O(1) + r(left) + r(right), so clearly r(t) is linear in the size of t.
What is the derivation of this technique? Is there a more principled way of deriving it? In general, when you're turning a data structure into a list, you want to foldr onto an empty list. I don't know enough ML to say what the equivalent of typeclasses is, but in Haskell, we would approach the situation as follows:
data Tree a = Empty | Node a (Tree a) (Tree a)
instance Foldable Tree where
foldr f acc t = foldrF t acc where
foldrF Empty acc = acc
foldrF (Node x left right) acc = f x (foldrF left (foldrF right acc))
To convert a Tree a to a [a], we would call Data.Foldable.toList, which is defined in Data.Foldable as
toList :: Foldable f => f a -> [a]
toList = foldr (:) []
Unfolding this definition gives us the equivalent of the ML definition above.
As you can see, your technique is actually a special case of a very principled way to turn data structures into lists.
In fact, in modern Haskell, we can do this totally automatically.
{-# LANGUAGE DeriveFoldable #-}
data Tree a = Empty | Node a (Tree a) (Tree a) deriving Foldable
will give us the equivalent(*) of the above Foldable implementation automatically, and we can then immediately use toList. I don't know what the equivalent is in ML, but I'm sure there's something analogous.
The difference between ML and Haskell is that Haskell is lazy. Haskell's laziness means that the evaluation of preOrder actually walks the tree in the pre-Order order. This is one of the reasons I prefer laziness. Laziness permits very fine-grained control over the order of evaluation without resorting to non-functional techniques.
(*) (up to the arguments order, which does not count in the lazy Haskell.)
What you show is not what I've seen usually referred to as difference list.
That would be, in pseudocode,
-- xs is a prefix of an eventual list xs # ys,
-- a difference between the eventual list and its suffix ys:
dl xs = (ys => xs # ys)
and then
pre Empty = (ys => ys) -- Empty contributes an empty prefix
pre (Node(x, left, right)) = (ys =>
-- [x] # pre left # pre right # ys -- this pre returns lists
(dl [x] . pre left . pre right) ys) -- this pre returns diff-lists
-- Node contributes an [x], then goes
-- prefix from `left`, then from `right`
so that
preOrder tree = pre tree []
where . is the functional composition operator,
(f . g) = (x => f (g x))
Of course since dl [x] = (ys => [x] # ys) = (ys => x::ys) this is equivalent to what you show, in the form of
--pre Empty = (ys => ys) -- Empty's resulting prefix is empty
pre' Empty ys = ys
--pre (Node(x, left, right)) = (ys =>
pre' (Node(x, left, right)) ys =
-- [x] # pre left # pre right # ys
-- (dl [x] . pre left . pre right) ys
x::( pre' left ( pre' right ys))
-- preOrder tree = pre' tree []
Operationally, this will traverse the tree right-to-left in an eager language, and left-to-right in a lazy one.
Conceptually, seen left-to-right, the resulting list has [x] and then the result of traversing left and then the result of traversing right, no matter what was the tree traversal order.
These difference lists are just partially applied # operators, and appending is just functional composition:
dl (xs # ys) == (dl xs . dl ys)
-- or:
dl (xs # ys) zs == (dl xs . dl ys) zs
== dl xs ( dl ys zs)
== xs # (ys # zs)
the prefix xs # ys is the prefix xs, followed by the prefix ys, followed by whatever the eventual suffix zs will be.
Thus appending these difference lists is an O(1) operation, the creation of a new lambda function which is a composition of the arguments:
append dl1 dl2 = (zs => dl1 ( dl2 zs))
= (zs => (dl1 . dl2) zs )
= (dl1 . dl2)
Now we can easily see how to code the in-order or post-order traversals, as
in_ Empty = (ys => ys)
in_ (Node(x, left, right)) = (ys =>
-- in_ left # [x] # in_ right # ys
(in_ left . dl [x] . in_ right) ys)
post Empty = (ys => ys)
post (Node(x, left, right)) = (ys =>
-- post left # post right # [x] # ys
(post left . post right . dl [x]) ys)
Focusing on just lists [x] and their appending # lets us treat this uniformly -- no need to concern ourselves with :: and its arguments which have different types.
The types of both arguments of # are the same, just as they are for + with integers and indeed . with functions. Such types paired with such operations are known as monoids, under the condition that the appending operation is associative, (a+b)+c == a+(b+c), and there is an "empty" element, e # s == s # e == s. This just means that the combination operation is "structural" in some way. This works with apples and oranges, but atomic nuclei -- not so much.
Related
I'm trying to self-learn some programming in a functional programming language and recently stumbled on the problem of generating all the permutations of length m from a list of length n, with repetition. Mathematically, this should result in a total of n^m possible permutations, because each of the m 'slots' can be filled with any of the n elements. The code I have currently, however, does not give me all the elements:
let rec permuts n list =
match n, list with
0, _ -> [[]]
| _, [] -> []
| n, h :: t -> (List.map (fun tt -> h::tt) (permuts (n-1) list))
# permuts n t;;
The algorithm basically takes one element out of a list with m elements, slaps it onto the front of all the combinations with the rest of the elements, and concatenates the results into one list, giving only n C m results.
For example, the output for permuts 2 [1;2;3] yields
[[1;1]; [1;2]; [1;3]; [2;2]; [2;3]; [3;3]]
whereas I actually want
[[1;1]; [1;2]; [1;3]; [2;1]; [2;2]; [2;3]; [3;1]; [3;2]; [3;3]]
-- a total of 9 elements. How do I fix my code so that I get the result I need? Any guidance is appreciated.
Your error appears on the second line of:
| n, h :: t -> List.map (fun tt -> h::tt) (permuts (n-1) list)
# permuts n t
Indeed, with this you are decomposing the set of n-tuples with k elements as the sum of
the set of (n-1)-tuples prefixed with the first element
the set of n-tuples with (k-1) elements
Looking at the cardinal of the three sets, there is an obvious mismatch since
k^n ≠ k^(n-1) + (k-1)^n
And the problem is that the second term doesn't fit.
To avoid this issue, it is probably better to write a couple of helper function.
I would suggest to write the following three helper functions:
val distribute: 'a list -> 'a list -> 'a list list
(** distribute [x_1;...;x_n] y returns [x_1::y;...x_n::y] *)
val distribute_on_all: 'a list -> 'a list list
(** distribute_on_all x [l_1;...;l_n] returns distribute x l_1 # ... # distribute x l_n *)
val repeat: int -> ('a -> 'a) -> 'a -> 'a
(** repeat n f x is f(...(f x)...) with f applied n times *)
then your function will be simply
let power n l = repeat n (distribute_on_all l) [[]]
In Haskell, it's very natural to do this using a list comprehension:
samples :: Int -> [a] -> [[a]]
samples 0 _ = [[]]
samples n xs =
[ p : ps
| p <- xs
, ps <- samples (n - 1) xs
]
It seems to me you never want to recurse on the tail of the list, since all your selections are from the whole list.
The Haskell code of #dfeuer looks right. Note that it never deconstructs the list xs. It just recurses on n.
You should be able to copy the Haskell code using List.map in place of the first two lines of the list comprehension, and a recursive call with (n - 1) in place of the next line.
Here's how I would write it in OCaml:
let perm src =
let rec extend remaining_count tails =
match remaining_count with
| 0 -> tails
| _ ->
(* Put an element 'src_elt' taken from all the possible elements 'src'
in front of each possible tail 'tail' taken from 'tails',
resulting in 'new_tails'. The elements of 'new_tails' are one
item longer than the elements of 'tails'. *)
let new_tails =
List.fold_left (fun new_tails src_elt ->
List.fold_left (fun new_tails tail ->
(src_elt :: tail) :: new_tails
) new_tails tails
) [] src
in
extend (remaining_count - 1) new_tails
in
extend (List.length src) [[]]
The List.fold_left calls may look a bit intimidating but they work well. So it's a good idea to practice using List.fold_left. Similarly, Hashtbl.fold is also common and idiomatic, and you'd use it to collect the keys and values of a hash table.
I'm trying to implement merge function in OCaml using Tail recursion but I face awkward results. Could anyone help me out with this. Thanks in advance.
let rec merge_helper l1 l2 accum =
match l1 with
[] -> l2#accum
| hd::tl -> (match l2 with
[] -> l1#accum
|x::xs -> merge_helper tl xs l1#l2#accum);;
let merge l1 l2 = merge_helper l1 l2 [];;
merge [1;2;4] [3;4;5];;
- : int list = [4; 5; 2; 4; 4; 5; 1; 2; 4; 3; 4; 5]
First of all your implementation doesn't run in a constant stack space. The xs # ys operation is not tail-recursive, and will make List.length xs calls (thus using this amount of stack frames). Also, the merge function usually preserves the ordering. So you need to have a comparison function, that will compare elements of the list. It is not absolutely clear, what you're expecting from you merge function, and why you classify your result as weird. For me the result matches with the code. What looks very strange to me is that although you're deconstructing l1 and l2 you're not using the result of the deconstruction and adds the whole lists l1 and l2 to the accumulator.
The approach should be the following, take an element from the first list, add this element to the accumulator, and switch the lists. So the induction step of the algorithm is the following:
let rec loop acc xs ys = match xs with
...
| x::xs -> loop (x::acc) ys xs
But if you want to merge two sorted lists preserving the order, then you need to take the smallest element of the two lists at each step.
let merge cmp xs ys =
let rec loop xs ys zs = match xs,ys with
| [],ss|ss,[] -> List.rev_append zs ss
| x :: txs, y :: tys ->
if cmp x y <= 0
then loop txs ys (x::zs)
else loop xs tys (y::zs) in
loop xs ys []
Here in the induction step, we take the smaller element, and continue with the two lists: the tail of the owner of the smaller element (because it is moved into the accumulator), and the second list is taken fully (because nothing is accumulated from it). Since we're prepending elements, they will be in a reversed order, so we will need to something to reverse the result (a usual trade off for tail-recursion). The base case, allows us to short-circuit our algorithm, when one or another list is shorter, and we don't need any more to compare them one-by-one, and can just append the rest part of the longer list to the accumulator zs. We use List.rev_append to append the leftover tail of the list to our accumulator. This function will prepend the reversed version of the first list to the second.
Since I had problems with stack overflows due to a non-tail recursion, I used the continuations so as to make sorting of large list feasible.
I implemeneted the sorting this way (you can see the whole code here: http://paste.ubuntu.com/13481004/)
let merge_sort l =
let rec merge_sort' l cont =
match l with
| [] -> cont []
| [x] -> cont [x]
| _ ->
let (a,b) = split l
in
merge_sort' a
(fun leftRes -> merge_sort' b
(* OVERFLOW HERE *) (fun rightRes -> cont (merge leftRes rightRes) )
)
in merge_sort' l (fun x -> x)
I get a stack overflow nevertheless, in the indicated line.
What am I doing wrong?
(#) of OCaml's standard library is not tail recursive. merge function in your code http://paste.ubuntu.com/13481004/ uses (#), and this is the cause of the stack overflow.
list.mli says:
val append : 'a list -> 'a list -> 'a list
(** Catenate two lists. Same function as the infix operator [#].
Not tail-recursive (length of the first argument). The [#]
operator is not tail-recursive either. *)
but unfortunately this fact is not written in pervasives.mli where (#) is really declared:
val ( # ) : 'a list -> 'a list -> 'a list
(** List concatenation. *)
This is not good :-( I have filed an issue for it at OCaml dev page.
I redefined (#) as fun x y -> rev_append (rev x) y then your code runs w/o stack overflow. More elegantly, you can replace codes like (rev a) # l by rev_append a l.
P.S. (#) in pervasives.mli will be commented as "not tail recursive" in the next release of OCaml.
This is related to Project Euler problem 523.
The following algorithm sorts a list:
Starting from the beginning of the list, check each pair of adjacent elements in turn.
If the elements are out of order:
Move the smallest element of the pair at the beginning of the list.
Restart the process from step 1.
If all pairs are in order, stop.
I am attempting (for practice) to code this up in Isabelle. I have obtained the following function definitions:
fun firstpos :: "nat list ⇒ nat"
where "firstpos [] = 0"
| "firstpos (x # y # xs) = (if (x > y) then 1 else (firstpos (y#xs))+1)"
| "firstpos [x] = Suc 0"
and
fun insertAtFront :: "(nat list) ⇒ (nat) ⇒ (nat list)"
where "insertAtFront l 0 = l"
| "insertAtFront l i = [hd (rev (take i l))] # (take (i-1) l) # (rev (take ((length l) - i) (rev l)))"
where firstpos determines the list-index of the first element which is greater than its successor in the list, and insertAtFront permutes the list so that the "second argument"th element is at the front.
That is, firstPos effectively tells us where to start doing Step 2 of the algorithm, and insertAtFront is step 2.1 of the algorithm.
Now, I wish to define the function which is basically the fixed point of the composition of these. In other words,
fun sortproc :: "nat list ⇒ nat list"
where "sortproc l = sortproc (insertAtFront l (firstpos l + 1))"
It is nontrivial that this procedure ever terminates, but I can prove it mathematically: at each stage, $\sum_{i=1}^n 2^i L_{n-i}$ decreases, where $n$ is the length of list $L$.
I am a novice in Isabelle. How can I code the idea "this definition is well-defined, because of this proof" in Isabelle? The documentation seems to talk mainly about size arguments, and the size of the input isn't changing: it's always a list of the same length.
I'm trying to find the largest value of a list using tail recursion. I can't use any auxiliary functions, though...so it must be done using recursion. I've written a function to find the max, starting from the head, but don't know how to implement it starting from the tail!
lmax [] = error "empty list"
lmax [x] = x
lmax (x::y::xs) =
if x > y then lmax (x::xs)
else lmax (y::xs)
The term "tail recursion" has nothing to do with the tail of a list, it is about the position of a function call.
You could say that a function call is in tail position, or that it is a tail call, if it's the last thing to happen in a function, i.e. no other computations depend on it.
Compare
fun foo xs = List.length xs
and
fun bar xs = 1 + List.length xs
In the first, the call to List.length is in tail position, because its result is returned immediately.
In the second, since we add 1 to the length, the call isn't a tail call.
"Tail recursion" is when a recursive function call is a tail call.
So you're in luck: your function already is tail recursive, since both conditional branches just return the value of a recursive call.
fun lmax l = let
fun lmaxh [] a = a
| lmaxh (x::xs) a = lmax xs Int.max(x,a)
in
lmaxh l 0
end
This works, assuming that the values are nonnegative integers.
Implementing tail recursion optimizes efficiency, because one doesn't have to evaluate and "pop-off" the stack after creating the recursive calls.
In general, to use tail-recursion, you must store some "memory" from prior computations to compare with in the current one, and update it for future computations, so as to immediately exit the function in the base case.
As such, your function is already tail recursive.
However, here is a tail-recursive maxList function, more in the spirit of SML :
fun maxList l =
let
fun maxListHelper l acc =
case l of
[] => acc
| x :: xs' => if x > acc
then (maxListHelper xs' x)
else (maxListHelper xs' acc)
in
case l of
[] => error "Empty List!"
| x :: xs' => maxListHelper xs' x
end
Your function is written in a very Haskell-like syntax with different cases handled on different lines without being explicitly declared as nested cases inside a function definition. This is quite alright, but is usually not done in SML.