I would like to create a function remove_duplicates that takes a list of any type (e.g. can be an int list or a bool list or a int list list or a whatever list) and returns the same list without duplicates, is this possible in Standard ML?
Is a function that takes a list of any type and returns the list without duplicates possible in Standard ML?
No.
To determine if one element is a duplicate of another, their values must be comparable. "Any type", or 'a in Standard ML, is not comparable for equality. So while you cannot have a val nub : 'a list -> 'a list that removes duplicates, here are four alternative options:
What #qouify suggests, the built-in equality type ''a, so anything you can use = on:
val nub : ''a list -> ''a list
What #kopecs suggests, a function that takes an equality operator as parameter:
val nub : ('a * 'a -> bool) -> 'a list -> 'a list
Which is a generalisation of 1., since here, nub op= : ''a list -> ''a list. This solution is kind of neat since it lets you remove not only duplicates, but also redundant representatives of arbitrary equivalence classes, e.g. nub (fn (x, y) => (x mod 3) = (y mod 3)) will only preserve integers that are distinct modulo 3. But its complexity is O(n²). (-_- )ﾉ⌒┻━┻
Because it is O(n²), nub is considered harmful.
As the article also suggests, the alternative is to use ordering rather than equality to reduce the complexity to O(n log n). While in Haskell this means only changing the type class constraint:
nub :: Eq a => [a] -> [a]
nubOrd :: Ord a => [a] -> [a]
and adjusting the algorithm, it gets a little more complicated to express this constraint in SML. While we do have ''a to represent Eq a => a (that we can use = on our input), we don't have a similar special syntax support for elements that can be compared as less/equal/greater, and we also don't have type classes. We do have the following built-in order type:
datatype order = LESS | EQUAL | GREATER
so if you like kopecs' solution, a variation with a better running time is:
val nubOrd : ('a * 'a -> order) -> 'a list -> 'a list
since it can use something like a mathematical set of previously seen elements, implemented using some kind of balanced search tree; n inserts each of complexity O(log n) takes a total of O(n log n) steps.
One of SML's winner features is its composable module system. Instead of using parametric polymorphism and feeding the function nubOrd with an order comparison function, you can create a module that takes another module as a parameter (a functor).
First, let's define a signature for modules that represent ordering of types:
signature ORD =
sig
type t
val compare : t * t -> order
end
(Notice that there isn't a ' in front of t.)
This means that anyone could make a struct ... end : ORD by specifying a t and a corresponding compare function for ts. Many built-in types have pre-defined compare functions: int has Int.compare and real has Real.compare.
Then, define a tree-based set data structure; I've used a binary search tree, and I've skipped most functions but the ones strictly necessary to perform this feat. Ideally you might extend the interface and use a better tree type, such as a self-balancing tree. (Unfortunately, since you've tagged this Q&A both as SML/NJ and Moscow ML, I wasn't sure which module to use, since they extend the standard library in different ways when it comes to balanced trees.)
functor TreeSet (X : ORD) =
struct
type t = X.t
datatype 'a tree = Leaf | Branch of 'a tree * 'a * 'a tree
val empty = Leaf
fun member (x, Leaf) = false
| member (x, Branch (left, y, right)) =
case X.compare (x, y) of
EQUAL => true
| LESS => member (x, left)
| GREATER => member (x, right)
fun insert (x, Leaf) = Branch (Leaf, x, Leaf)
| insert (x, Branch (left, y, right)) =
case X.compare (x, y) of
EQUAL => Branch (left, y, right)
| LESS => Branch (insert (x, left), y, right)
| GREATER => Branch (left, y, insert (x, right))
end
Lastly, the ListUtils functor contains the nubOrd utility function. The functor takes a structure X : ORD just like the TreeSet functor does. It creates an XSet structure by specialising the TreeSet functor using the same ordering module. It then uses this XSet to efficiently keep a record of the elements it has seen before.
functor ListUtils (X : ORD) =
struct
structure XSet = TreeSet(X)
fun nubOrd (xs : X.t list) =
let
val init = ([], XSet.empty)
fun go (x, (ys, seen)) =
if XSet.member (x, seen)
then (ys, seen)
else (x::ys, XSet.insert (x, seen))
in rev (#1 (foldl go init xs))
end
end
Using this functor to remove duplicates in an int list:
structure IntListUtils = ListUtils(struct
type t = int
val compare = Int.compare
end)
val example = IntListUtils.nubOrd [1,1,2,1,3,1,2,1,3,3,2,1,4,3,2,1,5,4,3,2,1]
(* [1, 2, 3, 4, 5] *)
The purpose of all that mess is a nubOrd without a direct extra function parameter.
Unfortunately, in order for this to extend to int list list, you need to create the compare function for that type, since unlike Int.compare, there isn't a generic one available in the standard library either. (This is where Haskell is a lot more ergonomic.)
So you might go and write a generic, lexicographical list compare function: If you know how to compare two elements of type 'a, you know how to compare two lists of those, no matter what the element type is:
fun listCompare _ ([], []) = EQUAL (* empty lists are equal *)
| listCompare _ ([], ys) = LESS (* empty is always smaller than non-empty *)
| listCompare _ (xs, []) = GREATER (* empty is always smaller than non-empty *)
| listCompare compare (x::xs, y::ys) =
case compare (x, y) of
EQUAL => listCompare compare (xs, ys)
| LESS => LESS
| GREATER => GREATER
And now,
structure IntListListUtils = ListUtils(struct
type t = int list
val compare = listCompare Int.compare
end)
val example2 = IntListListUtils.nubOrd [[1,2,3],[1,2,3,2],[1,2,3]]
(* [[1,2,3],[1,2,3,2]] *)
So even though [1,2,3] and [1,2,3,2] contain duplicates, they are not EQUAL when you compare them. But the third element is EQUAL to the first one, and so it gets removed as a duplicate.
Some last observations:
You may consider that even though each compare is only run O(log n) times, a single compare for some complex data structure, such as a (whatever * int) list list may still be expensive. So another improvement you can make here is to cache the result of every compare output, which is actually what Haskell's nubOrdOn operator does. ┳━┳ ヽ(ಠل͜ಠ)ﾉ
The functor approach is used extensively in Jane Street's OCaml Base library. The quick solution was to pass around an 'a * 'a -> order function around every single time you nub something. One moral, though, is that while the module system does add verbosity, if you provide enough of this machinery in a standard library, it will become quite convenient.
If you think the improvement from O(n²) to O(n log n) is not enough, consider Fritz Henglein's Generic top-down discrimination for sorting and partitioning in linear time (2012) and Edward Kmett's Haskell discrimination package's nub for a O(n) nub.
Yes. This is possible in SML through use of parametric polymorphism. You want a function of most general type 'a list -> 'a list where 'a is a type variable (i.e., variable that ranges over types) that would be read as alpha.
For some more concrete examples of how you might apply this (the explicit type variable after fun is optional):
fun 'a id (x : 'a) : 'a = x
Here we have the identity function with type 'a -> 'a.
We can declare similar functions with some degree of specialisation of the types, for instance
fun map _ [] = []
| map f (x::xs) = f x :: map f xs
Where map has most general type ('a -> 'b) -> 'a list -> 'b list, i.e, takes two curried arguments, one with some function type and another with some list type (agrees with function's domain) and returns a new list with type given by the codomain of the function.
For your specific problem you'll probably also want to take an equality function in order to determine what is a "duplicate" or you'll probably restrict yourself to "equality types" (types that can be compared with op=, represented by type variables with two leading apostrophes, e.g., ''a).
Yes sml provides polymorphism to do such things. In many cases you actually don't care for the type of the item in your lists (or other structures). For instance this function checks (already present in the List structure) for the existence of an item in a list:
fun exists _ [] = false
| exists x (y :: l) = x = y orelse exists x l
Such function works for any type of list as long as the equal operator is defined for this type (such type is called an equality type). You can do the same for remove_duplicates. In order to work with list of items of non equality types you will have to give remove_duplicates an additional function that checks if two items are equal.
I have doubt about what's the real meaning of "order" when we say "high order" functions? E.g., I've got an embedded function call like:
f.g.h
So is it called a "3 order" function?
Is "High order" function a concept of static function accumulation? Then when I have a recursive function f, and in runtime its calling stack is like f.f.f.f. can we say f is high order function?
Thanks a lot.
The order is basically the nesting level of arrows in the type.
This lecture slide on Functional Programming defines
The order of data
Order 0: Non function data
Order 1: Functions with domain and range of order 0
Order 2: Functions with domain and range of order 1
Order k: Functions with domain and range of order k-1
So basically a zeroth-order function is no function, a first-order normal function that only operates on data, and everything else is a higher-order function.
(These slides seem to have an off-by-one error)
Let's get to some (Haskell) examples:
-- no arrow, clearly some plain data
x :: Int
x = 0
-- one arrow, so it's a first-order function:
add2 :: Int -> Int
add2 = (+ 2)
-- still one arrow only:
add :: (Int, Int) -> Int
add = uncurry (+)
-- and this is a first-order function as well:
add4 :: Int -> Int
add4 = add2 . add2
As you can see, it doesn't matter whether you are using function composition (a higher-order function) to define your functions, only their result type matters. Therefore your f.g.h and f.f.f.f examples are only first-order functions (given that f is one).
Simple examples of higher-order functions are multivariate functions:
-- two arrows! Looks like a second-order function
plus :: Int -> Int -> Int
plus = (+)
The type of the curried function is actually Int -> (Int -> Int) where we can clearly see that it's a function that has a first-order function as the result, so it is of order 2. The lower order of the input (0) doesn't really matter.
We can see the same in a more interesting example, function composition:
compose :: ((b -> c), (a -> b)) -> a -> c
compose (f, g) x = f (g x)
Here both of the parameters and the result are each a first-order function, so compose is of order 2.
Another example is the fixpoint combinator fix :: (a -> a) -> a which does have a first-order function as the input and a zeroth-order result, making it second order overall.
And the curried composition operator as we know it
(.) :: (b -> c) -> ((a -> b) -> (a -> c))
would even be considered a third-order function.
An example (javaish)
Function<Double, Double> oneOf(Function<Double, Double>... fs) {
return fs[new Random().nextInt(fs.length)];
}
double y = oneOf(cos, sin, atan).applyTo(1.23);
This is order 2 for two reasons: parameter type and especially the result type are functions.
Is it possible to create and send a list of functions as an argument to another function, and then have some functions within this list call other functions in this list?
For example, I want a function that works on a list passed as an argument, and then performs the first function in the list of functions on this list of numbers, and if that function makes calls to other functions within that list, they can be retrieved and used.
e.g.: deviation(Numbers, Functions) -> %Functions = [fun master/1, fun avg/1, fun meanroots/1]
Master calls avg, then passes that result into meanroots, etc. but at the end of the call chain the master is the one that will return a value.
I'd like to know if this is functionally possible, whether within OTP alone or using NIFs, and if there are samples of implementation to look at.
How would your function know whether one of the functions in the list called another function in the list? I think your question is worded confusingly.
You could create a function to chain the results through a series of functions:
chain(Nums, []) -> Nums;
chain(Nums, [H | T]) -> chain(H(Nums), T).
which could be done with a standard function:
lists:foldl(fun (F, V) -> F(V) end, Nums, Funcs)
Of course you can:
1> F1 = fun(A) -> A*A end.
#Fun<erl_eval.6.50752066>
2> F2 = fun(A) -> A+A end.
#Fun<erl_eval.6.50752066>
3> F3 = fun(A) -> A*A+A end.
#Fun<erl_eval.6.50752066>
4> Funs = [F1, F2, F3].
[#Fun<erl_eval.6.50752066>,#Fun<erl_eval.6.50752066>,
#Fun<erl_eval.6.50752066>]
5> [F(X) || X <- [1,2,3], F <- Funs].
[1,2,2,4,4,6,9,6,12]
You could create tagged tuples with functions, e.g. {tag1, F1} (where F1 is defined as above), pass them to functions and do with them all sort of stuff you would normally do with any other variable in Erlang.
I'm reading Harper's book, Intro to Standard ML, and am a bit confused on section 11.3, Returning Functions.
1) He defines a function that creates constant functions. He writes:
"Given a value k, the application constantly k yields a function that yields k whenever it is applied. Here's a definition of constantly:
val constantly = fn k => (fn a => k)
The function constantly has type 'a -> ('b -> 'a)." This makes sense to me: you supply a value of type 'a and it returns a function that always returns that value (of type 'a), regardless of the input (of type 'b, which may or may not be same as type 'a).
But then he says we can also define this function as:
fun constantly k a = k
This just looks like a function that takes two arguments and returns the first...but not a function that returns another function...
What am I missing?
2) A little later, Harper discusses a map function. I understand the basic map function. He then talks about a map function that will let us apply the passed-in function to many lists (instead of calling our original map function several times with the same passed-in function for each call). He writes:
fun map' f nil = nil
| map' f (h::t) = (f h) :: (map' f t)
"The function map so defined has type ('a -> 'b) -> 'a list -> 'b list. It takes a function of type 'a -> 'b as argument and yields another function of type 'a list -> 'b list as result."
I'm pretty lost here as it looks like map' just applies f to all of the elements in a list and returns the resulting list. So I would have thought it would be of type:
('a -> 'b) * 'a list -> 'b list
Where am I going wrong?
Thanks for the help,
bclayman
Both of your questions arise from a lack of clarity about what arguments to a function actually are. In SML, every function (recall that functions are values) takes precisely one argument. That argument may be a tuple (of type 'a * 'b but it is still a single argument (that needs to be destructured).
In SML, fun f x1 x2 ... = T is syntactic sugar for val rec f = fn x1 => fn x2 => ... => T. So constantly k evaluates to fn a => k.
Whether you give map the type ('a -> 'b) * 'a list -> 'b list or ('a -> 'b) -> 'a list -> 'b list is a library design issue, but the effect is the same. They do the same thing, though the first takes a tuple of a function and a list, while the second takes the function first and returns a function from lists to lists. This is called "currying" in the programming languages literature. (The tuple version is "uncurried", the other is "curried".)
The magic word is "currying": https://en.wikipedia.org/wiki/Currying
In ML (as well as Haskell) function application binds tighter than any other operator. Thus constantly k a parses as (constantly k) a. constantly k binds to a function. To understand what function, you can mentally think of the definition
fun constantly k a = k
as being equivalent to
fun (constantly k) a = k
This says that (constantly k) is the function that maps any a to k.
Thus its type is 'a -> ('b -> 'a), as claimed in the text.
There is nothing illegal about defining constantly via
fun constantly (k,a) = k
It will have the type 'a * 'b -> 'a which you seem to expect, but calling it "constantly" then is a bit weird since it wouldn't be a constant function.
I'm sure that Harper will get to it sooner or later. Such functions are heavily used in both ML and Haskell (especially Haskell).
If somebody may help me to understand this code:
let rec fold_tree f (T(x,l))=
f x (map (fold_tree f) l);;
How it works? I mean mainly recursion.
If I were writing a fold function for trees I'd be trying to define something of type ('a -> 'b -> 'b) -> 'a tree -> b -> b. That is, the function to be folded would take the value from a tree node and an accumulated value and produce a new accumulated value. (The interesting question then would be the order to visit the nodes.)
In this fold_tree, the function to be folded has type ('a -> 'b list -> 'b). That is, it takes a list of accumulated values instead of a single one. That makes it a little clumsy to use.
However, it does work after a fashion. In essence the definition says the following in prose: for a leaf, call your function f passing it the node value x and an empty list. For a non-leaf, first call yourself recursively on all the subtrees, then call the function f passing it the node value and the list of results of the recursive calls.
I hope this helps.
I suppose your data type declaration to be the following :
type 'a ntree = T of 'a * 'a ntree list;;
Now the best way to figure it out is :
1- take a look to it type signature,
val fold_tree : ('a -> 'b list -> 'b) -> 'a ntree -> 'b = <fun>
2 - Write a test case, according to the type signature,
let test_rtree = T("a", [ T("b", []) ; T("c", [ T ("f", [])]) ; T("d", [])])
in fold_tree (List.fold_left (^)) test_rtree;;
- : string = "abcfd"
3 - Take a paper a pencil and try to reproduce the output of the test case.