Flattening a List of Lists - functional-programming

I'm new to Scheme and functional programming in general. Can someone explain this code — specifically what kons and knil are? The goal is to flatten a list of lists.
(define (fold1 kons knil lst)
(if (null? lst)
knil
(fold1 kons (kons (car lst) knil) (cdr lst))))
I'm fairly certain kons is a function as it's being applied to two arguments but still not totally sure about its functionality.

This is a (weird) fold
This is a generalized folding procedure. In Lisps, lists are represented by cons cells and the empty list, where each (proper) list is either the empty list (), or a cons cell whose car is an element of the list and whose cdr is the rest of the list. E.g., a list (1 2 3 4 5) can be produced by
(cons 1 (cons 2 (cons 3 (cons 4 (cons 5 '())))))
The fold1 function that you've shown:
(define (fold1 kons knil lst)
(if (null? lst)
knil
(fold1 kons (kons (car lst) knil) (cdr lst))))
is a a way of taking a list like the one shown above and transforming it to:
(kons 5 (kons 4 (kons 3 (kons 2 (kons 1 knil)))))
This is a fold. This is an efficient generalization of lots of operations. For instance, if you use 0 as knil and + as kons, you compute the sum of the elements in the list.
Usually folds are right or left associative. A proper left-associative fold would transform to
(kons (kons (kons (kons (kons knil 1) 2) 3) 4) 5)
which might be clearer when viewed with + and infix notation:
(((((0 + 1) + 2) + 3) + 4) + 5)
The right associative fold would become
(1 + (2 + (3 + (4 + (5 + 0)))))
The left associative fold can be more efficient because the natural implementation is tail recursive, and elements are consumed from the list in the order that they can be extracted from the list. E.g., in the proper left associatve example, (kons knil 1) can be evaluated first to produce some value v, and then, in the same stack space, (kons v 2) can be evaluated, and so on. The right associative method requires traversing to the end of the list first. A naïve implementation requires stack space proportional to the length of the list.
This fold1 mixes things up a bit, because it's processing the elements of the list in a left associative manner, but the order of the arguments to the combining function is reversed.
This type of definition can be used any time that you have a algebraic datatype. Since a list in Lisps is either the empty list, or an element and a list combined with cons, you can write a function that handles each of these cases, and produces a new value by “replacing” cons with a combination function and the empty list with some designated value.
Flattening a list of lists
So, if you've got a list of lists, e.g., ((a b) (c d) (e f)), it's constructed by
(cons '(a b) (cons '(c d) (cons '(e f) '())))
With a right associative fold, you transform it to:
(append '(a b) (append '(c d) (append '(e f) '())))
by using append for kons, and '() for knil. However, in this slightly mixed up fold, your structure will be
(kons '(e f) (kons '(c d) (kons '(a b) knil)))
so knil can still be '(), but kons will need to be a function that calls append, but swaps the argument order:
(define (flatten lists)
(fold1 (lambda (right left)
(append left right))
'()
lists))
And so we have:
(flatten '((a b) (c d) (e f)))
;=> (a b c d e f)
Flattening deeper lists of lists
Given that this is a folding exercise, I expected that the list of lists are nested only one layer deep. However, since we've seen how to implement a simple flatten
(define (flatten lists)
(fold1 (lambda (right left)
(append left right))
'()
lists))
we can modify this to make sure that deeper lists are flattened, too. The kons function now
(lambda (right left)
(append left right))
simply appends the two lists together. left is the already appended and flattened list that we've been building up. right is the new component that we're taking on now. If we make a call to flatten that, too, that should flatten arbitrarily nested lists:
(define (flatten lists)
(fold1 (lambda (right left)
(append left (flatten right))) ; recursively flatten sublists
'()
lists))
This is almost right, except that now when we call (flatten '((a b) (c d))), we'll end up making a call to (flatten '(a b)), which will in turn make a call to (flatten 'a), but flatten is a wrapper for fold1, and fold1 expects its arguments to be lists. We need to decide what to do when flatten is called with a non-list. A simple approach is to have it return a list containing the non-list argument. That return value will mesh nicely with the append that's receiving the value.
(define (flatten lists) ; lists is not necessarily a list of lists anymore,
(if (not (pair? lists)) ; perhaps a better name should be chosen
(list lists)
(fold1 (lambda (right left)
(append left (flatten right)))
'()
lists)))
Now we have
(flatten '(a (b (c)) (((d)))))
;=> (a b c d)

The procedure shown is an implementation of fold:
In functional programming, fold – also known variously as reduce, accumulate, aggregate, compress, or inject – refers to a family of higher-order functions that analyze a recursive data structure and recombine through use of a given combining operation the results of recursively processing its constituent parts, building up a return value
Take note:
The kons parameter is a two-argument function that's used for "combining" the current element of the list being processed with the accumulated value
The knil parameter is the accumulated output result
To see how this works, imagine for a moment that we have a function such as this:
(define (reverse knil lst)
(if (null? lst)
knil
(reverse (cons (car lst) knil) (cdr lst))))
(reverse '() '(1 2 3 4))
=> '(4 3 2 1)
In the above knil is used to accumulate the result, and it starts in a value of '() because we're building a list as output. And kons is called cons, which builds lists. Let's see another example:
(define (add knil lst)
(if (null? lst)
knil
(add (+ (car lst) knil) (cdr lst))))
(add 0 '(1 2 3 4))
=> 10
In the above knil is used to accumulate the result, and it starts in a value of 0 because we're building a number as output. And kons is called +, which adds numbers.
By now you must have realized that both examples share the same structure of a solution, both consume an input list and the only things that change is how we "combine" the values pulled from the list and the starting accumulated value. If we're smart, we can factor out the parts that change into a higher order procedure, that receives the changing parts as parameters - thus fold1 is born:
(define (fold1 kons knil lst)
(if (null? lst)
knil
(fold1 kons (kons (car lst) knil) (cdr lst))))
And both of the above examples can be easily expressed in terms of fold1, just pass along the right parameters:
(define (reverse lst)
(fold1 cons '() lst))
(define (add lst)
(fold1 + 0 lst))
Now for the second part of the question: if you want to flatten a list with fold1 you can try this:
(define (helper x lst)
(if (pair? x)
(fold1 helper lst x)
(cons x lst)))
(define (flatten lst)
(reverse (helper lst '())))
(flatten '(1 2 (3) (4 (5)) 6))
=> '(1 2 3 4 5 6)

Following code using 'named let' and 'for' loop can be used to flatten the list of elements which themselves may be lists:
(define (myflatten ll)
(define ol '())
(let loop ((ll ll))
(for ((i ll))
(if (list? i)
(loop i)
(set! ol (cons i ol)))))
(reverse ol))
(myflatten '(a () (b e (c)) (((d)))))
Output:
'(a b e c d)
However, it uses 'set!' which is generally not preferred.
The 'for' loop can also be replaced by 'named let' recursion:
(define (myflatten ll)
(define ol '())
(let outer ((ll ll))
(let inner ((il ll))
(cond
[(empty? il)]
[(list? (first il))
(outer (first il))
(inner (rest il))]
[else
(set! ol (cons (first il) ol))
(inner (rest il))])))
(reverse ol))

Even shorter
(define (myflatten lists)
(foldr append empty lists))
(myflatten (list (list 1 2) (list 3 4 5) (list 6 7) (list 8 9 10)))
> (list 1 2 3 4 5 6 7 8 9 10)

Related

Scheme: high order procedure and recursion

I'm using scheme as part of a course I'm taking. I've been told to use high order functions in my homework. However this instruction seems somewhat unclear.
I don't fully understand the idea of a high order procedure. I'm able to do all the questions using recursion but that it's the same thing. Can anyone explain it with an example of a recursive function vs one written with a high order procedure.
As a second question:
Example: try to grab all the odd numbers
I could use (flatten (map odd ((1 4 5) (4 5 1 4 9)))), but what if there were nested lists, can I use map on a nested lists like:
(flatten (map odd ((1 3 (9 5 7)))) ; is there a function for this or a clean way to do it?
The point of a higher-order function is to reduce the boilerplate in your code, and to decrease coupling between the looping (technically it's a recursion, but its purpose is looping, so I will refer to it as such) and the actual logic. Here's an example (re grabbing all the odd numbers): a manual loop would look like this:
(define (filter-odd lst)
(cond ((null? lst) '())
((odd? (car lst)) (cons (car lst) (filter-odd (cdr lst))))
(else (filter-odd (cdr lst)))))
But notice that you've got the looping and the filtering in one function. This makes it harder to figure out what the function is doing, since these two unrelated operations are coupled together. With higher-level functions, you can do differently:
(define (filter pred lst)
(cond ((null? lst) '())
((pred (car lst)) (cons (car lst) (filter pred (cdr lst))))
(else (filter pred (cdr lst)))))
(define (filter-odd lst)
(filter odd? lst))
Notice now, how odd? is separated from the looping logic, which has now been separated into filter? filter now takes a function object which decides whether the item is to be kept or not, and callers of filter can slot in any function of their choice.
That is what is meant by a higher-order function: it's a function that takes a function object as a parameter, to customise its operation.
As mentioned in the original edit of your question, map is another higher-order function, but instead of filtering items from a list, it returns a transformation of every item in the original list, where the specific transformation is given by map's caller via a parameter.
To answer your actual question about flattening, etc., (map filter-odd '((1 3 (9 5 7)))) will return a list with a single item, the result of calling (filter-odd '(1 3 (9 5 7))). So no, map will not recurse into sublists for you (and neither will filter).
But you can flatten the input first (since you're flattening the output anyway), then call filter-odd on that directly. I hope that will give you the result you expect.
(I renamed your odd to filter-odd, since that is less likely to be confused with odd? (the predicate).)
Bonus material
By the way, both filter and map are specialisations of a much more general higher-order function, called a fold (or more specifically, a right-fold). Folds can express things that cannot be accommodated by either filter or map, but that somehow involve traversing all the items in a list. Here's an example of a length function, expressed as a fold:
(define (foldl func init lst)
(if (null? lst) init
(foldl func (func (car lst) init) (cdr lst))))
(define (length lst)
(foldl (lambda (elem count)
(+ count 1))
0 lst))
The benefit here is that the length function does not have to worry about traversing the list: that is handled by the fold. It only needs to worry about what to do at each iteration (which, here, is simply adding 1 to count, which starts out as 0).
In this case, the length is the same whether we traverse from the left or the right, and in Scheme, traversing from the left is more space-efficient, so we prefer that. But for implementing map and filter, a right-fold is necessary (otherwise the elements come out reversed---try substituting the foldr with foldl in the below functions and you'll see):
(define (foldr func init lst)
(if (null? lst) init
(func (car lst) (foldr func init (cdr lst)))))
(define (map func lst)
(foldr (lambda (elem result)
(cons (func elem) result))
'() lst))
(define (filter pred lst)
(foldr (lambda (elem result)
(if (pred elem)
(cons elem result)
result))
'() lst))

Sum of values in a list squared in Scheme

I'm trying to have the following program work, but for some reason it keeps telling me that my input doesnt contain the correct amount of arguments, why? here is the program
(define (sum f lst)
(cond
((null? lst)
0)
((pair? (car lst))
(+(f(sum (f car lst))) (f(sum (f cdr lst)))))
(else
(+ (f(car lst)) (f(sum (f cdr lst)))))))
and here is my input: (sum (lambda (x) (* x x)) '(1 2 3))
Thanks!
btw I take no credit for the code, Im just having fun with this one (http://groups.engin.umd.umich.edu/CIS/course.des/cis400/scheme/listsum.htm)
You're indeed passing the wrong number of arguments to the procedures sum and f, notice that the expressions (sum (f car lst)), (sum (f cdr lst)) are wrong, surely you meant (sum f (car lst)), (sum f (cdr lst)) - you don't want to apply f (a single-parameter procedure) to the two parameters that you're passing, and sum expects two arguments, but only one is passed. Try this instead:
(define (sum f lst)
(cond ((null? lst)
0)
((pair? (car lst))
(+ (sum f (car lst)) (sum f (cdr lst))))
(else
(+ (f (car lst)) (sum f (cdr lst))))))
More important: you're calling the f procedure in the wrong places. Only one call is needed in the last line, for the case when (car lst) is just a number and not a list - in the other places, both (car lst) and (cdr lst) are lists that need to be traversed; simply pass f around as a parameter taking care of correctly advancing the recursion.
Let's try the corrected procedure with a more interesting input - as it is, the procedure is capable of finding the sum of a list of arbitrarily nested lists:
(sum (lambda (x) (* x x)) '(1 (2) (3 (4)) 5))
> 55
You should take a look at either The Little Schemer or How to Design Programs, both books will teach you how to structure the solution for this kind of recursive problems over lists of lists.

Scheme / Racket Best Practice - Recursion vs Variable Accumulation

I'm new to Scheme (via Racket) and (to a lesser extent) functional programming, and could use some advise on the pros and cons of accumulation via variables vs recursion. For the purposes of this example, I'm trying to calculate a moving average. So, for a list '(1 2 3 4 5), the 3 period moving average would be '(1 2 2 3 4). The idea is that any numbers before the period are not yet part of the calculation, and once we reach the period length in the set, we start averaging the subset of the list according the chosen period.
So, my first attempt looked something like this:
(define (avg lst)
(cond
[(null? lst) '()]
[(/ (apply + lst) (length lst))]))
(define (make-averager period)
(let ([prev '()])
(lambda (i)
(set! prev (cons i prev))
(cond
[(< (length prev) period) i]
[else (avg (take prev period))]))))
(map (make-averager 3) '(1 2 3 4 5))
> '(1 2 2 3 4)
This works. And I like the use of map. It seems composible and open to refactoring. I could see in the future having cousins like:
(map (make-bollinger 5) '(1 2 3 4 5))
(map (make-std-deviation 2) '(1 2 3 4 5))
etc.
But, it's not in the spirit of Scheme (right?) because I'm accumulating with side effects. So I rewrote it to look like this:
(define (moving-average l period)
(let loop ([l l] [acc '()])
(if (null? l)
l
(let* ([acc (cons (car l) acc)]
[next
(cond
[(< (length acc) period) (car acc)]
[else (avg (take acc period))])])
(cons next (loop (cdr l) acc))))))
(moving-average '(1 2 3 4 5) 3)
> '(1 2 2 3 4)
Now, this version is more difficult to grok at first glance. So I have a couple questions:
Is there a more elegant way to express the recursive version using some of the built in iteration constructs of racket (like for/fold)? Is it even tail recursive as written?
Is there any way to write the first version without the use of an accumulator variable?
Is this type of problem part of a larger pattern for which there are accepted best practices, especially in Scheme?
It's a little strange to me that you're starting before the first of the list but stopping sharply at the end of it. That is, you're taking the first element by itself and the first two elements by themselves, but you don't do the same for the last element or the last two elements.
That's somewhat orthogonal to the solution for the problem. I don't think the accumulator is making your life any easier here, and I would write the solution without it:
#lang racket
(require rackunit)
;; given a list of numbers and a period,
;; return a list of the averages of all
;; consecutive sequences of 'period'
;; numbers taken from the list.
(define ((moving-average period) l)
(cond [(< (length l) period) empty]
[else (cons (mean (take l period))
((moving-average period) (rest l)))]))
;; compute the mean of a list of numbers
(define (mean l)
(/ (apply + l) (length l)))
(check-equal? (mean '(4 4 1)) 3)
(check-equal? ((moving-average 3) '(1 3 2 7 6)) '(2 4 5))
Well, as a general rule, you want to separate the manner in which you recurse and/or iterate from the content of the iteration steps. You mention fold in your question, and this points in the right step: you want some form of higher-order function that will handle the list traversal mechanics, and call a function you supply with the values in the window.
I cooked this up in three minutes; it's probably wrong in many ways, but it should give you an idea:
;;;
;;; Traverse a list from left to right and call fn with the "windows"
;;; of the list. fn will be called like this:
;;;
;;; (fn prev cur next accum)
;;;
;;; where cur is the "current" element, prev and next are the
;;; predecessor and successor of cur, and accum either init or the
;;; accumulated result from the preceeding call to fn (like
;;; fold-left).
;;;
;;; The left-edge and right-edge arguments specify the values to use
;;; as the predecessor of the first element of the list and the
;;; successor of the last.
;;;
;;; If the list is empty, returns init.
;;;
(define (windowed-traversal fn left-end right-end init list)
(if (null? list)
init
(windowed-traversal fn
(car list)
right-end
(fn left-end
(car list)
(if (null? (cdr list))
right-end
(second list))
init)
(cdr list))))
(define (moving-average list)
(reverse!
(windowed-traversal (lambda (prev cur next list-accum)
(cons (avg (filter true? (list prev cur next)))
list-accum))
#f
#f
'()
list)))
Alternately, you could define a function that converts a list into n-element windows and then map average over the windows.
(define (partition lst default size)
(define (iter lst len result)
(if (< len 3)
(reverse result)
(iter (rest lst)
(- len 1)
(cons (take lst 3) result))))
(iter (cons default (cons default lst))
(+ (length lst) 2)
empty))
(define (avg lst)
(cond
[(null? lst) 0]
[(/ (apply + lst) (length lst))]))
(map avg (partition (list 1 2 3 4 5) 0 3))
Also notice that the partition function is tail-recursive, so it doesn't eat up stack space -- this is the point of result and the reverse call. I explicitly keep track of the length of the list to avoid either repeatedly calling length (which would lead to O(N^2) runtime) or hacking together a at-least-size-3 function. If you don't care about tail recursion, the following variant of partition should work:
(define (partition lst default size)
(define (iter lst len)
(if (< len 3)
empty
(cons (take lst 3)
(iter (rest lst)
(- len 1)))))
(iter (cons default (cons default lst))
(+ (length lst) 2)))
Final comment - using '() as the default value for an empty list could be dangerous if you don't explicitly check for it. If your numbers are greater than 0, 0 (or -1) would probably work better as a default value - they won't kill whatever code is using the value, but are easy to check for and can't appear as a legitimate average

Converting a function with two recursive calls in scheme to make it tail-recursive

Before I start: YES, this is homework from college. Before I get told that I'm lazy and evil: this part of the homework was to convert two functions we already had, this one is the 6th.
(define (flatten-list a-list)
(cond ((null? a-list) '())
((list? (car a-list))
(append (flatten-list (car a-list)) (flatten-list (cdr a-list))))
(else (cons (car a-list) (flatten-list (cdr a-list))))))
The function, as you can guess, flattens a list even if it's nested. My specific problem with the transformation comes in the (list? (car a-list)) condition, in which I'm doing two recursive calls. I already did fibonacci, which I can do by just having two "acummulators" on the tail recursion. However, my mind is not trained in this yet to know how it should go.
I would appreciate if I was given hints and not the result. Thanks!
Here's my solution:
(define (flatten-iter a-list)
(define (flat-do acc lst-interm lst)
(cond
((null? lst)
(reverse acc))
((and (list? lst-interm) (not (null? lst-interm)))
(flat-do acc (car lst-interm) (append (cdr lst-interm) lst)))
((not (list? lst-interm))
(flat-do (cons lst-interm acc) empty lst))
((list? (car lst))
(flat-do acc (car lst) (cdr lst)))
(else
(flat-do (cons (car lst) acc) empty (cdr lst)))))
(flat-do empty empty a-list))
(flatten-iter (list 1 (list 2 (list 3 4 (list 5 empty 6))) 7 8))
=> (1 2 3 4 5 6 7 8)
Tail-recrusive functions require that they never return, and thus you can't use stack for storing your program's state. Instead, you use function arguments to pass the state between function calls. Therefore, we need to determine how to maintain the state. Because the result of our function is list?, it's meaningful to grow an empty list; we're using acc for this purpose. You can see how it works in else branch above. But we should be able to process nested lists. And while we're going deeper, we should keep the rest elements of the nested list for further processing. Sample list: (list 1 (list 2 3) 4 5)
Until (list 2 3) we have already added 1 to accumulator. Since we can't use stack, we need some other place to store the rest elements of the list. And this place is the lst argument, which contains elements of the original list to be flattened. We can just append the lst to the rest elements (cdr (list 2 3)) which are (list 3), and proceed with the list's head we stumbled upon while flattening, i. e. (car (list 2 3)) which is just 2. Now, (and (list? lst-interm) (not (null? lst-interm))) succeeds because flat-do is called this way:
(flat-do (list 1) (list 2 3) (list 4 5))
and the condition triggers this code:
(flat-do (list 1) (car (list 2 3)) (append (cdr (list 2 3)) (list 4 5)))
flat-do again is called this way: (flat-do (list 1) 2 (list 3 4 5))
The condition (not (list? 2)) now succeeds and the code (flat-do (cons 2 1) empty (list 3 4 5)) is evaluated.
The rest processing is done with else branch until lst is null? and reverse is performed on acc. Function then returns the reversed accumulator.

Scheme accumulative recursion with lists

How can I pass a list as a parameter to a function adding elements to it recursively,and have it unmodified when it comes out of recursion?
I want to use the list at each level of recursion with the list having the values added by deeper recursion levels.
To be more specific I want to do a DFS search on a graph and I want to store in the list the nodes I visited.
One method of doing this is just to return the list so you have access to it at higher levels of recursion.
Another method is to have your list be stored in a variable outside of the recursion. In other words not stored on the stack. Since it is not a good idea to use a global variable for this we need to have some local recursion.
The following code is a foolish way to reverse a list but it does illustrate the technique I am talking about.
(define (letrecreverse lst)
(letrec ((retlist '())
(reverse (lambda (lst)
(if (null? lst)
'()
(begin
(set! retlist (cons (car lst) retlist))
(reverse (cdr lst)))))))
(reverse lst)
retlist))
(letrecreverse '(1 2 3 4))
;outputs '(4 3 2 1)
Can you adopt this technique for your purposes?
If you build a new list by consing a value onto an old list, that old list is unmodified.
(define old '(1 2 3))
(define new (cons 55 old))
new
>(55 1 2 3)
old
>(1 2 3)
The 'tail' of the first cons in "new" is the list "old". But old hasn't changed.
(cdr new)
> (1 2 3)
If I understood your question correctly, this could be one solution:
;; Just a helper to print the current list.
(define (show list)
(display "list = ")
(display list)
(newline)
(flush-output))
;; Maximum depth of recursion
(define max-recur 5)
;; Original list is backed-up here.
(define orig-list null)
(define (recur list depth)
(if (null? orig-list)
(set! orig-list list))
(cond ((< depth max-recur)
(show list)
(recur (cons (random max-recur) list) (add1 depth)))
(else orig-list)))
Sample run:
> (recur '(1) 0)
list = (1)
list = (1 1)
list = (2 1 1)
list = (3 2 1 1)
list = (4 3 2 1 1)
(1) ;; In the end you get the original list back.

Resources