Scheme / Racket Best Practice - Recursion vs Variable Accumulation - recursion

I'm new to Scheme (via Racket) and (to a lesser extent) functional programming, and could use some advise on the pros and cons of accumulation via variables vs recursion. For the purposes of this example, I'm trying to calculate a moving average. So, for a list '(1 2 3 4 5), the 3 period moving average would be '(1 2 2 3 4). The idea is that any numbers before the period are not yet part of the calculation, and once we reach the period length in the set, we start averaging the subset of the list according the chosen period.
So, my first attempt looked something like this:
(define (avg lst)
(cond
[(null? lst) '()]
[(/ (apply + lst) (length lst))]))
(define (make-averager period)
(let ([prev '()])
(lambda (i)
(set! prev (cons i prev))
(cond
[(< (length prev) period) i]
[else (avg (take prev period))]))))
(map (make-averager 3) '(1 2 3 4 5))
> '(1 2 2 3 4)
This works. And I like the use of map. It seems composible and open to refactoring. I could see in the future having cousins like:
(map (make-bollinger 5) '(1 2 3 4 5))
(map (make-std-deviation 2) '(1 2 3 4 5))
etc.
But, it's not in the spirit of Scheme (right?) because I'm accumulating with side effects. So I rewrote it to look like this:
(define (moving-average l period)
(let loop ([l l] [acc '()])
(if (null? l)
l
(let* ([acc (cons (car l) acc)]
[next
(cond
[(< (length acc) period) (car acc)]
[else (avg (take acc period))])])
(cons next (loop (cdr l) acc))))))
(moving-average '(1 2 3 4 5) 3)
> '(1 2 2 3 4)
Now, this version is more difficult to grok at first glance. So I have a couple questions:
Is there a more elegant way to express the recursive version using some of the built in iteration constructs of racket (like for/fold)? Is it even tail recursive as written?
Is there any way to write the first version without the use of an accumulator variable?
Is this type of problem part of a larger pattern for which there are accepted best practices, especially in Scheme?

It's a little strange to me that you're starting before the first of the list but stopping sharply at the end of it. That is, you're taking the first element by itself and the first two elements by themselves, but you don't do the same for the last element or the last two elements.
That's somewhat orthogonal to the solution for the problem. I don't think the accumulator is making your life any easier here, and I would write the solution without it:
#lang racket
(require rackunit)
;; given a list of numbers and a period,
;; return a list of the averages of all
;; consecutive sequences of 'period'
;; numbers taken from the list.
(define ((moving-average period) l)
(cond [(< (length l) period) empty]
[else (cons (mean (take l period))
((moving-average period) (rest l)))]))
;; compute the mean of a list of numbers
(define (mean l)
(/ (apply + l) (length l)))
(check-equal? (mean '(4 4 1)) 3)
(check-equal? ((moving-average 3) '(1 3 2 7 6)) '(2 4 5))

Well, as a general rule, you want to separate the manner in which you recurse and/or iterate from the content of the iteration steps. You mention fold in your question, and this points in the right step: you want some form of higher-order function that will handle the list traversal mechanics, and call a function you supply with the values in the window.
I cooked this up in three minutes; it's probably wrong in many ways, but it should give you an idea:
;;;
;;; Traverse a list from left to right and call fn with the "windows"
;;; of the list. fn will be called like this:
;;;
;;; (fn prev cur next accum)
;;;
;;; where cur is the "current" element, prev and next are the
;;; predecessor and successor of cur, and accum either init or the
;;; accumulated result from the preceeding call to fn (like
;;; fold-left).
;;;
;;; The left-edge and right-edge arguments specify the values to use
;;; as the predecessor of the first element of the list and the
;;; successor of the last.
;;;
;;; If the list is empty, returns init.
;;;
(define (windowed-traversal fn left-end right-end init list)
(if (null? list)
init
(windowed-traversal fn
(car list)
right-end
(fn left-end
(car list)
(if (null? (cdr list))
right-end
(second list))
init)
(cdr list))))
(define (moving-average list)
(reverse!
(windowed-traversal (lambda (prev cur next list-accum)
(cons (avg (filter true? (list prev cur next)))
list-accum))
#f
#f
'()
list)))

Alternately, you could define a function that converts a list into n-element windows and then map average over the windows.
(define (partition lst default size)
(define (iter lst len result)
(if (< len 3)
(reverse result)
(iter (rest lst)
(- len 1)
(cons (take lst 3) result))))
(iter (cons default (cons default lst))
(+ (length lst) 2)
empty))
(define (avg lst)
(cond
[(null? lst) 0]
[(/ (apply + lst) (length lst))]))
(map avg (partition (list 1 2 3 4 5) 0 3))
Also notice that the partition function is tail-recursive, so it doesn't eat up stack space -- this is the point of result and the reverse call. I explicitly keep track of the length of the list to avoid either repeatedly calling length (which would lead to O(N^2) runtime) or hacking together a at-least-size-3 function. If you don't care about tail recursion, the following variant of partition should work:
(define (partition lst default size)
(define (iter lst len)
(if (< len 3)
empty
(cons (take lst 3)
(iter (rest lst)
(- len 1)))))
(iter (cons default (cons default lst))
(+ (length lst) 2)))
Final comment - using '() as the default value for an empty list could be dangerous if you don't explicitly check for it. If your numbers are greater than 0, 0 (or -1) would probably work better as a default value - they won't kill whatever code is using the value, but are easy to check for and can't appear as a legitimate average

Related

How can I make my average function tail recursive in Lisp

I am simply trying to make this average function to be tail recursive. I have managed to get my function to work and that took some considerable effort. Afterwards I went to ask my professor if my work was satisfactory and he informed me that
my avg function was not tail recursive
avg did not produce the correct output for lists with more than one element
I have been playing around with this code for the past 2 hours and have hit a bit of a wall. Can anyone help me to identify what I am not understanding here.
Spoke to my professor he was != helpful
(defun avg (aList)
(defun sumup (aList)
(if (equal aList nil) 0
; if aList equals nil nothing to sum
(+ (car aList) (sumup (cdr aList)) )
)
)
(if
(equal aList nil) 0
; if aList equals nil length dosent matter
(/ (sumup aList) (list-length aList) )
)
)
(print (avg '(2 4 6 8 19))) ;39/5
my expected results for my test are commented right after it 39/5
So this is what I have now
(defun avg (aList &optional (sum 0) (length 0))
(if aList
(avg (cdr aList) (+ sum (car aList))
(+ length 1))
(/ sum length)))
(print (avg '(2 4 6 8 19))) ;39/5
(defun avg (list &optional (sum 0) (n 0))
(cond ((null list) (/ sum n))
(t (avg (cdr list)
(+ sum (car list))
(+ 1 n)))))
which is the same like:
(defun avg (list &optional (sum 0) (n 0))
(if (null list)
(/ sum n)
(avg (cdr list)
(+ sum (car list))
(+ 1 n))))
or more similar for your writing:
(defun avg (list &optional (sum 0) (n 0))
(if list
(avg (cdr list)
(+ sum (car list))
(+ 1 n))
(/ sum n)))
(defun avg (lst &optional (sum 0) (len 0))
(if (null lst)
(/ sum len)
(avg (cdr lst) (incf sum (car lst)) (1+ len))))
You could improve your indentation here by putting the entire if-then/if-else statement on the same line, because in your code when you call the avg function recursively the indentation bleeds into the next line. In the first function you could say that if the list if null (which is the base case of the recursive function) you can divide the sum by the length of the list. If it is not null, you can obviously pass the cdr of the list, the sum so far by incrementing it by the car of the list, and then increment the length of the list by one. Normally it would not be wise to use the incf or 1+ functions because they are destructive, but in this case they will only have a localized effect because they only impact the optional sum and len parameters for this particular function, and not the structure of the original list (or else I would have passed a copy of the list).
Another option would be to use a recursive local function, and avoid the optional parameters and not have to compute the length of the list on each recursive call. In your original code it looks like you were attempting to use a local function within the context of your avg function, but you should use the "labels" Special operator to do that, and not "defun":
(defun avg (lst)
(if (null lst)
0
(labels ((find-avg (lst sum len)
(if (null lst)
(/ sum len)
(find-avg (cdr lst) (incf sum (car lst)) len))))
(find-avg lst 0 (length lst))))
I'm not 100% sure if your professor would want the local function to be tail-recursive or if he was referring to the global function (avg), but that is how you could also make the local function tail-recursive if that is an acceptable remedy as well. It's actually more efficient in some ways, although it requires more lines of code. In this case a lambda expression could also work, BUT since they do not have a name tail-recursion is not possibly, which makes the labels Special operator is useful for local functions if tail-recursion is mandatory.

Referencing variables in racket

I'm trying to write a simple procedure for finding the n:th prime but I don't think I understand how to reference variables correctly in racket.
What I want is for the inner procedure sieve-iter to add primes to the list primlst which is in the namespace of sieve but I get an infinite loop. My guess is that primlst within sieve-iter is causing issues.
(define (sieve n) ;; returns the n:th prime (n>0)
(let [(primlst '(2))
(cand 3)]
(define (sieve-iter i lst)
(cond ((null? lst) (and (cons i primlst) (sieve-iter (+ i 2) primlst))) ;;prime
((= (length primlst) n) (car primlst)) ;;end
((= (modulo i (car lst)) 0) (sieve-iter (+ i 2) primlst)) ;;non-prime
(#t (sieve-iter n (cdr lst))))) ;;unclear if prime
(sieve-iter cand primlst)))
Any help is appreciated!
First of all, you shouldn't refer to primlist at all within the sieve-iter function. Instead, you should refer to lst.
Second of all, you appear to be mistaken on the effect of this expression:
(and (cons i primlst) (sieve-iter (+ i 2) primlst))
You seem to be interpreting that as meaning "Add i to the primlist and then start the next iteration."
(cons i primlist) changes nothing. Instead, it creates a new list consisting of primlist with i in front of it and then evaluates to that value. The original primlist (which should have been lst anyway) is left untouched.
Also, and is for Boolean logic, not for stringing commands together. It evaluates each of its subexpressions separately until it finds one that evaluates to #f and then it stops.
You should replace that whole expression with this:
(sieve-iter (+ i 2) (cons i lst))
...which passes the new list created by cons to the next run of sieve-iter.
Your trying to do too much in one function, let prime-iter just worry about the iteration to build up the list of primes. Make another internal function to recurse down the existing primes to test the new candidate.
(define (sieve n) ;; returns the n:th prime (n>0)
(define (sieve-iter i lst remaining)
(cond ;((null? lst) (and (cons i primlst) (sieve-iter (+ i 2) primlst))) ;;should never be null, we are building the list up
((<= remaining 0) (car lst)) ;;checking length every time gets expensive added a variable to the function
((sieve-prime? i lst) ;;if prime add to lst and recurse on next i
(sieve-iter (+ i 2) (cons i lst) (- remaining 1)))
(else
(sieve-iter (+ i 2) lst remaining)))) ; else try next
(define (sieve-prime? i lst)
(cond ((null? lst) #t)
((= 0 (modulo i (car lst))) #f)
(else (sieve-prime? i (cdr lst)))))
(let ((primlst '(2)) ;;you generally don't modify these,
(cand 3)) ;mostly they just bind values to name for convenience or keep from having to re-calculate the same thing more than once
(sieve-iter cand primlst (- n 1))))
You could have used set! to modify primlist where it was before, but the procedure is no longer obviously a pure function.
There is another low-handing optimization possible here, when calling sieve-prime? filter the lst argument to remove values larger than the square root of i.

Common lisp recursive macro in matrix addition

I have to write a recursive macro for list addition in Common Lisp (homework). What I have so far is :
(defmacro matrix-add-row (r1 r2 sum_row)
(if (not (and r1 r2)) `sum_row
(progn
`(matrix-add-row (cdr r1) (cdr r2) (cons sum_row (+ (car r1) (car r2))))
(reverse sum_row)
)
)
)
I call this function with
(matrix-add-row `(1 2) `(3 4) ())
and as an output I get unvaluated code instead of numbers (which leads going to infinite loop).
How to put , ` properly (or call the macro properly)?
Firstly, to me this seems a rather bizarre thing to do with a macro. I assume the point is that you use the macro to transform (matrix-add-row '(1 2) '(3 4)) to an explicit list of sums like (list (+ 1 3) (+ 2 4)).
Also, what you have written has several problems which look like you don't quite understand how the backtick works. So I think the easiest way to help is to solve an example for you.
Since this is homework, I'm going to solve a different (but similar) question. You should be able to take the answer and use it for your example. Suppose I want to solve the following:
Write a macro, diffs, which computes all differences of pairs of successive elements in a list. For example,
(diffs '(1 2 3)) should expand to (list (- 2 1) (- 3 2)), which will then evaluate to (1 1).
Note that my macro won't do the actual subtraction, so I can use it even if I don't know some of the numbers until runtime. (The reason I think this sort of question is a bit weird is that it does need to know the length of the list at compile time).
My solution is going to be used as a macro with one argument but if I want to use recursion I'll need to pass in an accumulator too, which I can start with nil. So I write something like this:
(defmacro diffs (lst &optional accumulator)
...)
Now what do I do with lst? If lst is nil, I want to bottom out and just return the accumulator, with a call to list at the front, which will be code to make my list. Something like this:
(defmacro diffs (lst &optional accumulator)
(cond
((null lst)
;; You could write `(list ,#accumulator) instead, but that seems
;; unnecessarily obfuscated.
(cons 'list accumulator))
(t
(error "Aargh. Unhandled"))))
Let's try it!
CL-USER> (diffs nil)
NIL
Not hugely exciting, but it looks plausible. Now use macroexpand, which just does the expansion without the evaluation:
CL-USER> (macroexpand '(diffs nil))
(LIST)
T
And what if we'd already got some stuff from a recursion?
CL-USER> (macroexpand '(diffs nil ((- a b) (- b c))))
(LIST (- A B) (- B C))
T
Looks good! Now we need to deal with the case when there's an actual list there. The test you want is consp and (for my example) it only makes sense when there's at least two elements.
(defmacro diffs (lst &optional accumulator)
(cond
;; A list of at least two elements
((and (consp lst) (consp (cdr lst)))
(list 'diffs (cdr lst)
(cons (list '- (cadr lst) (car lst)) accumulator)))
;; A list with at most one element
((listp lst)
(cons 'list accumulator))
(t
(error "Aargh. Unhandled"))))
This seems almost to work:
CL-USER> (macroexpand '(diffs (3 4 5)))
(LIST (- 5 4) (- 4 3))
T
but for two problems:
The list comes out backwards
The code is a bit horrible when we actually construct the recursive expansion
Let's fix the second part first by using the backtick operator:
(defmacro diffs (lst &optional accumulator)
(cond
;; A list of at least two elements
((and (consp lst) (consp (cdr lst)))
`(diffs ,(cdr lst)
,(cons `(- ,(cadr lst) ,(car lst)) accumulator)))
;; A list with at most one element
((listp lst)
(cons 'list accumulator))
(t
(error "Aargh. Unhandled"))))
Hmm, it's not actually much shorter, but I think it's clearer.
For the second part, we could proceed by adding each item to the end of the accumulator rather than the front, but that's not particularly quick in Lisp because lists are singly linked. Better is to construct the accumulator backwards and then reverse it at the end:
(defmacro diffs (lst &optional accumulator)
(cond
;; A list of at least two elements
((and (consp lst) (consp (cdr lst)))
`(diffs ,(cdr lst)
,(cons `(- ,(cadr lst) ,(car lst)) accumulator)))
;; A list with at most one element
((listp lst)
(cons 'list (reverse accumulator)))
(t
(error "Aargh. Unhandled"))))
Now we get:
CL-USER> (macroexpand '(diffs (3 4 5)))
(LIST (- 4 3) (- 5 4))
T
Much better!
Two last things. Firstly, I still have an error clause in my macro. Can you see how to trigger it? Can you think of a better behaviour than just outputting an error? (Your macro is going to have to deal with the same problem)
Secondly, for debugging recursive macros like this, I recommend using macroexpand-1 which just unfolds one level at once. For example:
CL-USER> (macroexpand-1 '(diffs (3 4 5)))
(DIFFS (4 5) ((- 4 3)))
T
CL-USER> (macroexpand-1 *)
(DIFFS (5) ((- 5 4) (- 4 3)))
T
CL-USER> (macroexpand-1 *)
(LIST (- 4 3) (- 5 4))
T
There are two problems with your logic. First you are calling reverse on each iteration instead of at the end of the iteration. Then you are accumulating the new values, through cons, in the cdr of the cons cell as opposed to the car.
Also I don't see why this have to be a macro so using a function.
(defun matrix-add-row (r1 r2 sum-row)
(if (or (endp r1) (endp r2))
(reverse sum-row)
(matrix-add-row (cdr r1)
(cdr r2)
(cons (+ (car r1) (car r2))
sum-row))))
(matrix-add-row '(1 2) '(3 4) ())
;; => (4 6)

Overloaded function failing giving Compiler recursion error

With the following code, I get #<CompilerException java.lang.UnsupportedOperationException: Can only recur from tail position (NO_SOURCE_FILE:4)> despite the fact that all recurs are in tail positions. If I remove the recur from the one-argument version, it stops complaining. Why is this happening?
(defn remove-duplicates "Removes duplicate elements of lst.
For example, given (1 2 3 1 4 1 2), remove-duplicates returns a sequence
containing the elements (1 2 3 4), in some order."
[lst] (recur (rest lst) (set (first lst)))
[lst uniques] (cond (zero? (count lst)) uniques
:else (cond
(some (partial = (first lst)) uniques)
(recur (rest lst) uniques)
:else
(recur (rest lst) (first lst)))))
You haven't split up the multi-arity bodies right. Should read (defn foo ([x] (...)) ([x y] (...))). This causes the compiler to think you're doing totally different stuff, which probably accounts for your issue.
First of all: you know that all you want is (def remove-duplicates set) or -- if you want a vector -- (def remove-duplicates-vec (comp vec set)), right?
Five things here:
As amalloy noticed, you should've added parens
As kotarak noticed, you can't recur between arities
You can't call (set (first lst)) because set wants coll. If you want, do something like (set (vector (first [1 2 3 2 3]))) but this is neither pretty nor idiomatic
Doing (cond pred1 code1 :else (cond pred2a code2a :else code2b)) could be made simplier: (cond pred1 code1 pred2a code2a :else code2b) -- what you did is treated cond macro as if it were if (which is a built-in as far as I know)
Your last tail-call is also wrong. Assume we've started with [1 2 3 2 1]
When you call it first you have following arguments: ([2 3 2 1] #{1}) (I've skipped the boring part)
Then you have last predicate true, so you go with ([3 2 1] 2) and this is obviously wrong because you wanted ([3 2 1] #{1 2}). You probably want to call (recur (rest lst) (conj uniques (first lst)))
Summing up:
(defn remove-duplicates
([lst] (remove-duplicates (rest lst) #{(first coll)}))
([lst uniques]
(cond
(zero? (count lst)) uniques
(some (partial = (first lst)) uniques)
(recur (rest lst) uniques)
:else
(recur (rest lst) (conj uniques (first lst))))))

Scheme accumulative recursion with lists

How can I pass a list as a parameter to a function adding elements to it recursively,and have it unmodified when it comes out of recursion?
I want to use the list at each level of recursion with the list having the values added by deeper recursion levels.
To be more specific I want to do a DFS search on a graph and I want to store in the list the nodes I visited.
One method of doing this is just to return the list so you have access to it at higher levels of recursion.
Another method is to have your list be stored in a variable outside of the recursion. In other words not stored on the stack. Since it is not a good idea to use a global variable for this we need to have some local recursion.
The following code is a foolish way to reverse a list but it does illustrate the technique I am talking about.
(define (letrecreverse lst)
(letrec ((retlist '())
(reverse (lambda (lst)
(if (null? lst)
'()
(begin
(set! retlist (cons (car lst) retlist))
(reverse (cdr lst)))))))
(reverse lst)
retlist))
(letrecreverse '(1 2 3 4))
;outputs '(4 3 2 1)
Can you adopt this technique for your purposes?
If you build a new list by consing a value onto an old list, that old list is unmodified.
(define old '(1 2 3))
(define new (cons 55 old))
new
>(55 1 2 3)
old
>(1 2 3)
The 'tail' of the first cons in "new" is the list "old". But old hasn't changed.
(cdr new)
> (1 2 3)
If I understood your question correctly, this could be one solution:
;; Just a helper to print the current list.
(define (show list)
(display "list = ")
(display list)
(newline)
(flush-output))
;; Maximum depth of recursion
(define max-recur 5)
;; Original list is backed-up here.
(define orig-list null)
(define (recur list depth)
(if (null? orig-list)
(set! orig-list list))
(cond ((< depth max-recur)
(show list)
(recur (cons (random max-recur) list) (add1 depth)))
(else orig-list)))
Sample run:
> (recur '(1) 0)
list = (1)
list = (1 1)
list = (2 1 1)
list = (3 2 1 1)
list = (4 3 2 1 1)
(1) ;; In the end you get the original list back.

Resources