I am pretty much of a newbie to common-lisp and only use it for fun. But I assume to know the difference between mapper and mapcan, as the documentation in the hyperspec and other places is pretty clear.
But what happens if the function mapcan calls on the list elements evaluates to an atom instead of a list? As mapcan uses nconc to append lists, I had expected that there would be an error if there is no list.
But if I try
(mapcan (lambda (x) (+ 2 x)) '(1 2 3 4))
it evaluates to '6' in sbcl and clisp. (There might not be a practical need for this example; I am just curious) I see the point that returning a value might be nicer than a simpler error but could find anything about mapcan returning the last value if there are no lists to nconc.
Is there a reason for this behavior?
According to the documentation for mapcan (mapcan (lambda (x) (+ 2 x)) '(1 2 3 4)) Should do the same as (apply #'nconc (mapcar (lambda (x) (+ 2 x)) '(1 2 3 4))) and it signals an error the error *** - NCONC: 5 is not a list in clisp.
The hyperspec only shows what nconc should be doing with proper lists and nil as the arguments before last. It does not have anything else described so what you are seeing is that sbcl and clisp perhaps share the algorithm from a lisp in public domain or that they have implemented it so similar they have the same implementation specific results.
You probably cannot assume other implementations will do the same so you should make sure the function passed to mapcan always return a fresh list or nil that can be nconc-ed within the specification.
Related
I am trying to learn Common Lisp with the book Common Lisp: A gentle introduction to Symbolic Computation. In addition, I am using SBCL, Emacs, and Slime.
By the end of chapter 10, on the advanced section there is this question:
10.9. Write a destructive function CHOP that shortens any non-NIL list to a list of one element. (CHOP '(FEE FIE FOE FUM)) should return
(FEE).
This is the answer-sheet solution:
(defun chop (x)
(if (consp x) (setf (cdr x) nil))
x)
I understand this solution. However, before checking out the official solution I tried:
(defun chop (xs)
(cond ((null xs) xs)
(t (setf xs (list (car xs))))))
Using this as a global variable for tests:
(defparameter teste-chop '(a b c d))
I tried on the REPL:
CL-USER> (chop teste-chop)
(A)
As you can see, the function returns the expected result.
Unfortunately, the side-effect to mutate the original list does not happen:
CL-USER> teste-chop
(A B C D)
Why it did not change?
Since I was setting the field (setf) of the whole list to be only its car wrapped as a new list, I was expecting the cdr of the original list to be vanished.
Apparently, the pointers are not automatically removed.
Since I have very strong gaps in low-level stuff (like pointers) I thought that some answer to this question could educate me on why this happens.
The point is about how parameters to functions are passed in Common Lisp. They are passed by value. This means that, when a function is called, all arguments are evaluated, and their values are assigned to new, local variables, the parameters of the function. So, consider your function:
(defun chop (xs)
(cond ((null xs) xs)
(t (setf xs (list (car xs))))))
When you call it with:
(chop teste-chop)
the value of teste-chop, that is the list (a b c d) is assigned to the function parameter xs. In the last line of the body of function, by using setf, you are assigning a new value, (list (car xs)) to xs, that is you are assigning the list (a) to this local variable.
Since this is the last expression of the function, such value is also returned by the function, so that the evaluation of (chop test-chop) returns the value (a).
In this process, as you can see, the special variable teste-chop is not concerned in any way, apart from calculating its value at the beginning of the evaluation of the function call. And for this reason its value is not changed.
Other forms of parameter passing are used in other languages, like for instance by name, so that the behaviour of function call could be different.
Note that instead, in the first function, with (setf (cdr x) nil) a data structure is modified, that is a part of a cons cell. Since the global variable is bound to that cell, also the global variable will appear modified (even if, in a certain sense, it is not modified, since it remains bound to the same cons cell).
As a final remark, in Common Lisp it is better not to modify constant data structures (like those obtained by evaluating '(a b c d)), since it could produce an undefined behaviour, depending on the implementation. So, if some structure should be modifiable, it should be built with the usual operators like cons or list (e.g. (list 'a 'b 'c 'd)).
I wonder how do you, experienced lispers / functional programmers usually make decision what to use. Compare:
(define (my-map1 f lst)
(reverse
(let loop ([lst lst] [acc '()])
(if (empty? lst)
acc
(loop (cdr lst) (cons (f (car lst)) acc))))))
and
(define (my-map2 f lst)
(if (empty? lst)
'()
(cons (f (car lst)) (my-map2 f (cdr lst)))))
The problem can be described in the following way: whenever we have to traverse a list, should we collect results in accumulator, which preserves tail recursion, but requires list reversion in the end? Or should we use unoptimized recursion, but then we don't have to reverse anything?
It seems to me the first solution is always better. Indeed, there's additional complexity (O(n)) there. However, it uses much less memory, let alone calling a function isn't done instantly.
Yet I've seen different examples where the second approach was used. Either I'm missing something or these examples were only educational. Are there situations where unoptimized recursion is better?
When possible, I use higher-order functions like map which build a list under the hood. In Common Lisp I also tend to use loop a lot, which has a collect keyword for building list in a forward way (I also use the series library which also implements it transparently).
I sometimes use recursive functions that are not tail-recursive because they better express what I want and because the size of the list is going to be relatively small; in particular, when writing a macro, the code being manipulated is not usually very large.
For more complex problems I don't collect into lists, I generally accept a callback function that is being called for each solution. This ensures that the work is more clearly separated between how the data is produced and how it is used.
This approach is to me the most flexible of all, because no assumption is made about how the data should be processed or collected. But it also means that the callback function is likely to perform side-effects or non-local returns (see example below). I don't think it is particularly a problem as long the the scope of the side-effects is small (local to a function).
For example, if I want to have a function that generates all natural numbers between 0 and N-1, I write:
(defun range (n f)
(dotimes (i n)
(funcall f i)))
The implementation here iterates over all values from 0 below N and calls F with the value I.
If I wanted to collect them in a list, I'd write:
(defun range-list (N)
(let ((list nil))
(range N (lambda (v) (push v list)))
(nreverse list)))
But, I can also avoid the whole push/nreverse idiom by using a queue. A queue in Lisp can be implemented as a pair (first . last) that keeps track of the first and last cons cells of the underlying linked-list collection. This allows to append elements in constant time to the end, because there is no need to iterate over the list (see Implementing queues in Lisp by P. Norvig, 1991).
(defun queue ()
(let ((list (list nil)))
(cons list list)))
(defun qpush (queue element)
(setf (cdr queue)
(setf (cddr queue)
(list element))))
(defun qlist (queue)
(cdar queue))
And so, the alternative version of the function would be:
(defun range-list (n)
(let ((q (queue)))
(range N (lambda (v) (qpush q v)))
(qlist q)))
The generator/callback approach is also useful when you don't want to build all the elements; it is a bit like the lazy model of evaluation (e.g. Haskell) where you only use the items you need.
Imagine you want to use range to find the first empty slot in a vector, you could do this:
(defun empty-index (vector)
(block nil
(range (length vector)
(lambda (d)
(when (null (aref vector d))
(return d))))))
Here, the block of lexical name nil allows the anonymous function to call return to exit the block with a return value.
In other languages, the same behaviour is often reversed inside-out: we use iterator objects with a cursor and next operations. I tend to think it is simpler to write the iteration plainly and call a callback function, but this would be another interesting approach too.
Tail recursion with accumulator
Traverses the list twice
Constructs two lists
Constant stack space
Can crash with malloc errors
Naive recursion
Traverses list twice (once building up the stack, once tearing down the stack).
Constructs one list
Linear stack space
Can crash with stack overflow (unlikely in racket), or malloc errors
It seems to me the first solution is always better
Allocations are generally more time-expensive than extra stack frames, so I think the latter one will be faster (you'll have to benchmark it to know for sure though).
Are there situations where unoptimized recursion is better?
Yes, if you are creating a lazily evaluated structure, in haskell, you need the cons-cell as the evaluation boundary, and you can't lazily evaluate a tail recursive call.
Benchmarking is the only way to know for sure, racket has deep stack frames, so you should be able to get away with both versions.
The stdlib version is quite horrific, which shows that you can usually squeeze out some performance if you're willing to sacrifice readability.
Given two implementations of the same function, with the same O notation, I will choose the simpler version 95% of the time.
There are many ways to make recursion preserving iterative process.
I usually do continuation passing style directly. This is my "natural" way to do it.
One takes into account the type of the function. Sometimes you need to connect your function with the functions around it and depending on their type you can choose another way to do recursion.
You should start by solving "the little schemer" to gain a strong foundation about it. In the "little typer" you can discover another type of doing recursion, founded on other computational philosophy, used in languages like agda, coq.
In scheme you can write code that is actually haskell sometimes (you can write monadic code that would be generated by a haskell compiler as intermediate language). In that case the way to do recursion is also different that "usual" way, etc.
false dichotomy
You have other options available to you. Here we can preserve tail-recursion and map over the list with a single traversal. The technique used here is called continuation-passing style -
(define (map f lst (return identity))
(if (null? lst)
(return null)
(map f
(cdr lst)
(lambda (r) (return (cons (f (car lst)) r))))))
(define (square x)
(* x x))
(map square '(1 2 3 4))
'(1 4 9 16)
This question is tagged with racket, which has built-in support for delimited continuations. We can accomplish map using a single traversal, but this time without using recursion. Enjoy -
(require racket/control)
(define (yield x)
(shift return (cons x (return (void)))))
(define (map f lst)
(reset (begin
(for ((x lst))
(yield (f x)))
null)))
(define (square x)
(* x x))
(map square '(1 2 3 4))
'(1 4 9 16)
It's my intention that this post will show you the detriment of pigeonholing your mind into a particular construct. The beauty of Scheme/Racket, I have come to learn, is that any implementation you can dream of is available to you.
I would highly recommend Beautiful Racket by Matthew Butterick. This easy-to-approach and freely-available ebook shatters the glass ceiling in your mind and shows you how to think about your solutions in a language-oriented way.
I'm Haruo. My pleasure is solving SPOJ in Common Lisp(CLISP). Today I solved Classical/Balk! but in SBCL not CLISP. My CLISP submit failed due to runtime error (NZEC).
I hope my code becomes more sophisticated. Today's problem is just a chance. Please the following my code and tell me your refactoring strategy. I trust you.
https://github.com/haruo-wakakusa/SPOJ-ClispAnswers/blob/0978813be14b536bc3402f8238f9336a54a04346/20040508_adrian_b.lisp
Haruo
Take for example get-x-depth-for-yz-grid.
(defun get-x-depth-for-yz-grid (planes//yz-plane grid)
(let ((planes (get-planes-including-yz-grid-in planes//yz-plane grid)))
(unless (evenp (length planes))
(error "error in get-x-depth-for-yz-grid"))
(sort planes (lambda (p1 p2) (< (caar p1) (caar p2))))
(do* ((rest planes (cddr rest)) (res 0))
((null rest) res)
(incf res (- (caar (second rest)) (caar (first rest)))))))
style -> ERROR can be replaced by ASSERT.
possible bug -> SORT is possibly destructive -> make sure you have a fresh list consed!. If it is already fresh allocated by get-planes-including-yz-grid-in, then we don't need that.
bug -> SORT returns a sorted list. The sorted list is possibly not a side-effect. -> use the returned value
style -> DO replaced with LOOP.
style -> meaning of CAAR unclear. Find better naming or use other data structures.
(defun get-x-depth-for-yz-grid (planes//yz-plane grid)
(let ((planes (get-planes-including-yz-grid-in planes//yz-plane grid)))
(assert (evenp (length planes)) (planes)
"error in get-x-depth-for-yz-grid")
(setf planes (sort (copy-list planes) #'< :key #'caar))
(loop for (p1 p2) on planes by #'cddr
sum (- (caar p2) (caar p1)))))
Some documentation makes a bigger improvement than refactoring.
Your -> macro will confuse sbcl’s type inference. You should have (-> x) expand into x, and (-> x y...) into (let (($ x)) (-> y...))
You should learn to use loop and use it in more places. dolist with extra mutation is not great
In a lot of places you should use destructuring-bind instead of eg (rest (rest )). You’re also inconsistent as sometimes you’d write (cddr...) for that instead.
Your block* suffers from many problems:
It uses (let (foo) (setf foo...)) which trips up sbcl type inference.
The name block* implies that the various bindings are scoped in a way that they may refer to those previously defined things but actually all initial value may refer to any variable or function name and if that variable has not been initialised then it evaluates to nil.
The style of defining lots of functions inside another function when they can be outside is more typical of scheme (which has syntax for it) than Common Lisp.
get-x-y-and-z-ranges really needs to use loop. I think it’s wrong too: the lists are different lengths.
You need to define some accessor functions instead of using first, etc. Maybe even a struct(!)
(sort foo) might destroy foo. You need to do (setf foo (sort foo)).
There’s basically no reason to use do. Use loop.
You should probably use :key in a few places.
You write defvar but I think you mean defparameter
*t* is a stupid name
Most names are bad and don’t seem to tell me what is going on.
I may be an idiot but I can’t tell at all what your program is doing. It could probably do with a lot of work
I found a code snippet somewhere online:
(letrec
([id (lambda (v) v)]
[ctx0 (lambda (v) `(k ,v))]
.....
.....
(if (memq ctx (list ctx0 id)) <---- condition always return false
.....
where ctx is also a function:
However I could never make the test-statement return true.
Then I have the following test:
(define ctx0 (lambda (v) `(k ,v)))
(define ctx1 (lambda (v) `(k ,v)))
(eq? ctx0 ctx1)
=> #f
(eqv? ctx0 ctx1)
=> #f
(equal? ctx0 ctx1)
=> #f
Which make me suspect that two function are always different since they have different memory location.
But if functions can be compared against other functions, how can I test if two function are the same? and what if they have different variable name? for example:
(lambda (x) (+ x 1)) and (lambda (y) (+ y 1))
P.S. I use DrRacket to test the code.
You can’t. Functions are treated as opaque values: they are only compared by identity, nothing more. This is by design.
But why? Couldn’t languages implement meaningful ways to compare functions that might sometimes be useful? Well, not really, but sometimes it’s hard to see why without elaboration. Let’s consider your example from your question—these two functions seem equivalent:
(define ctx0 (lambda (v) `(k ,v)))
(define ctx1 (lambda (v) `(k ,v)))
And indeed, they are. But what would comparing these functions for equality accomplish? After all, we could just as easily implement another function:
(define ctx2 (lambda (w) `(k ,w)))
This function is, for all intents and purposes, identical to the previous two, but it would fail a naïve equality check!
In order to decide whether or not two values are equivalent, we must define some algorithm that defines equality. Given the examples I’ve provided thus far, such an algorithm seems obvious: two functions should be considered equal if (and only if) they are α-equivalent. With this in hand, we can now meaningfully check if two functions are equal!
...right?
(define ctx3 (lambda (v) (list 'k v)))
Uh, oh. This function does exactly the same thing, but it’s not implemented exactly the same way, so it fails our equality check. Surely, though, we can fix this. Quasiquotation and using the list constructor are pretty much the same, so we can define them to be equivalent in most circumstances.
(define ctx4 (lambda (v) (reverse (list v 'k))))
Gah! That’s also operationally equivalent, but it still fails our equivalence algorithm. How can we possibly make this work?
Turns out we can’t, really. Functions are units of abstraction—by their nature, we are not supposed to need to know how they are implemented, only what they do. This means that function equality can really only be correctly defined in terms of operational equivalence; that is, the implementation doesn’t matter, only the behavior does.
This is an undecidable problem in any nontrivial language. It’s impossible to determine if any two functions are operationally equivalent because, if we could, we could solve the halting problem.
Programming languages could theoretically provide a best-effort algorithm to determine function equivalency, perhaps using α-equivalency or some other sort of metric. Unfortunately, this really wouldn’t be useful—depending on the implementation of a function rather than its behavior to determine the semantics of a program breaks a fundamental law of functional abstraction, and as such any program that depended on such a system would be an antipattern.
Function equality is a very tempting problem to want to solve when the simple cases seem so easy, but most languages take the right approach and don’t even try. That’s not to say it isn’t a useful idea: if it were possible, it would be incredibly useful! But since it isn’t, you’ll have to use a different tool for the job.
Semantically, two function f and g are equal if they agree for every input, i.e. if for all x, we have (= (f x) (g x)). Of course, there's no way to test that for every possible value of x.
If all you want to do is be reasonably confident that (lambda (x) (+ x 1)) and (lambda (y) (+ y 1)) are the same, then you might try asserting that
(map (lambda (x) (+ x 1)) [(-5) (-4) (-3) (-2) (-1) 0 1 2 3 4 5])
and
(map (lambda (y) (+ y 1)) [(-5) (-4) (-3) (-2) (-1) 0 1 2 3 4 5])
are the same in your unit tests.
In common lisp there is map, which lets you do this kind of thing:
(map (lambda (x y) (/ x y)) (list 2 4 6 8 10 12) (list 1 2 3 4 5 6))
returning (2 2 2 2 2 2)
However now I am working at ACL2 and there is no such a thing as map.
So in my mind the only choice left I have is doing recursion to calculate what I want, unless there is another simpler and/or more efficient way of doing it.
... Which is exactly my question. Is there a better way of doing it than to create a recursive function called something like divide-two-lists? It just feels like something that a lisp-based language should naturally do instead of having you to create another function specifically just for it, hence why I am asking.
You could pretty easily write your own map. From the GNU Emacs guide:
(defun mapcar* (function &rest args)
"Apply FUNCTION to successive cars of all ARGS.
Return the list of results."
;; If no list is exhausted,
(if (not (memq nil args))
;; apply function to cars.
(cons (apply function (mapcar 'car args))
(apply 'mapcar* function
;; Recurse for rest of elements.
(mapcar 'cdr args)))))
(mapcar* 'cons '(a b c) '(1 2 3 4))
⇒ ((a . 1) (b . 2) (c . 3))
I'm unfamiliar with acl2, so you might have to change some functions (e.g. memq), or deal differently with how apply or &rest arguments work, but this is the meat of the code.
ACL2 is based on first order logic. In first order logic, statements like
(define (P R A) (R A))
are not allowed because R is being used as both a parameter and a function.
It is theoretically possible to get around this limitation by literally defining your own language within first order logic that includes the constructs for higher order logic. Otherwise, you are correct, your best option is to define something like divide-two-lists every single time you want to use a map function.
That's tedious, but it is how ACL2 was meant to be used.
This isn't exactly suitable to your question, but it's related, and so I mention it in case it helps someone else who is looking at your question.
Consider the book "std/util/defprojection", which provides a macro that lets you map a function across a list.