With my recursive function, I have a cond with a fail conditon.
When the concequence of the fail condition is nil, everything works fine.
But when I try to change the concequence of the fail condition to something else:
(defun rec ()
(cond ((null t) (format t ....))
....)
it says that the (format t....) function is unreachable... why?
In Common Lisp, t is a boolean constant that means “true”. So, the expression (null t) checks if t is the empty list and this is obviously always false. So the compiler is smart enough to infer that the code (format ... is never reached and does not compile it.
Errors like this arise sometimes when one forgets the meaning of t, and tries to use it as a variable.
Related
This question is somewhat related to an earlier one on programmatically generating symbol macros. I'm using that function in a convenience macro that throws undefined variable warnings. This macro and function:
(defmacro define-data (d body &optional doc)
(if (and doc (not (stringp doc))) (error "Documentation is not a string"))
`(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
;; Should we have an eval-when (:compile-toplevel) here?
(defparameter ,d ,body ,doc)
(export ',d old-package)
(define-column-names ,d)))
(defun define-column-names (d)
(maphash #'(lambda (key index)
(eval `(cl:define-symbol-macro ,key (cl:aref (columns ,d) ,index))))
(ordered-keys-table (slot-value d 'ordered-keys))))
are intended to be like defparameter, but additionally set up a few niceties for the user by defining:
a package with the name of d
a parameter in the current package with the data that will be sucked in by body
symbol-macros in package d for access to the individual data vectors
If I use defparameter from the REPL, and then call define-column-names, all is well. However when using the macro I get:
; in: DEFINE-COLUMN-NAMES FOO
; (DEFINE-COLUMN-NAMES CL-USER::FOO)
;
; caught WARNING:
; undefined variable: CL-USER::FOO
I suspect that this is because the compiler has no way of knowing that FOO will actually be defined when define-symbol-macro is called. Everything works fine, but I don't want the warning to frighten users, so am thinking of suppressing it. I hate suppressing warnings though, so thought I'd come here for a second opinion.
EDIT: I've marked an answer correct because it does correctly answer the question as asked. For an answer to the problem see my comments.
My answer to the 'when to muffle warnings' question in the title is: if it's your own code then never, under any circumstances. If it is someone else's code, then rewrite it not to warn unless you can't.
As to solving the problem I haven't thought about this hard enough, but the problem is that you definitely want the defparameter to be at top-level so the compiler can see it, and it can't really be if it's inside a let. But you can raise it to toplevel trivially since it depends on nothing inside the let.
I am then pretty certain that you want the rest of the macro to happen at compile time, because you definitely want the symbol-macros available at compile-time. So an attempt at the first macro would be (note I've fixed the handling of the docstring: (defparameter foo 1 nil) is bad):
(defmacro define-data (d body &optional doc)
(when (and doc (not (stringp doc)))
(error "Documentation is not a string"))
`(progn
(defparameter ,d ,body ,#(if doc (list doc) '()))
(eval-when (:compile-toplevel :load-toplevel :execute)
(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
(export ',d old-package)
(define-column-names ,d)))))
As a side note: although I think the fact that programmatically defining symbol macros is hard because CL left that out for some reason, I think I'd personally use some other approach rather than this, because eval is just so horrid. That's just me however: if you want to do this you do need eval I think (it is very rare that this is true!).
I am not sure exactly how define-columns-names works so I replaced it with a stub function that returns d.
Note also that you can use check-type and should try not injecting symbols in generated code, this introduces potential variable capture that can be avoided with gensym.
As far as I know you cannot use eval-when as suggested by your comment (see Issue EVAL-WHEN-NON-TOP-LEVEL Writeup for details).
But I have no warning if I declare the symbol as being special around the call.
(defmacro define-data (d body &optional doc)
(check-type doc (or null string))
(check-type d symbol)
(let ((d-str (string d)))
(alexandria:with-gensyms (old-package)
`(let* ((,old-package *package*)
(*package* (if (find-package ,d-str) ;exists?
(find-package ,d-str) ;yes, return it
(make-package ,d-str)))) ;no, make it
(defparameter ,d ,body ,doc)
(export ',d ,old-package)
(locally (declare (special ,d))
(define-column-names ,d))))))
It is also a bit strange that you expand into a call to define-column-names, which in turns evaluated a form built at runtime. I think it might be possible to do all you want during macroexpansion time, but as said earlier what you are trying to do is a bit unclear to me. What I have in mind is to replace define-column-names by:
,#(expand-column-names-macros d)
... where expand-column-names-macros builds a list of define-symbol-macro forms.
SBCL generates spurious style warnings about undefined functions. (The functions are defined, just later in the file.) I want to solve this problem once and for all. Fortunately, there is a way to do this:
(declaim (sb-ext:muffle-conditions style-warning))
The downside is that CCL, for obvious reasons, barfs on a program containing the above. I try to solve this problem with a conditional:
(#+sbcl (declaim (sb-ext:muffle-conditions style-warning)))
but now SBCL is unhappy: "illegal function call".
How do you put such a declaim into a portable program?
Note that while the existing answer is right, disabling warnings is not a good practice. In your case, it is probably not necessary.
Common Lisp has a notion of compilation unit, where multiple definitions are grouped together. This gives a chance for the compiler/interpreter to take care of cross-references among functions (an interpreter could collect warnings and keep only those that are not found later, for example).
For example, in file #P"/tmp/foo.pl":
(defun mut-rec-foo (x)
(when (plusp x)
(mut-rec-bar (1- x))))
(defun mut-rec-bar (x)
(print x)
(mut-rec-foo (1- x)))
Do not evaluate anything in the file; instead do:
(compile-file #P"/tmp/foo.pl")
; compiling (DEFUN MUT-REC-FOO ...)
; compiling (DEFUN MUT-REC-BAR ...)
; /tmp/foo.fasl written
; compilation finished in 0:00:00.002
No warning. You can then call (load #P"/tmp/foo.fasl") to have the definitions in your current lisp environment, without warnings.
Typically, ASDF and by extension Quicklisp use COMPILE-FILE, so your problem should disappear as soon as you bundle your files into a system.
You can also do:
(with-compilation-unit ()
(defun mut-rec-foo/bis (x)
(when (plusp x)
(mut-rec-bar/bis (1- x))))
(defun mut-rec-bar/bis (x)
(print x)
(mut-rec-foo/bis (1- x))))
Evaluating the whole block shows no warning for *EVALUATOR-MODE* being both :COMPILE or :INTERPRET.
What you witnessed happens when you evaluate each expression one after the other (or maybe one region after another one). There, the compiler has no way to know that the function already exists. Silencing the warning is the worse option, because you might actually have made an error.
If you know in advance that a function will exist, but not in your compilation unit (maybe it is only defined at runtime), the you can declaim that fact, as follows:
(declaim (ftype function my-function))
The above says that my-function must be assumed to be fbound to an object of type function. You could also give more information by refining what kind of function you claim it to be:
(declaim (ftype (function (number) (values string &optional)) num-to-string))
... for a function that accepts a number and returns exactly one value, a string.
(declaim (ftype (function () nil) forever-loop))
... for a function that accepts nothing and never return a value (loop or signals an error).
Omit the outer pair of parentheses:
#+sbcl (declaim (sb-ext:muffle-conditions style-warning))
As you are using declaim, I assume, that the declaration appears at the top-level of a compilation unit. If you need to group multiple top-level statements, you can wrap them all with a progn (which doesn't change the "top-level"-ness).
The reason SBCL did complain is, that its reader reads
((declaim (sb-ext:muffle-conditions style-warning)))
(as the :SBCL feature is present), which is simply a syntax error. CCL does not complain, because its reader reads
()
which is simply another way to spell nil.
I've been trying to write a recursive loop in clojure that will print me out the very last number in the list. The point is not that I need to get the last number (for which I'm sure there's a built in function for that) but that I want to better understand recursion and macros in clojure. So I have this macro...
(defmacro loop-do [the-list]
`(if (= (count '~the-list) 1)
(println (first '~the-list))
(loop-do (rest '~the-list))))
But I get a stackoverflow error. What am I doing wrong?
How will people use your macro?
Somewhere, someone will call:
(loop-do list)
As a piece of code, those are only two symbols in a list. The first one is recognized as your macro, and the second one, list, is a symbol that represents a variable that will be bound at runtime. But your macro only knows that this is a symbol.
The same goes for:
(loop-do (compute-something))
The argument is a form, but you do not want to get the last element of that form, only the last element of the list obtained after evaluating the code.
So: you only know that in your macro, the-list will be bound to an expression that, at runtime, will have to be a list. You cannot use the-list as-if it was a list itself: neither (count 'list) nor (count '(compute-something)) does what you want.
You could expand into (count list) or (count (compute-something)), though, but the result would only be computed at runtime. The job of the macro is only to produce code.
Recursive macros
Macros are not recursive: they expand into recursive calls.
(and a b c)
might expand as:
(let [a0 a] (if a0 a0 (and b c)))
The macroexpansion process is a fixpoint that should terminate, but the macro does not call itself (what would that mean, would you expand the code while defining the macro?). A macro that is "recursive" as-in "expands into recursive invocations" should have a base case where it does not expand into itself (independently of what will, or will not, happen at runtime).
(loop-do x)
... will be replaced by:
(loop-do (rest 'x))
... and that will be expanded again.
That's why the comments say the size actually grows, and that's why you have a stackoverflow error: macroexpansion never finds a fixpoint.
Debugging macros
You have a stackoverflow error. How do you debug that?
Use macroexpand-1, which only performs one pass of macroexpansion:
(macroexpand-1 '(loop-do x))
=> (if (clojure.core/= (clojure.core/count (quote x)) 1)
(clojure.core/println (clojure.core/first (quote x)))
(user/loop-do (clojure.core/rest (quote x))))
You can see that the generated code still contains a call to usr/loop-do , but that the argument is (clojure.core/rest (quote x)). That's the symptom you should be looking for.
Consider this question. Here the basic problem is the code:
(progv '(op arg) '(1+ 1)
(eval '(op arg)))
The problem here is that progv binds the value to the variable as variable's symbol-value not symbol-function. But, that's obvious because we didn't explicitly suggest which values are functions.
The Plan
So, in order to solve this problem, I thought of manually dynamically binding the variables, to their values based on the type of values. If the values are fboundp then they should be bound to the symbol-function of the variable. A restriction, is that match-if can't be a macro. It has to be a function, because it is called by a funcall.
Macro : functioner:
(defmacro functioner (var val)
`(if (and (symbolp ',val)
(fboundp ',val))
(setf (symbol-function ',var) #',val)
(setf ,var ,val)))
Function: match-if:
(defun match-if (pattern input bindings)
(eval `(and (let ,(mapcar #'(lambda (x) (list (car x))) bindings)
(declare (special ,# (mapcar #'car bindings)))
(loop for i in ',bindings
do (eval `(functioner ,(first i) ,(rest i))))
(eval (second (first ,pattern))))
(pat-match (rest ,pattern) ,input ,bindings))))
Here, the let part declares all the variables lexically (supposedly). Then declare declares them special. Then functioner binds the variables and their values aptly. Then the code in the pattern is evaluated. If the code part is true, then only the pattern-matcher function pat-match is invoked.
The Problem
The problem is that in the function, all it's arguments are evaluated. Thus bindings in the let and declare parts will be replaced by something like :
((v1 . val1)(v2 . val2)(v3 . val3))
not
'((v1 . val1)(v2 . val2)(v3 . val3))
So, it's treated as code, not a list.
So, I'm stuck here. And macros won't help me on this one.
Any help appreciated.
Not the answer you are looking for, but PROGV is a special operator; it is granted the ability to modify the dynamic bindings of variables at runtime; AFAIK, you can't simply hack it to operate on "dynamic function bindings".
The point of progv is to use list of symbols and values that are evaluated, meaning that you can generate symbols at runtime and bind them dynamically to the corresponding values.
You might be able to find a solution with eval but note that if you macroexpand into (eval ...), then you loose the surrounding lexical context, which is generally not what you want ("eval" operates on the null lexical environment). I speculate that you could also have a custom code walker which works on top-level forms but reorganizes them, when it finds your special operator, to bring the context back in, producing something like (eval '(let (...) ...)).
My goal is a function which can call itself again indefinitely upon
encountering an error.
I am describing different approaches I tried based on the Common Lisp HyperSpec and would appreciate if someone could reveal the secrets of why
they act as they do.
I'm using SBCL 1.3.8 with enabled tail call optimization and verified that it is working properly on a simple tail recursive function.
unwind-protect
With the first approach I tried, m0 gets called twice. Once as a result of the original call and once as part of the cleanup form in the unwind-protect.
After encountering the error in the second body, it does not execute the cleanup form properly.
I would have expected for the function to call itself over and over again, and to run into a stack overflow or for SBCL to be able to recognize the call as a tail call and to optimize it.
(defun m0 ()
(unwind-protect
(progn
(write-line "body")
(error "error"))
(write-line "cleanup")
(m0)))
(m0)
Intrigued by the result, I investigated whether it was an occurrence with nested unwind-protects in general, and it seems to be. The following program displays the same behavior:
(unwind-protect
(progn
(write-line "body 0")
(error "error 0"))
(unwind-protect
(progn
(write-line "body 1")
(error "error 1"))
(write-line "body 2")
(error "error 2"))))
Is this behavior related to the extent of the exit of the inner unwind-protect?
Is there a way to get it to work and especially a way which supports tail call elimination?
Why can the unwind-protects not be nested arbitrarily?
handler-case
The second approach I tried runs into a stack overflow. This is not as surprising as the result of the first approach, but without knowing the inner details of the condition system, I would have expected the function to be tail recursive and therefore I would have expected for SBCL to optimize the tail call.
(define-condition m-error () nil)
(defun m1 ()
(handler-case
(progn (write-line "body")
(error 'm-error))
(m-error ()
(progn (write-line "cleanup")
(m1)))))
(m1)
Is there a way in which the function could be slightly modified to ensure that tail call elimination will occur?
handler-bind
Throws an error due to reaching the maximum-error-depth defined for the runtime environment.
I would have expected this to perform roughly equal to the handler-case solution. The stack is not unwound before executing the cleanup forms in this case due to the different behavior of handler-case and handler-bind, but I still would have expected for the call to m to be recognized as a tail call and to be optimized in the grand scheme of things.
(defun m2 ()
(handler-bind
((m-error #'(lambda (c)
(progn (write-line "cleanup")
(m2)))))
(write-line "body")
(error 'm-error)))
(m2)
The question related to m1 applies here, too.
I would like to know why these cases do not work as I expected them to work, based on the documentation. The people in #lisp on freenode were also puzzled by this behavior.
And if there is no way in which these examples can be fixed, then I would appreciate a pointer to some construct with which this behavior could be implemented, without returning control to a higher level.
Firstly, there is no guarantee that this is possible at all: CL the language is not specified to be tail-recursive at all, and thus it is entirely up to implementations both as to whether they optimise tail calls and, when they do, what is in tail position with respect to what.
Secondly, your first, unwind-protect implementation probably does not do what you think it does, and neither does your third. In the case of the third implementation your handler fails to handle the error which essentially means that there is no hope of the code being tail-recursive, since the handler must remain on the stack until it either returns normally or handles the error, neither of which it does.
The handler-bind implementation
As I think handler-bind is not widely understood, here is a version of your third implementation which might stand a chance of being tail-recursive: the handler does handle the error, and then the code it jumps to recurses.
(define-condition m-error ()
())
(defun m4 ()
(let* ((errored nil)
(result
(block escape
(handler-bind ((m-error
#'(lambda (c)
(declare (ignorable c))
(setf errored t)
(return-from escape nil))))
(error 'm-error)))))
(if (not errored)
result
(m4))))
However, in neither of the implementations to which I have immediate access (LW and CCL) will this easily compile as a tail call to m4 (both implementations do optimise tail calls).
I also tried a more horrible but explicit version of this solution:
(defun m5 ()
(tagbody
(return-from m5
(handler-bind ((m-error
#'(lambda (c)
(declare (ignorable c))
(go recurse))))
(error 'm-error)))
recurse
(m5)))
And I can't get either implication to compile the recursive call to m5 as a tail call. Probably to understand why they won't would require looking at the assembler.
The unwind-protect implementation
It's not clear to me that this can work. In particular, remember that
unwind-protect evaluates protected-form and guarantees that cleanup-forms are executed before unwind-protect exits, whether it terminates normally or is aborted by a control transfer of some kind.
(From the CLHS.)
So any code which looks like
(defun m6 ()
(unwind-protect
...any form...
(m6)))
is going to call itself recursively whatever happens. In particular it will almost certainly do so when you exit the debugger after any error in ...any form..., will certainly do so if there is no error in ...any form..., so long as it terminates, and it may very well try to call itself when you exit the Lisp implementation itself. Indeed this function may make it reasonably hard to regain control: it is not at all obvious that it terminates or that it is easily possible to force it to do so, even by interrupting evaluation.
Something like the following gives you more chance of escape:
(defun m7 ()
(let ((errored nil))
(unwind-protect
(handler-case
(error 'm-error)
(m-error ()
(setf errored t)))
(when errored
(m7)))))
A deeply horrid implementation
Real Programmers (who are correctly known as REAL PROGRAMMERS) would of course write the following version, which avoids having to worry about all this hipster 'tail recursion' nonsense:
(defun m8 ()
(tagbody
loop
(return-from m8
(handler-bind ((m-error
#'(lambda (c)
(declare (ignorable c))
(go loop))))
(error 'm-error)))))
(except they would write it in UPPERCASE).