My goal is a function which can call itself again indefinitely upon
encountering an error.
I am describing different approaches I tried based on the Common Lisp HyperSpec and would appreciate if someone could reveal the secrets of why
they act as they do.
I'm using SBCL 1.3.8 with enabled tail call optimization and verified that it is working properly on a simple tail recursive function.
unwind-protect
With the first approach I tried, m0 gets called twice. Once as a result of the original call and once as part of the cleanup form in the unwind-protect.
After encountering the error in the second body, it does not execute the cleanup form properly.
I would have expected for the function to call itself over and over again, and to run into a stack overflow or for SBCL to be able to recognize the call as a tail call and to optimize it.
(defun m0 ()
(unwind-protect
(progn
(write-line "body")
(error "error"))
(write-line "cleanup")
(m0)))
(m0)
Intrigued by the result, I investigated whether it was an occurrence with nested unwind-protects in general, and it seems to be. The following program displays the same behavior:
(unwind-protect
(progn
(write-line "body 0")
(error "error 0"))
(unwind-protect
(progn
(write-line "body 1")
(error "error 1"))
(write-line "body 2")
(error "error 2"))))
Is this behavior related to the extent of the exit of the inner unwind-protect?
Is there a way to get it to work and especially a way which supports tail call elimination?
Why can the unwind-protects not be nested arbitrarily?
handler-case
The second approach I tried runs into a stack overflow. This is not as surprising as the result of the first approach, but without knowing the inner details of the condition system, I would have expected the function to be tail recursive and therefore I would have expected for SBCL to optimize the tail call.
(define-condition m-error () nil)
(defun m1 ()
(handler-case
(progn (write-line "body")
(error 'm-error))
(m-error ()
(progn (write-line "cleanup")
(m1)))))
(m1)
Is there a way in which the function could be slightly modified to ensure that tail call elimination will occur?
handler-bind
Throws an error due to reaching the maximum-error-depth defined for the runtime environment.
I would have expected this to perform roughly equal to the handler-case solution. The stack is not unwound before executing the cleanup forms in this case due to the different behavior of handler-case and handler-bind, but I still would have expected for the call to m to be recognized as a tail call and to be optimized in the grand scheme of things.
(defun m2 ()
(handler-bind
((m-error #'(lambda (c)
(progn (write-line "cleanup")
(m2)))))
(write-line "body")
(error 'm-error)))
(m2)
The question related to m1 applies here, too.
I would like to know why these cases do not work as I expected them to work, based on the documentation. The people in #lisp on freenode were also puzzled by this behavior.
And if there is no way in which these examples can be fixed, then I would appreciate a pointer to some construct with which this behavior could be implemented, without returning control to a higher level.
Firstly, there is no guarantee that this is possible at all: CL the language is not specified to be tail-recursive at all, and thus it is entirely up to implementations both as to whether they optimise tail calls and, when they do, what is in tail position with respect to what.
Secondly, your first, unwind-protect implementation probably does not do what you think it does, and neither does your third. In the case of the third implementation your handler fails to handle the error which essentially means that there is no hope of the code being tail-recursive, since the handler must remain on the stack until it either returns normally or handles the error, neither of which it does.
The handler-bind implementation
As I think handler-bind is not widely understood, here is a version of your third implementation which might stand a chance of being tail-recursive: the handler does handle the error, and then the code it jumps to recurses.
(define-condition m-error ()
())
(defun m4 ()
(let* ((errored nil)
(result
(block escape
(handler-bind ((m-error
#'(lambda (c)
(declare (ignorable c))
(setf errored t)
(return-from escape nil))))
(error 'm-error)))))
(if (not errored)
result
(m4))))
However, in neither of the implementations to which I have immediate access (LW and CCL) will this easily compile as a tail call to m4 (both implementations do optimise tail calls).
I also tried a more horrible but explicit version of this solution:
(defun m5 ()
(tagbody
(return-from m5
(handler-bind ((m-error
#'(lambda (c)
(declare (ignorable c))
(go recurse))))
(error 'm-error)))
recurse
(m5)))
And I can't get either implication to compile the recursive call to m5 as a tail call. Probably to understand why they won't would require looking at the assembler.
The unwind-protect implementation
It's not clear to me that this can work. In particular, remember that
unwind-protect evaluates protected-form and guarantees that cleanup-forms are executed before unwind-protect exits, whether it terminates normally or is aborted by a control transfer of some kind.
(From the CLHS.)
So any code which looks like
(defun m6 ()
(unwind-protect
...any form...
(m6)))
is going to call itself recursively whatever happens. In particular it will almost certainly do so when you exit the debugger after any error in ...any form..., will certainly do so if there is no error in ...any form..., so long as it terminates, and it may very well try to call itself when you exit the Lisp implementation itself. Indeed this function may make it reasonably hard to regain control: it is not at all obvious that it terminates or that it is easily possible to force it to do so, even by interrupting evaluation.
Something like the following gives you more chance of escape:
(defun m7 ()
(let ((errored nil))
(unwind-protect
(handler-case
(error 'm-error)
(m-error ()
(setf errored t)))
(when errored
(m7)))))
A deeply horrid implementation
Real Programmers (who are correctly known as REAL PROGRAMMERS) would of course write the following version, which avoids having to worry about all this hipster 'tail recursion' nonsense:
(defun m8 ()
(tagbody
loop
(return-from m8
(handler-bind ((m-error
#'(lambda (c)
(declare (ignorable c))
(go loop))))
(error 'm-error)))))
(except they would write it in UPPERCASE).
Related
With my recursive function, I have a cond with a fail conditon.
When the concequence of the fail condition is nil, everything works fine.
But when I try to change the concequence of the fail condition to something else:
(defun rec ()
(cond ((null t) (format t ....))
....)
it says that the (format t....) function is unreachable... why?
In Common Lisp, t is a boolean constant that means “true”. So, the expression (null t) checks if t is the empty list and this is obviously always false. So the compiler is smart enough to infer that the code (format ... is never reached and does not compile it.
Errors like this arise sometimes when one forgets the meaning of t, and tries to use it as a variable.
This question is somewhat related to an earlier one on programmatically generating symbol macros. I'm using that function in a convenience macro that throws undefined variable warnings. This macro and function:
(defmacro define-data (d body &optional doc)
(if (and doc (not (stringp doc))) (error "Documentation is not a string"))
`(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
;; Should we have an eval-when (:compile-toplevel) here?
(defparameter ,d ,body ,doc)
(export ',d old-package)
(define-column-names ,d)))
(defun define-column-names (d)
(maphash #'(lambda (key index)
(eval `(cl:define-symbol-macro ,key (cl:aref (columns ,d) ,index))))
(ordered-keys-table (slot-value d 'ordered-keys))))
are intended to be like defparameter, but additionally set up a few niceties for the user by defining:
a package with the name of d
a parameter in the current package with the data that will be sucked in by body
symbol-macros in package d for access to the individual data vectors
If I use defparameter from the REPL, and then call define-column-names, all is well. However when using the macro I get:
; in: DEFINE-COLUMN-NAMES FOO
; (DEFINE-COLUMN-NAMES CL-USER::FOO)
;
; caught WARNING:
; undefined variable: CL-USER::FOO
I suspect that this is because the compiler has no way of knowing that FOO will actually be defined when define-symbol-macro is called. Everything works fine, but I don't want the warning to frighten users, so am thinking of suppressing it. I hate suppressing warnings though, so thought I'd come here for a second opinion.
EDIT: I've marked an answer correct because it does correctly answer the question as asked. For an answer to the problem see my comments.
My answer to the 'when to muffle warnings' question in the title is: if it's your own code then never, under any circumstances. If it is someone else's code, then rewrite it not to warn unless you can't.
As to solving the problem I haven't thought about this hard enough, but the problem is that you definitely want the defparameter to be at top-level so the compiler can see it, and it can't really be if it's inside a let. But you can raise it to toplevel trivially since it depends on nothing inside the let.
I am then pretty certain that you want the rest of the macro to happen at compile time, because you definitely want the symbol-macros available at compile-time. So an attempt at the first macro would be (note I've fixed the handling of the docstring: (defparameter foo 1 nil) is bad):
(defmacro define-data (d body &optional doc)
(when (and doc (not (stringp doc)))
(error "Documentation is not a string"))
`(progn
(defparameter ,d ,body ,#(if doc (list doc) '()))
(eval-when (:compile-toplevel :load-toplevel :execute)
(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
(export ',d old-package)
(define-column-names ,d)))))
As a side note: although I think the fact that programmatically defining symbol macros is hard because CL left that out for some reason, I think I'd personally use some other approach rather than this, because eval is just so horrid. That's just me however: if you want to do this you do need eval I think (it is very rare that this is true!).
I am not sure exactly how define-columns-names works so I replaced it with a stub function that returns d.
Note also that you can use check-type and should try not injecting symbols in generated code, this introduces potential variable capture that can be avoided with gensym.
As far as I know you cannot use eval-when as suggested by your comment (see Issue EVAL-WHEN-NON-TOP-LEVEL Writeup for details).
But I have no warning if I declare the symbol as being special around the call.
(defmacro define-data (d body &optional doc)
(check-type doc (or null string))
(check-type d symbol)
(let ((d-str (string d)))
(alexandria:with-gensyms (old-package)
`(let* ((,old-package *package*)
(*package* (if (find-package ,d-str) ;exists?
(find-package ,d-str) ;yes, return it
(make-package ,d-str)))) ;no, make it
(defparameter ,d ,body ,doc)
(export ',d ,old-package)
(locally (declare (special ,d))
(define-column-names ,d))))))
It is also a bit strange that you expand into a call to define-column-names, which in turns evaluated a form built at runtime. I think it might be possible to do all you want during macroexpansion time, but as said earlier what you are trying to do is a bit unclear to me. What I have in mind is to replace define-column-names by:
,#(expand-column-names-macros d)
... where expand-column-names-macros builds a list of define-symbol-macro forms.
SBCL generates spurious style warnings about undefined functions. (The functions are defined, just later in the file.) I want to solve this problem once and for all. Fortunately, there is a way to do this:
(declaim (sb-ext:muffle-conditions style-warning))
The downside is that CCL, for obvious reasons, barfs on a program containing the above. I try to solve this problem with a conditional:
(#+sbcl (declaim (sb-ext:muffle-conditions style-warning)))
but now SBCL is unhappy: "illegal function call".
How do you put such a declaim into a portable program?
Note that while the existing answer is right, disabling warnings is not a good practice. In your case, it is probably not necessary.
Common Lisp has a notion of compilation unit, where multiple definitions are grouped together. This gives a chance for the compiler/interpreter to take care of cross-references among functions (an interpreter could collect warnings and keep only those that are not found later, for example).
For example, in file #P"/tmp/foo.pl":
(defun mut-rec-foo (x)
(when (plusp x)
(mut-rec-bar (1- x))))
(defun mut-rec-bar (x)
(print x)
(mut-rec-foo (1- x)))
Do not evaluate anything in the file; instead do:
(compile-file #P"/tmp/foo.pl")
; compiling (DEFUN MUT-REC-FOO ...)
; compiling (DEFUN MUT-REC-BAR ...)
; /tmp/foo.fasl written
; compilation finished in 0:00:00.002
No warning. You can then call (load #P"/tmp/foo.fasl") to have the definitions in your current lisp environment, without warnings.
Typically, ASDF and by extension Quicklisp use COMPILE-FILE, so your problem should disappear as soon as you bundle your files into a system.
You can also do:
(with-compilation-unit ()
(defun mut-rec-foo/bis (x)
(when (plusp x)
(mut-rec-bar/bis (1- x))))
(defun mut-rec-bar/bis (x)
(print x)
(mut-rec-foo/bis (1- x))))
Evaluating the whole block shows no warning for *EVALUATOR-MODE* being both :COMPILE or :INTERPRET.
What you witnessed happens when you evaluate each expression one after the other (or maybe one region after another one). There, the compiler has no way to know that the function already exists. Silencing the warning is the worse option, because you might actually have made an error.
If you know in advance that a function will exist, but not in your compilation unit (maybe it is only defined at runtime), the you can declaim that fact, as follows:
(declaim (ftype function my-function))
The above says that my-function must be assumed to be fbound to an object of type function. You could also give more information by refining what kind of function you claim it to be:
(declaim (ftype (function (number) (values string &optional)) num-to-string))
... for a function that accepts a number and returns exactly one value, a string.
(declaim (ftype (function () nil) forever-loop))
... for a function that accepts nothing and never return a value (loop or signals an error).
Omit the outer pair of parentheses:
#+sbcl (declaim (sb-ext:muffle-conditions style-warning))
As you are using declaim, I assume, that the declaration appears at the top-level of a compilation unit. If you need to group multiple top-level statements, you can wrap them all with a progn (which doesn't change the "top-level"-ness).
The reason SBCL did complain is, that its reader reads
((declaim (sb-ext:muffle-conditions style-warning)))
(as the :SBCL feature is present), which is simply a syntax error. CCL does not complain, because its reader reads
()
which is simply another way to spell nil.
So consider the following code:
(define-condition some-condition (error) nil)
(defmethod print-object ((obj some-condition) stream)
(format stream "HELLO THERE"))
(defmacro error-report-test-aux (fn-to-cause-error error-type-to-catch fn-to-handle-error expected-message)
`(let ((result-message
(handler-case (funcall ,fn-to-cause-error)
(,error-type-to-catch (e) (funcall ,fn-to-handle-error e)))))
(assert (string= result-message
,expected-message))
t))
I can use it like so:
(error-report-test-aux (lambda () (error 'some-condition))
some-condition
#'princ-to-string
"HELLO THERE")
But I wanted to make error-report-test-aux a function instead of macro, so that i can pass to it a type of condition within a variable.
To simply write defun instead of defmacro and remove backquote and commas doesn't work because handler-case is macro and it doesn't evaluate error-type-to-catch.
My question is: Is there something like handler-case that would evaluate it's arguments (specifically condition type argument)?
Yes and no :-)
No to your exact question
There is no standard function that would do what you want because catching errors requires establishing bindings and one usually want to bind constant symbols (like in let/let*) because it is easier to optimize.
You might consider creating a "universal" handler using handler-bind and then declining to handle "uninteresting" conditions (as suggested by #jkiiski in comments), but I am not sure if that fits your exact requirements (untested!):
(defun error-report-test-aux (fn-to-cause-error error-type-to-catch expected-message)
(catch 'trap
(handler-bind ((error
(lambda (condition)
(when (typep condition error-type-to-catch)
(throw 'trap (string= (princ-to-string condition)
expected-message))))))
(funcall fn-to-cause-error))))
Yes, implementation-specific
IF your implementation implements handler-case/handler-bind by binding an internal global variable, you can use progv to bind it yourself and thus implement your error-report-test-aux as a function.
This is probably not the best idea (your code becomes wedded to a specific implementation).
Yes, kinda
You can use the fact that some-condition names a CLOS class and use generic functions instead of the macro.
When I am using SBCL (my favourite implementation) I often see this sort of error message in the REPL
CL-USER> (vgplot:fred) ; Evaluation aborted on
#<SB-INT:SIMPLE-READER-PACKAGE-ERROR "Symbol ~S not found in the ~A package." {1004289123}>. CL-USER>
I presume the message is meant to say "Symbol FRED not found in the VGPLOT package."
I imagine I have done something wrong in building SBCL (though I can't seem to find what exactly) , just wondering if anyone can offer advice on how to make this go away as it is annoying.
You could do this:
(defmethod print-object :around ((condition simple-condition) stream)
(print-unreadable-object (condition stream)
(apply #'format stream (simple-condition-format-control condition)
(simple-condition-format-arguments condition))
(terpri stream)
(call-next-method)))
This will add the formatted description of all simple-conditions (of which sb-int:simple-reader-error is one) to their printed form. I wouldn't really consider this a "good idea" and it might have some unintended consequences.
A much better solution would be to modify slime/swank to print this information itself, but I couldn't figure out how to do it.