I've been struggling with this for two days now, and I can't find the answer.
What I want is to define three variables, a, b, and c each with a value of 0.
Naive:
(dolist (lbl '(a b c)) (defvar lbl 0))
Doesn't do what I want. a, b, and c remain undefined, and lbl now has a value of 0.
I think I may understand why this can't work: defvar is a macro, not a function, and as such I am passing it the form lbl, not the current value of label (which is a, b, c in turn). I think.
But in the resulting macroexpansion, shouldn't lbl eventually be linked-up(?) or evaluated(?) to the value I'm intending? Obviously not, either because it can't be done or I'm doing it wrong.
I want to understand:
How to make this work: (dolist (lbl '(a b c)) (defvar lbl 0))
What's going wrong under the hood. I have a feeling it has something to do with symbols or the mechanics of the quote operator.
Here are a few options:
With eval, by building a defvar expression:
(dolist (lbl '(a b c))
(eval `(defvar ,lbl 0))
With proclaim and setf of symbol-value (note: set is deprecated, since 1994 for what it's worth):
(dolist (lbl '(a b c))
(proclaim `(special ,lbl))
(setf (symbol-value lbl) 0))
This is actually mostly what defvar does (see notes in the linked page), but each Lisp implementation usually also records source file location, as they do for other defining macros.
Under the hood, defvar is a macro that makes the variable special (i.e. with dynamic extent bindings in the current dynamic environment; note: there's no portable undoing for this!), and optionally initializes it if it's not yet bound.
The fact that it's a macro means it doesn't evaluate its arguments, so it can take the variable name literally, and it does so. As such, (defvar lbl 0) will define the variable lbl, not the symbol stored in a lbl variable.
The fact that it optionally initializes the variable means that the initializing expression will not even be evaluated if the variable is boundp. So, its secondary effects won't happen if the variable is already initialized. This might or might not be expected.
Note that this expression isn't actually evaluated at macro-expansion time, it's left for evaluation when the expansion is evaluated, which in a REPL means right after macro expansion (and possibly after compilation, depending on the Lisp implementation; read more about evaluation and compilation, it's quite interesting).
Similar:
(dolist (lbl '(a b c))
(let ((lbl 0))
(print lbl)))
Why is lbl 0 and not some of a, b, c?
Because LET binds the symbol lbl and not its value.
Similar with (DEFVAR FOO 3).
Imagine following code:
(DEFVAR FOO 3)
(LET ((FOO 3)) ...)
Now, if we compile this code, the Lisp compiler recognizes the DEFVAR declaration and now knows that FOO is a special global variable. Thus in the let form FOO will be dynamically bound.
Compare this code:
(dolist (v '(FOO)) (eval `(DEFVAR ,v 3)))
(LET ((FOO 3)) ...)
The compiler won't see the DEFVAR and does not know that it should be a global special variable. In the LET form, FOO will have a lexical binding.
Thus DEFVAR needs to be a macro which knows the symbol at compile time (!) and which expands into a form that informs the compiler that the symbol is a special global variable. The form also sets the value when executed.
Thus the best way to create multiple DEFVAR declarations from a list of variables is to write a macro, which expands into a PROGN form with multiple DEFVARs. Inside the PROGN, the compiler will still recognize them.
CL-USER 21 > (pprint (macroexpand '(defvar* (a b c) 0)))
(PROGN (DEFVAR A 0) (DEFVAR B 0) (DEFVAR C 0))
Implemented as:
(defmacro defvar* (vars initial-value)
`(progn
,#(loop for var in vars
do (check-type var symbol)
collect `(defvar ,var ,initial-value))))
Note that it makes sense to check that the variables are really provided as symbols.
defvar is a special form which makes sure the symbol of it's first argument is a bound variable. If the variable is not bound the evaluated expression of the second argument becomes the bound variables value. Thus:
(defvar *x* 10) ; if *x* was not bound it's now 10
(defvar *x* 20) ; since *x* is defined nothing happens
Notice that *x* is not evaluated but is used unevaluated. In order to get the same functionality by using a variable that evaluates to a symbol which you want to exist as a variable in global scope you need to do something like this:
(defvar b 10)
(dolist (lbl '(a b c))
(when (not (boundp lbl))
(setf (symbol-value lbl) 0)))
Still, neither of the ones not already bound becomes special like with defvar, but at least you get the same behaviour:
(list a b c) ; => (0 10 0)
Perhaps you should just do:
(defvar *a* 0)
(defvar *b* 0)
(defvar *c* 0)
If you have a lot of variables you need to do this with you can do:
(defmacro defvars (lst value)
(loop :for e :in lst
:collect `(defvar ,e ,value) :into result
:finally (return (cons 'progn result))))
(defparameter *w* 10)
(defvars (*q* *w* *e*) 1)
(list *q* *w* *e* ; ==> (1 10 1)
Also, it's really important to earmuff your global variables. Once special it will follow dynamic binding. eg.
(defun test ()
(let ((*b* 15))
(test2)))
(defun test2 ()
*b*)
(test) ; ==> 15
Reimplementing DEFVAR
You can approximate the behavior of defvar with a function like this:
(defun %defvar (symbol value documentation)
"Define a global special variable.
symbol---a symbol
value---nil or a function of zero arguments
documentation---nil or a documentation string
returns symbol
Proclaim SYMBOL globally as a special variable. If VALUE is non-nil,
then if SYMBOL is not already bound, SYMBOL is assigned the value
returned by calling VALUE. DOCUMENATION is assigned as the
documentation of type variable to for SYMBOL."
(prog1 symbol
;; make it globally special
(proclaim (list 'special symbol))
;; if a value is provided, and symbol isn't
;; already bound, set its value to the result
;; of calling the value-function
(when (not (null value))
(unless (boundp symbol)
(setf (symbol-value symbol)
(funcall value))))
;; set the documentation
(setf (documentation symbol 'variable) documentation)))
Then you can do, e.g.,
CL-USER> (%defvar '*the-answer* (lambda () 42) "the answer")
*THE-ANSWER*
CL-USER> *the-answer*
42
CL-USER> (documentation '*the-answer* 'variable)
"the answer"
And with your original code, you could do something like:
(dolist (lbl '(a b c)) (%defvar lbl (lambda () 0)))
Now, how does this relate to what defvar actually does? Well, you could now implement a defvar like macro by doing:
(defmacro define-var (symbol &optional (value nil value-p) documentation)
`(%defvar
',symbol
,(if value-p `(lambda () ,value) 'nil)
,documentation))
This expands as we'd expect:
CL-USER> (macroexpand-1 '(define-var *the-answer* 42 "the answer"))
(%DEFVAR '*THE-ANSWER* (LAMBDA () 42) "the answer")
You can actually use macroexpand to look at what an implementation does, too. E.g., in SBCL:
CL-USER> (macroexpand-1 '(defvar *the-answer* 42 "the answer"))
(PROGN
(EVAL-WHEN (:COMPILE-TOPLEVEL) (SB-IMPL::%COMPILER-DEFVAR '*THE-ANSWER*))
(SB-IMPL::%DEFVAR '*THE-ANSWER* (UNLESS (BOUNDP '*THE-ANSWER*) 42) 'T
"the answer" 'T (SB-C:SOURCE-LOCATION)))
This isn't too much different from what we wrote above, though it's handling the non-evaluation of the form when the variable is already bound in a slightly different way, and it's also got some handling for recording a source location. The general idea is the same, though.
Why things don't get "linked up"
But in the resulting macroexpansion, shouldn't lbl eventually be
linked-up(?) or evaluated(?) to the value I'm intending?
The original code is:
(dolist (lbl '(a b c)) (defvar lbl 0))
We can macroexpand this to see what it becomes (in SBCL):
CL-USER> (macroexpand '(dolist (lbl '(a b c)) (defvar lbl 0)))
(BLOCK NIL
(LET ((#:N-LIST1022 '(A B C)))
(TAGBODY
#:START1023
(UNLESS (ENDP #:N-LIST1022)
(LET ((LBL (TRULY-THE (MEMBER C B A) (CAR #:N-LIST1022))))
(SETQ #:N-LIST1022 (CDR #:N-LIST1022))
(TAGBODY (DEFVAR LBL 0)))
(GO #:START1023))))
NIL)
T
Now, we can still see LBL in two places, including in (defvar LBL 0). So why don't things get "matched up"? To see that, we need to remember that the defvar inside the let will also be macroexpanded. To what? This:
CL-USER> (macroexpand '(DEFVAR LBL 0))
(PROGN
(EVAL-WHEN (:COMPILE-TOPLEVEL) (SB-IMPL::%COMPILER-DEFVAR 'LBL))
(SB-IMPL::%DEFVAR 'LBL (UNLESS (BOUNDP 'LBL) 0) 'T NIL 'NIL
(SB-C:SOURCE-LOCATION)))
But now we see that SBCL's internals are getting the symbol named "LBL"; the call (sb-impl::%defvar 'lbl …) is calling the function sb-impl::%defvar with the symbol lbl, and there's no connection between that symbol and the lexical variable that happens to be represented in the source by the same symbol. After all, if you write:
CL-USER> (let ((a 89))
(list 'a a))
(A 89)
You want to be able to get the symbol a and the number 89, right? The macroexpansion of defvar includes a call to a function with the quotation of one of the arguments to macro.
Related
I am trying to write an fx in lisp to tell if an object ends in nil.
(setq isList (lambda (listOfValues)
(if (null listOfValues) t)
( funcall isList (cdr listOfValues) )
)
)
However, I am having trouble checking if its nil in all cases. In particular, cdr would fail at last elt if it is not a list. How can I resolve this?
Before we get closer to answer your actual question, a few things. First, use defun to define functions, not "set a variable to a lambda", it will make you happier down the line. Second, Common Lisp style would vale been one of values, list-of-values, or just list (that would indicate we knew it was a list, so I would probably just have gone with values here), not "listOfValues" (case is typically smashed, and neither "listofvalues" nor "LISTOFVALUES" are easy to read).
So, back to the code. A list is composed of cons cells, of either atoms or other cons cells. We have two test functions, either consp or atom that would be useful in this case. We know that if we're looking at a cons, we need to recurse on its cdr, otherwise we're at the last element and can just check if we're looking at nil.
(defun is-proper-list (values)
(if (consp values)
(is-proper-list (cdr values))
(null values))) ;; We could do this test as (eql nil) as well
It can be done faster with
(defun listp (l)
(tailp nil l))
(tailp nil ...) tests, whether nil is the end of a given object after cdr-ing to the end.
tailp is a very special function. So don't use it without understanding it.
(tailp '(b c) '(a b c)) is e.g. NOT T, because '(b c) is not the same object like the ( ... b c). But in this case, because NIL is '() and is unique in Lisp, any nil is object-identical. Therefore one can use tailp here for this specific test, whether a given list ends with NIL.
listp is a lisp-convention conform name for this.
(predicate functions returning booleans ending with p for predicate.
Since no - used in the name, attach p without - otherwise attach -p).
(tailp (cdr '(a b c)) '(a b c)) ;; NIL
;; because the two lists are not object-identical
(setq l '(a b c))
(tailp (cddr l) l) ;; T ;; object-identical
I have a code which takes a list and returns all possible permutations by the parameter result.
But when I compile I have an error which says *** - =: (1+ INDEX) is not a number.
Is this message true or I messed up the code generally?
I am new to lisp I can looking for a fix and also open to suggestions from fucntional programmers.
;; Creates permutatiions of a given list and returns it via parameter
(defun create-permuations (source)
(setf result (list))
(create-permuations-helper source 0 '() result)
result)
(defmacro create-permuations-helper (source index cur result)
(if (= (list-length cur) index)
(cons cur result)
(loop for i from 0 to (list-length cur) do
(create-permuations-helper source (1+ index)
(append cur (list (nth i source))) result))))
99% of times when a compiler reports an error you can trust it to be true. Here Index is the list (1+ index), literally the 1+ symbol followed by the index symbol. This is so because you are using a macro, and macros operate on code.
In your macro, you do not return a form to be evaluated, you execute code during macro-expansion that depends on itself. That alone is an undefined behaviour. For example:
(defmacro a (x)
(if (plusp x)
(a (- x 1))
nil))
In the body of a, you want to expand code using a recursive call to itself. But the macro is not yet fully known and cannot be until the whole macro is defined.
Maybe the particular lisp implementation binds a to the macro function in body of the macro, which is a strange thing to do, or you evaluated the definition twice. The first time the compiler assumes a is an unknown function, then binds a to a macro, and the second time it tries to expand the macro.
Anyway macro are not supposed to be recursive.
In the example, since the macro does not evaluate its argument, the nested call to the macro is given the literal expression (- x 1), and not its actual value, which cannot be known anyway since x is unknown. You are crossing a level of abstraction here by trying to evaluate things at macroexpansion time.
But, macros can expand into code that refers to themselves.
(defmacro a (x)
(if (plusp x)
`(b (a ,(- x 1)))
nil))
Now, (a 2) expands into (b (a 1)), which itself macroexpands into (b (b (a 0))), and finally reaches a fixpoint which is (b (b nil)).
The difference is that the macro produces a piece of code and returns, which the compiler macroexpands again, whereas in the first example, the macro must already be expanded in the body of its own definition.
Possible implementation
One way to solve your problem is to define a local function that has access to a variable defined in your main function. Then, the local function can set it, and you do not need to pass a variable by reference (which is not possible to do):
(defun permut (list)
(let (result)
(labels ((recurse (stack list)
(if list
(dolist (x list)
(recurse (cons x stack)
(remove x list :count 1)))
(push stack result))))
(recurse nil list))
result))
Alternatively, you can split the process in two; first, define permut-helper, which is a higher-order function that takes a callback function; it generates permutations and calls the callback for each one:
(defun permut-helper (stack list callback)
(if list
(dolist (x list)
(permut-helper (cons x stack)
(remove x list :count 1)
callback))
(funcall callback stack)))
You call it with a function that pushes results into a list of permutations:
(defun permut (list)
(let (result)
(flet ((add-result (permutation)
(push permutation result)))
(permut-helper nil list #'add-result))
result))
I am wondering how one can achieve the following. Suppose I have a list of variables that are bound by some let above. I would like to turn this list into a list of the values to which those variables are bound.
That is, suppose we have
(define make-plist-from-variables (variables)
(let ((keys variables)
(values (mapcar #'identity variables)))
(if (eq (length keys) (length values))
(make-plist keys values)
nil))))
What can I use in place of #'identity to unpack those values properly?
At the moment, the following call produces the following output.
CL-USER> (let ((a 2) (b 3)) (make-plist-from-variables '(a b)))
(A A B B)
I would like it to be (A 2 B 3)
It needs to be a macro because there is no way to fetch a variable's lexical value based on its symbol.
(defmacro make-plist-from-variables (&rest variables)
(loop :for binding :in variables
:collect `',binding :into result
:collect binding :into result
:finally (return `(list ,#result))))
(macroexpand-1 '(make-plist-from-variables a b))
; ==> (list 'a a 'b b)
(let ((a 2) (b 3))
(make-plist-from-variables a b))
; ==> (a 2 b 3)
EDIT
Implementation without loop using mapcan:
(defmacro make-plist-from-variables (&rest variables)
`(list ,#(mapcan (lambda (v) `(',v ,v)) variables))
Functions don't have access to the lexical environment of their callers.
More precisely, during evaluation you cannot access the values of lexical variables knowing only their symbols. Only macros have access to environment objects.
Special variables
You can use dynamic binding:
(defun foo ()
(declare (special a))
(symbol-value 'a))
(let ((a 3))
(declare (special a))
(foo))
=> 3
In your case, you would collect the symbol along its value, by using SYMBOL-vaLUE on all your symbols.
Related to your question is how to dynamically bind variables to values where the variable names and/or values are known at evaluation time; see special operator PROGV.
Macros
You could obtain e.g. an association list by writing the following code:
(acons 'a a (acons 'b b nil))
Depending on the use case behind your question, you may want to have a macro that expands into such code, that references the variables you want to evaluate.
I'm trying to implement the macro-function OR in Lisp
My attempt:
(defmacro or2 (test &rest args)
`(if ,test ,test (if (list ,#args) (or2 ,#args) nil)) )
However, if I test with something like this:
(or2 (print 1) 2 )
1
1
1
Whereas with the default OR:
(or (print 1) 2)
1
1
I understand that this is because of my two ,test at the beginning of my if clause, but I don't see how i could avoid it. How could I avoid applying twice the test effects ?
How would you solve the problem of side-effects if you had to code it by hand?
(or2 (print 1) 2)
Intermediate variable
Most probably, you would do this:
(let ((value (print 1)))
(if value value 2))
You need to define a local variable which holds the value of the first expression, so that later you can reference the variable instead of re-evaluating the same expression more than once.
But what if you already have a variable named value in the lexical context where you expand the code? What if, instead of 2, you were referencing that other value? This problem is named variable capture.
Gensym
In Common Lisp, you introduce a fresh symbol, that is guaranteed to not be already bound to anything, using GENSYM.
(let ((symbol (gensym)))
`(let ((,symbol ,test))
(if ,symbol ,symbol ...)))
Recursive expansion
(list ,#args)
The above is the same as writing directly ,args.
But you are confusing macroexpansion and execution times. If you inject args directly in the code, it will be evaluated (most likely, this is going to fail as a bad function call). What you want instead is to test if args is non-null during macroexpansion.
Besides, you should probably first test if your list of expression contains more than one element, in order to simplify the generated code.
Roughly speaking, you have to take into account the following cases:
(or2) is nil
(or2 exp) is the same as exp
(or2 exp &rest args) is the same as the following, where var is a fresh symbol:
`(let ((,var ,exp))
(if ,var ,var (or2 ,#args)))
Please make use of macroexpand-1:
(macroexpand-1 '(or2 (print 1) 2))
; ==> (if (print 1) (print 1) (if (list 2) (or2 2) nil)) ;
; ==> t
With macros you wish the order of evaluation to be expected and you wish expressions to only be evaluated once. Thus the expansion should have been something like this:
(let ((tmp (print 1)))
(if tmp
tmp
(or2 2)))
And tmp should be a symbol generated by gensym. Also when args is nil you should expand or2 to only test:
(defmacro or2 (test &rest args)
(if (endp args)
test
(let ((tmp (gensym "tmp")))
`(let ((,tmp ,test))
(if ,tmp
,tmp
(or2 ,#args))))))
you can make use of macros to simplify this:
(defmacro or2 (test &rest args)
(if (endp args)
test
(once-only (test)
`(if ,test
,test
(or2 ,#args)))))
I'm trying to write functions that wrap another function but I'm not sure how to pass parameters correctly while maintaining a sensible lambda-list.
E.g. if I have a function
(defun f (x &key y z) ...)
I want to write something like
(defun g (x &key y z)
(h (f x :y y :z z)))
This isn't satisfactory because I want to call f from g with the exact arguments g was called with, which doesn't happen (e.g. I don't want to supply keyword arguments to f that weren't supplied to g by the caller).
I initially wrote something like:
(defun g (&rest f-args)
(apply #'f f-args))
And that's the effect I want, however the lambda list for g is now very cryptic and I keep having to navigate to f to see what the arguments should be.
I did come up with a solution (and it's mostly satisfactory so I posted it as an answer), but I need to be explicit with every single key argument, and with large lambda-lists (e.g. if I want to wrap drakma:http-request), it will be a pain. I hope that maybe there's a better way.
You could write a macro that defines a function by copying the lambda list from another function. The problem is that there isn't a standard way to get the lambda list, but for SBCL you can use SB-INTROSPECT:FUNCTION-LAMBDA-LIST (although that won't work with (declaim (optimize (debug 0)))). You could try reading Swank source code to see how it gets the lambda lists for various implementations.
(defmacro define-wrapper (name lambda-source &body body)
`(defun ,name ,(sb-introspect:function-lambda-list lambda-source)
,#body))
(defun f (x &key (y 3) (z 4))
(+ x y z))
(define-wrapper g f
(* 2 (f x :y y :z z)))
(f 2) ;=> 9
(g 2) ;=> 18
That's a bit ugly since the code doesn't show the variable definitions. A bit more complex solution might be to do something like
;; Requires Alexandria.
(defmacro define-wrapper (name lambda-source &body body)
(let ((lambda-list (sb-introspect:function-lambda-list lambda-source)))
(multiple-value-bind (required optional rest keywords)
(alexandria:parse-ordinary-lambda-list lambda-list)
(declare (ignore rest))
`(defun ,name ,lambda-list
,#(sublis `((_ . (,lambda-source ,#(loop for r in required collect r)
,#(loop for (name init suppliedp)
in optional collect name)
,#(loop for ((k-name name) init suppliedp)
in keywords
append (list k-name name)))))
body)))))
(defun f (x &key (y 3) (z 4))
(+ x y z))
(define-wrapper g f
(* 2 _))
Where the _ in the wrapper is replaced with a call to the function F with the given arguments. You do still have to remember that the argument variables exist and can conflict with ones you define yourself.
That passes all arguments to the function regardless of whether they were given. That might mess up a function that behaves differently depending on whether an argument was supplied or not. You could avoid that by using APPLY, but it's a bit more complex.
(defmacro define-wrapper (name lambda-source &body body)
(let ((lambda-list (sb-introspect:function-lambda-list lambda-source)))
(alexandria:with-gensyms (deparsed-arglist-sym
key-sym val-sym suppliedp-sym)
(multiple-value-bind (required optional rest keywords)
(alexandria:parse-ordinary-lambda-list lambda-list)
(declare (ignore rest))
(multiple-value-bind (body declarations docstring)
(alexandria:parse-body body :documentation t)
`(defun ,name ,lambda-list
,#(when docstring (list docstring))
,#declarations
(let ((,deparsed-arglist-sym
(nconc (loop for ,val-sym in (list ,#required) collect ,val-sym)
(loop for (,val-sym . ,suppliedp-sym)
in (list ,#(loop for (name init suppliedp)
in optional
collect (list 'cons name
(or suppliedp t))))
when ,suppliedp-sym collect ,val-sym)
(loop for (,key-sym ,val-sym ,suppliedp-sym)
in (list ,#(loop for ((kname name) init suppliedp)
in keywords
collect (list 'list kname name
(or suppliedp t))))
when ,suppliedp-sym append (list ,key-sym ,val-sym)))))
,#(sublis `((_ . (apply #',lambda-source ,deparsed-arglist-sym)))
body))))))))
(define-wrapper bar drakma:http-request
"Return the length of a response to http-request."
;; HTTP-REQUEST has some &aux variables.
(declare (ignore drakma::unparsed-uri
drakma::args))
(length _))
(bar "http://www.google.com") ;=> 11400 (14 bits, #x2C88)
I came up with this:
(defun g (x &rest f-keys &key y z)
(declare (ignorable y z))
(apply #'f x f-keys))
It's great for small lambda-lists but I hope I could do better.
I also can't see default values unless I type them explicitly.