Common lisp and infix package - common-lisp

There is the infix package for common lisp (see http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/syntax/infix/infix.cl), which transforms infix form to prefix.
For example:
(string->prefix "1+2") ;; => (+ 1 2)
It also provides the reader macro #I, which can evaluate an infix form, e.g.,
#I(1+2) ;; => 3
But I don't want to use this reader macro.
I have written a simple function, which uses cl-ppcre, to substitute a string with a number, i.e.,
(prepare-form "1+x*x" "x" 3) ;; => "1+3*3"
Finally, I have a function which evaluates the infix form
(defun eval-infix (form &rest args)
(eval (string->prefix (apply #'prepare-form form args))))
Can the function eval-infix be implemented without using the eval function?
My final goal is to call eval-infix like this:
(eval-infix "1+x*x" "x" (+ 1 2))

Well I think what you want to use is string->prefix. This will intern symbols so to avoid polluting your own package lets define one. Finally you can avoid the security problems of eval by defining your own eval. There is still a problem where symbols from other packages can be written and this makes me sad. There is also the problem that the infix reader can escape to the Lisp reader which can do arbitrary evaluation. Here is a sketch of the parts of the solution. I’ve probably got the arguments for some functions in the wrong order.
(defpackage infix-vars)
(defun read-infix-string (s) (let ((*package* (find-package "INFIX-VARS"))) (string->prefix s)))
(defun substitute-var (expr v val)
(let ((v (etypecase v (symbol v) (string (intern v (find-package "INFIX-VARS"))))))
(subst expr v val)))
(defun eval-expr (e)
(etypecase e
(number e)
(list
(ecase (car e)
(+ (apply #'+ (mapcar #'eval-expr (cdr e))))))))

Related

Variable Not A Number Error in Lisp (Which is not true)

I have a code which takes a list and returns all possible permutations by the parameter result.
But when I compile I have an error which says *** - =: (1+ INDEX) is not a number.
Is this message true or I messed up the code generally?
I am new to lisp I can looking for a fix and also open to suggestions from fucntional programmers.
;; Creates permutatiions of a given list and returns it via parameter
(defun create-permuations (source)
(setf result (list))
(create-permuations-helper source 0 '() result)
result)
(defmacro create-permuations-helper (source index cur result)
(if (= (list-length cur) index)
(cons cur result)
(loop for i from 0 to (list-length cur) do
(create-permuations-helper source (1+ index)
(append cur (list (nth i source))) result))))
99% of times when a compiler reports an error you can trust it to be true. Here Index is the list (1+ index), literally the 1+ symbol followed by the index symbol. This is so because you are using a macro, and macros operate on code.
In your macro, you do not return a form to be evaluated, you execute code during macro-expansion that depends on itself. That alone is an undefined behaviour. For example:
(defmacro a (x)
(if (plusp x)
(a (- x 1))
nil))
In the body of a, you want to expand code using a recursive call to itself. But the macro is not yet fully known and cannot be until the whole macro is defined.
Maybe the particular lisp implementation binds a to the macro function in body of the macro, which is a strange thing to do, or you evaluated the definition twice. The first time the compiler assumes a is an unknown function, then binds a to a macro, and the second time it tries to expand the macro.
Anyway macro are not supposed to be recursive.
In the example, since the macro does not evaluate its argument, the nested call to the macro is given the literal expression (- x 1), and not its actual value, which cannot be known anyway since x is unknown. You are crossing a level of abstraction here by trying to evaluate things at macroexpansion time.
But, macros can expand into code that refers to themselves.
(defmacro a (x)
(if (plusp x)
`(b (a ,(- x 1)))
nil))
Now, (a 2) expands into (b (a 1)), which itself macroexpands into (b (b (a 0))), and finally reaches a fixpoint which is (b (b nil)).
The difference is that the macro produces a piece of code and returns, which the compiler macroexpands again, whereas in the first example, the macro must already be expanded in the body of its own definition.
Possible implementation
One way to solve your problem is to define a local function that has access to a variable defined in your main function. Then, the local function can set it, and you do not need to pass a variable by reference (which is not possible to do):
(defun permut (list)
(let (result)
(labels ((recurse (stack list)
(if list
(dolist (x list)
(recurse (cons x stack)
(remove x list :count 1)))
(push stack result))))
(recurse nil list))
result))
Alternatively, you can split the process in two; first, define permut-helper, which is a higher-order function that takes a callback function; it generates permutations and calls the callback for each one:
(defun permut-helper (stack list callback)
(if list
(dolist (x list)
(permut-helper (cons x stack)
(remove x list :count 1)
callback))
(funcall callback stack)))
You call it with a function that pushes results into a list of permutations:
(defun permut (list)
(let (result)
(flet ((add-result (permutation)
(push permutation result)))
(permut-helper nil list #'add-result))
result))

Function with rest arguments calling a function with rest arguments

Let us suppose we have a function func1 :
(defun func1 (&rest values)
; (do something with values...)
(loop for i in values collect i))
Now, we have a function func2 which calls func1 :
(defun func2 (&rest values)
; (do something with values...)
(func1 ???))
What should I put instead of ??? to "copy" all the parameters of func2's values to func1's values ?
For instance, I would have the following behavior :
(func2 1 2 3 4) ; result is (1 2 3 4) and not ((1 2 3 4)).
In an earlier question I tried to do something like this :
(defun func2 (&rest values)
(macrolet ((my-macro (v)
`(list ,#v)))
(func1 (my-macro values))))
But the defun cannot get the value because it is not runtime. In this answer, he suggested that I use apply, but this function takes a &rest parameter too, so it doesn't solve my problem...
If possible, I would rather avoid to change the prototype of both functions, and the behavior of func1.
In common lisp, it has to be
(apply #'func1 values) ;; since `func1` has to be looked up in function namespace
remember, Clojure and Racket/Scheme are Lisp1, and common lisp is Lisp2.
Alternative solution (just for the sake)
I was asking myself, how to get it done without apply - just for the sake.
The problem with
`(func2 ,#values)
is, that if e.g.
(func2 (list 1 2 3) (list 4) 5)
is called, the values variable is ((1 2 3) (4) 5)
But when it is spliced into (func1 ,#values), what is created is
(func1 (1 2 3) (4) 5). But if we compare this with the func2 call,
it should be rather (func1 (list 1 2 3) (list 4) 5) which is perhaps not possible, because when (func2 (list 1 2 3) (list 4) 5) is called -
in the lisp manner - the arguments of func2 are each evaluated, before they enter the function body of func2, so we end up with values as a list of already evaluated arguments, namely ((1 2 3) (4) 5).
So somehow, concerning the arguments for func1 in the last expression, we are one evaluation-step offbeat.
But there is a solution with quote, that we manage to quote each of the arguments before giving it to func1 in the last expression, to "synchronize" the func1 function call - to let the arguments' evaluation pause for one round.
So my first aim was to generate a new values list inside the func2 body where each of the values list's argument is quoted (this is done in the let-binding).
And then at the end to splice this quoted-values list into the last expression: (func1 '(1 2 3) '(4) '5) which can be regarded as equivalent to (func1 (list 1 2 3) (list 4) 5) for this kind of problems / for this kind of calls.
This was achieved by this code:
(defun func2 (&rest vals)
(let ((quoted-values (loop for x in vals
collect `',x)))
; do sth with vals here - the func2 function -
(eval `(func1 ,#quoted-values))))
This is kind of a macro (it creates code btw. it organizes new code) but executed and created in run-time - not in pre-compile time. Using an eval we execute that generated code on the fly.
And like macroexpand-1, we can look at the result - the code - to which the func1 expression "expands", by removing eval around it - I call it func2-1:
(defun func2-1 (&rest vals)
(let ((quoted-values (loop for x in vals
collect `',x)))
; do sth with vals here - the func2 function -
`(func1 ,#quoted-values)))
And if we run it, it returns the last expression as code immediately before it is evluated in the func2 version:
(func2-1 (list 1 2 3) (list 4) 5)
;; (FUNC1 '(1 2 3) '(4) '5) ;; the returned code
;; the quoted arguments - like desired!
And this happens if we call it using func2 (so with evaluation of the func1 all:
(func2 (list 1 2 3) (list 4) 5)
;; ((1 2 3) (4) 5) ;; the result of (FUNC1 '(1 2 3) '(4) '5)
So I would say this is exactly what you desired!
lists vs. spread arguments
In Common Lisp it is good style to pass lists as lists and not as spread arguments:
(foo (list 1 2 3)) ; better interface
(foo 1 2 3) ; interface is not so good
The language has been defined in a way that efficient function calling can be used by a compiler and this means that the number of arguments which can be passed to a function is limited. There is a standard variable which will tell us how many arguments a particular implementation supports:
This is LispWorks on my Mac:
CL-USER 13 > call-arguments-limit
2047
Some implementations allow much larger number of arguments. But this number can be as low as 50 - for example ABCL, Common Lisp on the JVM, allows only 50 arguments.
Computing with argument lists
But sometimes we want the arguments as a list and then we can use the &rest parameter:
(lambda (&rest args)
(print args))
This is slightly in-efficient, since a list will be consed for the arguments. Usually Lisp tries to avoid to cons lists for arguments - they will be passed in registers or on the stack - if possible.
If we know that the argument list will not be used, then we can give the compiler a hint to use stack allocation - if possible:
(lambda (&rest args)
(declare (dynamic-extent args))
(reduce #'+ args))
In above function, the list of arguments can be deallocated when leaving the function - because the argument list is no longer used then.
If you want to pass these arguments to another function you can use FUNCALL and usually more useful APPLY:
(lambda (&rest args)
(funcall #'write (first args) (second args) (third args)))
or more useful:
(lambda (&rest args)
(apply #'write args))
One can also add additional arguments to APPLY before the list to apply:
CL-USER 19 > ((lambda (&rest args)
(apply #'write
(first args) ; the object
:case :downcase ; additional args
(rest args))
(values))
'(defun foo () 'bar)
:pretty t
:right-margin 15)
(defun foo ()
'bar)

Why are this list's contents retained between function calls? [duplicate]

Could someone explain to me what's going on in this very simple code snippet?
(defun test-a ()
(let ((x '(nil)))
(setcar x (cons 1 (car x)))
x))
Upon a calling (test-a) for the first time, I get the expected result: ((1)).
But to my surprise, calling it once more, I get ((1 1)), ((1 1 1)) and so on.
Why is this happening? Am I wrong to expect (test-a) to always return ((1))?
Also note that after re-evaluating the definition of test-a, the return result resets.
Also consider that this function works as I expect:
(defun test-b ()
(let ((x '(nil)))
(setq x (cons (cons 1 (car x))
(cdr x)))))
(test-b) always returns ((1)).
Why aren't test-a and test-b equivalent?
The Bad
test-a is self-modifying code. This is extremely dangerous. While the variable x disappears at the end of the let form, its initial value persists in the function object, and that is the value you are modifying. Remember that in Lisp a function is a first class object, which can be passed around (just like a number or a list), and, sometimes, modified. This is exactly what you are doing here: the initial value for x is a part of the function object and you are modifying it.
Let us actually see what is happening:
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote (nil)))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1 1))))) (setcar x (cons 1 (car x))) x))
The Good
test-b returns a fresh cons cell and thus is safe. The initial value of x is never modified. The difference between (setcar x ...) and (setq x ...) is that the former modifies the object already stored in the variable x while the latter stores a new object in x. The difference is similar to x.setField(42) vs. x = new MyObject(42) in C++.
The Bottom Line
In general, it is best to treat quoted data like '(1) as constants - do not modify them:
quote returns the argument, without evaluating it. (quote x) yields x.
Warning: quote does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader (see info node
Printed Representation). This means that (a . b) is not
identical to (cons 'a 'b): the former does not cons. Quoting should
be reserved for constants that will never be modified by side-effects,
unless you like self-modifying code. See the common pitfall in info
node Rearrangement for an example of unexpected results when
a quoted object is modified.
If you need to modify a list, create it with list or cons or copy-list instead of quote.
See more examples.
PS1. This has been duplicated on Emacs.
PS2. See also Why does this function return a different value every time? for an identical Common Lisp issue.
PS3. See also Issue CONSTANT-MODIFICATION.
I found the culprit is indeed 'quote. Here's its doc-string:
Return the argument, without evaluating it.
...
Warning: `quote' does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader
...
Quoting should be reserved for constants that will
never be modified by side-effects, unless you like self-modifying code.
I also rewrote for convenience
(setq test-a
(lambda () ((lambda (x) (setcar x (cons 1 (car x))) x) (quote (nil)))))
and then used
(funcall test-a)
to see how 'test-a was changing.
It looks like the '(nil) in your (let) is only evaluated once. When you (setcar), each call is modifying the same list in-place. You can make (test-a) work if you replace the '(nil) with (list (list)), although I presume there's a more elegant way to do it.
(test-b) constructs a totally new list from cons cells each time, which is why it works differently.

Pass a "variable name" to defvar

I've been struggling with this for two days now, and I can't find the answer.
What I want is to define three variables, a, b, and c each with a value of 0.
Naive:
(dolist (lbl '(a b c)) (defvar lbl 0))
Doesn't do what I want. a, b, and c remain undefined, and lbl now has a value of 0.
I think I may understand why this can't work: defvar is a macro, not a function, and as such I am passing it the form lbl, not the current value of label (which is a, b, c in turn). I think.
But in the resulting macroexpansion, shouldn't lbl eventually be linked-up(?) or evaluated(?) to the value I'm intending? Obviously not, either because it can't be done or I'm doing it wrong.
I want to understand:
How to make this work: (dolist (lbl '(a b c)) (defvar lbl 0))
What's going wrong under the hood. I have a feeling it has something to do with symbols or the mechanics of the quote operator.
Here are a few options:
With eval, by building a defvar expression:
(dolist (lbl '(a b c))
(eval `(defvar ,lbl 0))
With proclaim and setf of symbol-value (note: set is deprecated, since 1994 for what it's worth):
(dolist (lbl '(a b c))
(proclaim `(special ,lbl))
(setf (symbol-value lbl) 0))
This is actually mostly what defvar does (see notes in the linked page), but each Lisp implementation usually also records source file location, as they do for other defining macros.
Under the hood, defvar is a macro that makes the variable special (i.e. with dynamic extent bindings in the current dynamic environment; note: there's no portable undoing for this!), and optionally initializes it if it's not yet bound.
The fact that it's a macro means it doesn't evaluate its arguments, so it can take the variable name literally, and it does so. As such, (defvar lbl 0) will define the variable lbl, not the symbol stored in a lbl variable.
The fact that it optionally initializes the variable means that the initializing expression will not even be evaluated if the variable is boundp. So, its secondary effects won't happen if the variable is already initialized. This might or might not be expected.
Note that this expression isn't actually evaluated at macro-expansion time, it's left for evaluation when the expansion is evaluated, which in a REPL means right after macro expansion (and possibly after compilation, depending on the Lisp implementation; read more about evaluation and compilation, it's quite interesting).
Similar:
(dolist (lbl '(a b c))
(let ((lbl 0))
(print lbl)))
Why is lbl 0 and not some of a, b, c?
Because LET binds the symbol lbl and not its value.
Similar with (DEFVAR FOO 3).
Imagine following code:
(DEFVAR FOO 3)
(LET ((FOO 3)) ...)
Now, if we compile this code, the Lisp compiler recognizes the DEFVAR declaration and now knows that FOO is a special global variable. Thus in the let form FOO will be dynamically bound.
Compare this code:
(dolist (v '(FOO)) (eval `(DEFVAR ,v 3)))
(LET ((FOO 3)) ...)
The compiler won't see the DEFVAR and does not know that it should be a global special variable. In the LET form, FOO will have a lexical binding.
Thus DEFVAR needs to be a macro which knows the symbol at compile time (!) and which expands into a form that informs the compiler that the symbol is a special global variable. The form also sets the value when executed.
Thus the best way to create multiple DEFVAR declarations from a list of variables is to write a macro, which expands into a PROGN form with multiple DEFVARs. Inside the PROGN, the compiler will still recognize them.
CL-USER 21 > (pprint (macroexpand '(defvar* (a b c) 0)))
(PROGN (DEFVAR A 0) (DEFVAR B 0) (DEFVAR C 0))
Implemented as:
(defmacro defvar* (vars initial-value)
`(progn
,#(loop for var in vars
do (check-type var symbol)
collect `(defvar ,var ,initial-value))))
Note that it makes sense to check that the variables are really provided as symbols.
defvar is a special form which makes sure the symbol of it's first argument is a bound variable. If the variable is not bound the evaluated expression of the second argument becomes the bound variables value. Thus:
(defvar *x* 10) ; if *x* was not bound it's now 10
(defvar *x* 20) ; since *x* is defined nothing happens
Notice that *x* is not evaluated but is used unevaluated. In order to get the same functionality by using a variable that evaluates to a symbol which you want to exist as a variable in global scope you need to do something like this:
(defvar b 10)
(dolist (lbl '(a b c))
(when (not (boundp lbl))
(setf (symbol-value lbl) 0)))
Still, neither of the ones not already bound becomes special like with defvar, but at least you get the same behaviour:
(list a b c) ; => (0 10 0)
Perhaps you should just do:
(defvar *a* 0)
(defvar *b* 0)
(defvar *c* 0)
If you have a lot of variables you need to do this with you can do:
(defmacro defvars (lst value)
(loop :for e :in lst
:collect `(defvar ,e ,value) :into result
:finally (return (cons 'progn result))))
(defparameter *w* 10)
(defvars (*q* *w* *e*) 1)
(list *q* *w* *e* ; ==> (1 10 1)
Also, it's really important to earmuff your global variables. Once special it will follow dynamic binding. eg.
(defun test ()
(let ((*b* 15))
(test2)))
(defun test2 ()
*b*)
(test) ; ==> 15
Reimplementing DEFVAR
You can approximate the behavior of defvar with a function like this:
(defun %defvar (symbol value documentation)
"Define a global special variable.
symbol---a symbol
value---nil or a function of zero arguments
documentation---nil or a documentation string
returns symbol
Proclaim SYMBOL globally as a special variable. If VALUE is non-nil,
then if SYMBOL is not already bound, SYMBOL is assigned the value
returned by calling VALUE. DOCUMENATION is assigned as the
documentation of type variable to for SYMBOL."
(prog1 symbol
;; make it globally special
(proclaim (list 'special symbol))
;; if a value is provided, and symbol isn't
;; already bound, set its value to the result
;; of calling the value-function
(when (not (null value))
(unless (boundp symbol)
(setf (symbol-value symbol)
(funcall value))))
;; set the documentation
(setf (documentation symbol 'variable) documentation)))
Then you can do, e.g.,
CL-USER> (%defvar '*the-answer* (lambda () 42) "the answer")
*THE-ANSWER*
CL-USER> *the-answer*
42
CL-USER> (documentation '*the-answer* 'variable)
"the answer"
And with your original code, you could do something like:
(dolist (lbl '(a b c)) (%defvar lbl (lambda () 0)))
Now, how does this relate to what defvar actually does? Well, you could now implement a defvar like macro by doing:
(defmacro define-var (symbol &optional (value nil value-p) documentation)
`(%defvar
',symbol
,(if value-p `(lambda () ,value) 'nil)
,documentation))
This expands as we'd expect:
CL-USER> (macroexpand-1 '(define-var *the-answer* 42 "the answer"))
(%DEFVAR '*THE-ANSWER* (LAMBDA () 42) "the answer")
You can actually use macroexpand to look at what an implementation does, too. E.g., in SBCL:
CL-USER> (macroexpand-1 '(defvar *the-answer* 42 "the answer"))
(PROGN
(EVAL-WHEN (:COMPILE-TOPLEVEL) (SB-IMPL::%COMPILER-DEFVAR '*THE-ANSWER*))
(SB-IMPL::%DEFVAR '*THE-ANSWER* (UNLESS (BOUNDP '*THE-ANSWER*) 42) 'T
"the answer" 'T (SB-C:SOURCE-LOCATION)))
This isn't too much different from what we wrote above, though it's handling the non-evaluation of the form when the variable is already bound in a slightly different way, and it's also got some handling for recording a source location. The general idea is the same, though.
Why things don't get "linked up"
But in the resulting macroexpansion, shouldn't lbl eventually be
linked-up(?) or evaluated(?) to the value I'm intending?
The original code is:
(dolist (lbl '(a b c)) (defvar lbl 0))
We can macroexpand this to see what it becomes (in SBCL):
CL-USER> (macroexpand '(dolist (lbl '(a b c)) (defvar lbl 0)))
(BLOCK NIL
(LET ((#:N-LIST1022 '(A B C)))
(TAGBODY
#:START1023
(UNLESS (ENDP #:N-LIST1022)
(LET ((LBL (TRULY-THE (MEMBER C B A) (CAR #:N-LIST1022))))
(SETQ #:N-LIST1022 (CDR #:N-LIST1022))
(TAGBODY (DEFVAR LBL 0)))
(GO #:START1023))))
NIL)
T
Now, we can still see LBL in two places, including in (defvar LBL 0). So why don't things get "matched up"? To see that, we need to remember that the defvar inside the let will also be macroexpanded. To what? This:
CL-USER> (macroexpand '(DEFVAR LBL 0))
(PROGN
(EVAL-WHEN (:COMPILE-TOPLEVEL) (SB-IMPL::%COMPILER-DEFVAR 'LBL))
(SB-IMPL::%DEFVAR 'LBL (UNLESS (BOUNDP 'LBL) 0) 'T NIL 'NIL
(SB-C:SOURCE-LOCATION)))
But now we see that SBCL's internals are getting the symbol named "LBL"; the call (sb-impl::%defvar 'lbl …) is calling the function sb-impl::%defvar with the symbol lbl, and there's no connection between that symbol and the lexical variable that happens to be represented in the source by the same symbol. After all, if you write:
CL-USER> (let ((a 89))
(list 'a a))
(A 89)
You want to be able to get the symbol a and the number 89, right? The macroexpansion of defvar includes a call to a function with the quotation of one of the arguments to macro.

(compose) in Common Lisp

We find this function builder to realize composition in P.Graham's "ANSI Common Lisp" (page 110).
The arguments are n>0 quoted function names. I don't understand it completely, so I'll quote the code here and specify my questions underneath it:
(defun compose (&rest fns)
(destructuring-bind (fn1 . rest) (reverse fns)
#'(lambda (&rest args)
(reduce #'(lambda (v f) (funcall f v))
rest
:initial-value (apply fn1 args)))))
The argument list to compose is reversed and unpacked, its (now first) element bound to 'fn1' and the rest to 'rest'.
The body of the outermost lambda is a reduce: (funcall fi (funcall fi-1 ... ) ), with operands in inverted order to restore the initial one.
1) What is the role of the outermost lambda expression? Namely, where does it get its 'args' from? Is it the data structure specified as the first argument of destructuring-bind?
2) Where does the innermost lambda take its two arguments from?
I mean I can appreciate what the code does but still the lexical scope is a bit of a mystery to me.
Looking forward to any and all comments!
Thanks in advance,
//Marco
It's probably easier if you consider first a couple of practical examples:
(defun compose1 (a)
(lambda (&rest args)
(apply a args)))
(defun compose2 (a b)
(lambda (&rest args)
(funcall a (apply b args))))
(defun compose3 (a b c)
(lambda (&rest args)
(funcall a (funcall b (apply c args)))))
So the outermost lambda is the return value: a function that takes any arguments, what it does with it is applying the last function and chaining all the others in reverse order on the result got from last function.
Note: compose1 could be defined more simply as (defun compose1 (a) a).
A somewhat equivalent but less efficient version could be
(defun compose (&rest functions)
(if (= (length functions) 1)
(car functions)
(lambda (&rest args)
(funcall (first functions)
(apply (apply #'compose (rest functions))
args)))))
1) The outermost lambda creates a closure for you, because the result of (combine ...) is a function that calulates the composition of other functions.
2) The innermost lambda gets ists argument from the function reduce. Reduce takes a function (the innermost lambda) of two arguments and applies it stepwise to a list, e.g.
(reduce #'- '(1 2 3 4)) is (- (- (- 1 2) 3) 4)

Resources