Why do these forms behave this way?
CL-USER>
(setf *closures*
(loop for num in (list 1 2 3 4)
collect (lambda ()
num)))
(
#<COMPILED-LEXICAL-CLOSURE #x302004932E1F>
#<COMPILED-LEXICAL-CLOSURE #x302004932DCF>
#<COMPILED-LEXICAL-CLOSURE #x302004932D7F>
#<COMPILED-LEXICAL-CLOSURE #x302004932D2F>)
CL-USER>
(funcall (first *closures*))
4
CL-USER>
(funcall (second *closures*))
4
I would have expected the first funcall to return 1, and the second to return 2, etc. This behavior is consistent with both Clozure Common Lisp and Steel-Bank Common Lisp implementations.
If I rework the loop macro to a version using dolist, what I'd expect is what's returned:
(setf *closures*
(let ((out))
(dolist (item (list 1 2 3 4) (reverse out))
(push (lambda () item) out))))
(
#<COMPILED-LEXICAL-CLOSURE #x302004A12C4F>
#<COMPILED-LEXICAL-CLOSURE #x302004A12BFF>
#<COMPILED-LEXICAL-CLOSURE #x302004A12BAF>
#<COMPILED-LEXICAL-CLOSURE #x302004A12B5F>)
CL-USER>
(funcall (first *closures*))
1
CL-USER>
(funcall (second *closures*))
2
CL-USER>
What's going on with the loop macro version?
num is same variable shared by all lambdas.
Use
(setf *closures*
(loop for num in (list 1 2 3 4)
collect (let ((num1 num))
(lambda ()
num1))))
num1 is fresh variable for each iteration.
As of dolist, "It is implementation-dependent whether dolist establishes a new binding of var on each iteration or whether it establishes a binding for var once at the beginning and then assigns it on any subsequent iterations." (CLHS, Macro DOLIST). So it may work on one implementation and fail on other.
The name num represents the same binding during the evaluation of LOOP.
Maybe you want to write:
(mapcar 'constantly (list 1 2 3 4))
to get what you meant.
Related
I'm having problems with the following macro:
(defmacro gather-params (&rest body)
"Return plist of params"
`(concatenate 'list
(map 'list
#'(lambda (plist)
(if (typep (first plist) 'keyword)
(cons 'list plist)
plist))
',body)))
Within the macro, I can't make plist evaluate by adding a comma in front of it, e.g: ,plist when I do that, the compiler complains that the variable ,plist does not exist.
Is there something I'm not understand about the scope within a macro?
Current result:
input: (gather-params (:mykey (+ 1 1)) (list 1 2 3))
result: ((LIST :MYKEY (+ 1 1)) (LIST 1 2 3))
Desired result:
input: (gather-params (:mykey (+ 1 1)) (list 1 2 3))
result: ((LIST :MYKEY 2) (LIST 1 2 3))
(concatenate 'list '(1 2 3)) is just (copy-list '(1 2 3))
-> which both evaluate to (1 2 3).
The macro generates code. It has a backquoted list. This backquoted list is computed at macroexpansion time -> when the macro expansion happens, for example during compilation.
The generated code has a function with a parameter called plist. This function executes at runtime. It does not exist during macroexpansion. Thus plist is no variable during macroexpansion. Thus you can't compute a value of plist during macroexpansion, since the variable does not exist then.
If you want to evaluate code, then Common Lisp has the function eval for that. One can for example call eval at runtime.
CL-USER 31 > (defmacro gather-params (&rest body)
"Return plist of params"
`(map 'list
#'(lambda (plist)
(if (typep (first plist) 'keyword)
(list 'list
(first plist)
(eval (second plist)))
plist))
',body))
GATHER-PARAMS
CL-USER 32 > (gather-params (:mykey (+ 1 1)) (list 1 2 3))
((LIST :MYKEY 2) (LIST 1 2 3))
Check out this example, with a different approach:
CL-USER 42 > (defmacro gather-params (&rest body)
"Return plist of params"
`(list ,#(mapcar
(lambda (plist)
(if (typep (first plist) 'keyword)
(list 'list
''list
(first plist)
(second plist))
plist))
body)))
GATHER-PARAMS
CL-USER 43 > (let ((foo 1))
(gather-params (:mykey (+ foo foo)) (list 1 2 3)))
((LIST :MYKEY 2) (1 2 3))
I am working on a complicated macro and have run into a roadblock.
(defmacro for-each-hashtable-band (body vars on &optional counter name)
`(block o
(with-hash-table-iterator (next-entry ,on)
(destructuring-bind
,(apply #'append vars)
(let ((current-band (list ,#(mapcar #'not (apply #'append vars)))))
(for (i 1 ,(length (apply #'append vars)) 2)
(multiple-value-bind
(succ k v) (next-entry)
(if succ
(progn
(setf (nth i current-band) k)
(setf (nth (+ 1 i) current-band) v))
(return-from o nil))))
current-band)
,#body))))
im getting "Evaluation aborted on #<UNDEFINED-FUNCTION NEXT-ENTRY {100229C693}>"
i dont understand why next-entry appears to be invisible to the macro i have created.
I've tried stripping down this to a small replicable example but i couldnt find a minimal scenario without the macro i created where next-entry would be invisible besides this scenario no matter what I tried, i've always managed to find a way to call next-entry in my other examples so im stumped as to why i cannot get it working here
I've tested the for macro ive created and it seems to generally work in most cases but for some reason it cannot see this next-entry variable. How do i make it visible?
In your code there are multiple places where the macro generates bindings in a way that is subject to variable capture (pdf).
(defmacro for-each-hashtable-band (body vars on &optional counter name)
`(block o ;; VARIABLE CAPTURE
(with-hash-table-iterator (next-entry ,on) ;; VARIABLE CAPTURE
(destructuring-bind ,(apply #'append vars)
(let ((current-band ;;; VARIABLE CAPTURE
(list ,#(mapcar #'not (apply #'append vars)))))
(for
(i ;;; VARIABLE CAPTURE
1 ,(length (apply #'append vars)) 2)
(multiple-value-bind (succ k v) ;;; VARIABLE CAPTURE
,(next-entry) ;;; WRONG EVALUATION TIME
(if succ
(progn
(setf (nth i current-band) k)
(setf (nth (+ 1 i) current-band) v))
(return-from o nil))))
current-band)
,#body))))
A simplified example of such a capture is:
`(let ((x 0)) ,#body)
Here above, the x variable is introduced, but if the code is expanded in a context where xis already bound, then body will not be able to reference that former x binding and will always see x bound to zero (you generally don't want this behavior).
Write a function instead
Instead of writing a big macro for this, let's first try understanding what you want to achieve and write instead a higher-order function, ie. a function that calls user-provided functions.
If I understand correctly, your function iterates over a hash-table by bands of entries. I assume vars holds a list of (key value) pairs of symbols, for example ((k1 v1) (k2 v2)). Then, body works on all the key/value pairs in the band.
In the following code, the function map-each-hashtable-band accepts a function, a hash-table, and instead of vars it accepts a size, the width of the band (the number of pairs).
Notice how in your code, you only have one loop, which builds a band using the hash-table iterator. But then, since the macro is named for-each-hashtable-band, I assume you also want to loop over all the bands. The macro with-hash-table-iterator provides an iterator but does not loop itself. That's why here I have two loops.
(defun map-each-hashtable-band (function hash-table band-size)
(with-hash-table-iterator (next-entry hash-table)
(loop :named outer-loop :do
(loop
:with key and value and next-p
:repeat band-size
:do (multiple-value-setq (next-p key value) (next-entry))
:while next-p
:collect key into current-band
:collect value into current-band
:finally (progn
(when current-band
(apply function current-band))
(unless next-p
(return-from outer-loop)))))))
For example:
(map-each-hashtable-band (lambda (&rest band) (print `(:band ,band)))
(alexandria:plist-hash-table
'(:a 0 :b 1 :c 2 :d 3 :e 4 :f 5 :g 6))
2)
NB. Iterating over a hash-table happens in an arbitrary order, there is no guarantee that you'll see the entries in any particular kind of order, this is implementation-dependant.
With my current version of SBCL this prints the following:
(:BAND (:A 0 :B 1))
(:BAND (:C 2 :D 3))
(:BAND (:E 4 :F 5))
(:BAND (:G 6))
Wrap the function in a macro
The previous function might not be exactly the behavior you want, so you need to adapt to your needs, but once it does what you want, you can wrap a macro around it.
(defmacro for-each-hashtable-band (vars hash-table &body body)
`(map-each-hashtable-band (lambda ,(apply #'append vars) ,#body)
,hash-table
,(length vars)))
For example:
(let ((test (alexandria:plist-hash-table '(:a 0 :b 1 :c 2 :d 3 :e 4 :f 5))))
(for-each-hashtable-band ((k1 v1) (k2 v2)) test
(format t "~a -> ~a && ~a -> ~a ~%" k1 v1 k2 v2)))
This prints:
A -> 0 && B -> 1
C -> 2 && D -> 3
E -> 4 && F -> 5
Macro-only solution, for completeness
If you want to have only one, single macro, you can start by inlining the body of the above function in the macro, you don't need to use apply anymore, but instead you need to establish bindings around the body, using destructuring-bind as you did. A first draft would be to simply as follows, but notice that this is not a proper solution:
(defmacro for-each-hashtable-band (vars hash-table &body body)
(let ((band-size (length vars)))
`(with-hash-table-iterator (next-entry ,hash-table)
(loop :named outer-loop :do
(loop
:with key and value and next-p
:repeat ,band-size
:do (multiple-value-setq (next-p key value) (next-entry))
:while next-p
:collect key into current-band
:collect value into current-band
:finally (progn
(when current-band
(destructuring-bind ,(apply #'append vars) current-band
,#body))
(unless next-p
(return-from outer-loop))))))))
In order to be free of variable capture problems with macros, each temporary variable you introduce must be named after a symbol that cannot exist in any context you expand your code. So instead we first unquote all the variables, making the macro definition fail to compile:
(defmacro for-each-hashtable-band (vars hash-table &body body)
(let ((band-size (length vars)))
`(with-hash-table-iterator (,next-entry ,hash-table)
(loop :named ,outer-loop :do
(loop
:with ,key and ,value and ,next-p
:repeat ,band-size
:do (multiple-value-setq (,next-p ,key ,value) (,next-entry))
:while ,next-p
:collect ,key into ,current-band
:collect ,value into ,current-band
:finally (progn
(when ,current-band
(destructuring-bind ,(apply #'append vars) ,current-band
,#body))
(unless ,next-p
(return-from ,outer-loop))))))))
When compiling the macro, the macro is supposed to inject symbols into the code, but here we have a compilation error that says undefined variables:
;; undefined variables: CURRENT-BAND KEY NEXT-ENTRY NEXT-P OUTER-LOOP VALUE
So now, those variables should be fresh symbols:
(defmacro for-each-hashtable-band (vars hash-table &body body)
(let ((band-size (length vars)))
(let ((current-band (gensym))
(key (gensym))
(next-entry (gensym))
(next-p (gensym))
(outer-loop (gensym))
(value (gensym)))
`(with-hash-table-iterator (,next-entry ,hash-table)
(loop :named ,outer-loop :do
(loop
:with ,key and ,value and ,next-p
:repeat ,band-size
:do (multiple-value-setq (,next-p ,key ,value) (,next-entry))
:while ,next-p
:collect ,key into ,current-band
:collect ,value into ,current-band
:finally (progn
(when ,current-band
(destructuring-bind ,(apply #'append vars) ,current-band
,#body))
(unless ,next-p
(return-from ,outer-loop)))))))))
This above is a bit verbose, but you could simplify that.
Here is what the previous for-each-hashtable-band example expands into with this new macro:
(with-hash-table-iterator (#:g1576 test)
(loop :named #:g1578
:do (loop :with #:g1575
and #:g1579
and #:g1577
:repeat 2
:do (multiple-value-setq (#:g1577 #:g1575 #:g1579) (#:g1576))
:while #:g1577
:collect #:g1575 into #:g1574
:collect #:g1579 into #:g1574
:finally (progn
(when #:g1574
(destructuring-bind
(k1 v1 k2 v2)
#:g1574
(format t "~a -> ~a && ~a -> ~a ~%" k1 v1 k2
v2)))
(unless #:g1577 (return-from #:g1578))))))
Each time you expand it, the #:gXXXX variables are different, and cannot possibly shadow existing bindings, so for example, the body can use variables named like current-band or value without breaking the expanded code.
Why is it not possible to push directly on a list like '(1 2 3) or NIL?
Specifically:
Why is possible to do
> (let ((some-list nil))
(push 42 some-list))
(42)
but not to do something like
(push 42 nil)
or
(push 42 '(1 2 3))
What is the reasoning behind this implementation?
With macro push the second argument needs to be a place to be modified. Here are some examples:
Lets make two variables:
(defparameter *v* (list 2 4))
(defparameter *v-copy* *v*)
Then we push 0
(push 1 *v*) ; ==> (1 2 4)
*v-copy* ; ==> (2 4) (unaltered)
; the reason is that the variable is changed, not its value
(macroexpand '(push 1 v))
; ==> (setq v (cons 1 v))
push can use other things as second argument. Lets try a cons
(push 3 (cdr *v-copy*))
*v-copy* ; ==> (2 3 4)
; since the tail of *v* is the *v-copy* *v* is changed too
*v* ; ==> (1 2 3 4)
(macroexpand-1 '(push 2 (cdr *v-copy*)))
; ==> (rplacd v (cons 2 (cdr *v-copy*)))
If your examples were valid, what should it really have done? Lets do the nil first:
(macroexpand '(push 42 nil))
; ==> (setq nil (cons 42 nil))
This treats nil just as any other variable and if this worked nil would never be the empty list again. It would have been a list with one element, 42 and a different value than (). In Common Lisp nil is a constant and cannot be mutated. I've created a lisp once where nil was a variable like any other and a small typo redefined nil making the programs behave strange with no apparent reason.
Lets try your literal quoted list.
(macroexpand '(push 42 (quote (1 2 3))))
; ==> (let ((tmp (1 2 3)))
; (funcall #'(setf quote) (cons 42 'tmp) tmp))
It doesn't seem the push macro differentiates between special form quote and those types that has set their setf function. It won't work and it doesn't make sense. Anyway in the same manner as mutating the binding nil if this changed the literal data '(1 2 3) to '(43 1 2 3) would you then expect to get (43 1 2 3) every time you evaluated (1 2 3) from there on? I imagine that would be the only true effect of mutating a constant. If this was allowed you should be allowed to redefine 4 to be 5 so that evaluating 4 or (+ 2 2) shows the result 5.
I want to save a reference (pointer) to a part of some Data I saved in another variable:
(let ((a (list 1 2 3)))
(let ((b (car (cdr a)))) ;here I want to set b to 2, but it is set to a copy of 2
(setf b 4))
a) ;evaluates to (1 2 3) instead of (1 4 2)
I could use macros, but then there would ever be much code to be executed if I want to change some Data in the middle of a list and I am not very flexible:
(defparameter *list* (create-some-list-of-arrays))
(macrolet ((a () '(nth 1000 *list*)))
(macrolet ((b () `(aref 100 ,(a))))
;; I would like to change the macro a here if it were possible
;; but then b would mean something different
(setf (b) "Hello")))
Is it possible, to create a variable as a reference and not as a copy?
cl-user> (let ((a '(1 2 3)))
(let ((b (car (cdr a))))
(setf b 4))
a)
;Compiler warnings :
; In an anonymous lambda form: Unused lexical variable B
(1 2 3)
A cons cell is a pair of pointers. car dereferences the first, and cdr dereferences the second. Your list is effectively
a -> [ | ] -> [ | ] -> [ | ] -> NIL
| | |
1 2 3
Up top where you're defining b, (cdr a) gets you that second arrow. Taking the car of that dereferences the first pointer of that second cell and hands you its value. In this case, 2. If you want to change the value of that pointer, you need to setf it rather than its value.
cl-user> (let ((a '(1 2 3)))
(let ((b (cdr a)))
(setf (car b) 4))
a)
(1 4 3)
If all you need is some syntactic sugar, try symbol-macrolet:
(let ((a (list 1 2 3 4)))
(symbol-macrolet ((b (car (cdr a))))
(format t "~&Old: ~S~%" b)
(setf b 'hello)
(format t "~&New: ~S~%" b)))
Note, that this is strictly a compile-time thing. Anywhere (in the scope of the symbol-macrolet), where b is used as variable, it is expanded into (car (cdr a)) at compile time. As Sylwester already stated, there are no "references" in Common Lisp.
I wouldn't recommend this practice for general use, though.
And by the way: never change quoted data. Using (setf (car ...) ...) (and similar) on a constant list literal like '(1 2 3) will have undefined consequences.
Building on what Baggers suggested. Not exactly what you are looking for but you can define setf-expanders to create 'accessors'. So lets say your list contains information about people in the for of (first-name last-name martial-status) and when someone marries you can update it as:
(defun marital-status (person)
(third person))
(defun (setf marital-status) (value person)
(setf (third person) value))
(let ((person (list "John" "Doe" "Single")))
(setf (marital-status person) "Married")
person)
;; => ("John" "Doe" "Married")
Is it possible to write a Common Lisp macro that takes a list of dimensions and variables, a body (of iteration), and creates the code consisting of as many nested loops as specified by the list?
That is, something like:
(nested-loops '(2 5 3) '(i j k) whatever_loop_body)
should be expanded to
(loop for i from 0 below 2 do
(loop for j from 0 below 5 do
(loop for k from 0 below 3 do
whatever_loop_body)))
Follow up
As huaiyuan correctly pointed out, I have to know the parameters to pass to macro at compile time. If you actually need a function as I do, look below.
If you are ok with a macro, go for the recursive solution of 6502, is wonderful.
You don't need the quotes, since the dimensions and variables need to be known at compile time anyway.
(defmacro nested-loops (dimensions variables &body body)
(loop for range in (reverse dimensions)
for index in (reverse variables)
for x = body then (list y)
for y = `(loop for ,index from 0 to ,range do ,#x)
finally (return y)))
Edit:
If the dimensions cannot be decided at compile time, we'll need a function
(defun nested-map (fn dimensions)
(labels ((gn (args dimensions)
(if dimensions
(loop for i from 0 to (car dimensions) do
(gn (cons i args) (cdr dimensions)))
(apply fn (reverse args)))))
(gn nil dimensions)))
and to wrap the body in lambda when calling.
CL-USER> (nested-map (lambda (&rest indexes) (print indexes)) '(2 3 4))
(0 0 0)
(0 0 1)
(0 0 2)
(0 0 3)
(0 0 4)
(0 1 0)
(0 1 1)
(0 1 2)
(0 1 3)
(0 1 4)
(0 2 0)
(0 2 1)
...
Edit(2012-04-16):
The above version of nested-map was written to more closely reflect the original problem statement. As mmj said in the comments, it's probably more natural to make index range from 0 to n-1, and moving the reversing out of the inner loop should improve efficiency if we don't insist on row-major order of iterations. Also, it's probably more sensible to have the input function accept a tuple instead of individual indices, to be rank independent. Here is a new version with the stated changes:
(defun nested-map (fn dimensions)
(labels ((gn (args dimensions)
(if dimensions
(loop for i below (car dimensions) do
(gn (cons i args) (cdr dimensions)))
(funcall fn args))))
(gn nil (reverse dimensions))))
Then,
CL-USER> (nested-map #'print '(2 3 4))
Sometimes an approach that is useful is writing a recursive macro, i.e. a macro that generates code containing another invocation of the same macro unless the case is simple enough to be solved directly:
(defmacro nested-loops (max-values vars &rest body)
(if vars
`(loop for ,(first vars) from 0 to ,(first max-values) do
(nested-loops ,(rest max-values) ,(rest vars) ,#body))
`(progn ,#body)))
(nested-loops (2 3 4) (i j k)
(print (list i j k)))
In the above if the variable list is empty then the macro expands directly to the body forms, otherwise the generated code is a (loop...) on the first variable containing another (nested-loops ...) invocation in the do part.
The macro is not recursive in the normal sense used for functions (it's not calling itself directly) but the macroexpansion logic will call the same macro for the inner parts until the code generation has been completed.
Note that the max value forms used in the inner loops will be re-evaluated at each iteration of the outer loop. It doesn't make any difference if the forms are indeed numbers like in your test case, but it's different if they're for example function calls.
Hm. Here's an example of such a macro in common lisp. Note, though, that I am not sure, that this is actually a good idea. But we are all adults here, aren't we?
(defmacro nested-loop (control &body body)
(let ((variables ())
(lower-bounds ())
(upper-bounds ()))
(loop
:for ctl :in (reverse control)
:do (destructuring-bind (variable bound1 &optional (bound2 nil got-bound2)) ctl
(push variable variables)
(push (if got-bound2 bound1 0) lower-bounds)
(push (if got-bound2 bound2 bound1) upper-bounds)))
(labels ((recurr (vars lowers uppers)
(if (null vars)
`(progn ,#body)
`(loop
:for ,(car vars) :upfrom ,(car lowers) :to ,(car uppers)
:do ,(recurr (cdr vars) (cdr lowers) (cdr uppers))))))
(recurr variables lower-bounds upper-bounds))))
The syntax is slightly different from your proposal.
(nested-loop ((i 0 10) (j 15) (k 15 20))
(format t "~D ~D ~D~%" i j k))
expands into
(loop :for i :upfrom 0 :to 10
:do (loop :for j :upfrom 0 :to 15
:do (loop :for k :upfrom 15 :to 20
:do (progn (format t "~d ~d ~d~%" i j k)))))
The first argument to the macro is a list of list of the form
(variable upper-bound)
(with a lower bound of 0 implied) or
(variable lower-bound upper-bounds)
With a little more love applied, one could even have something like
(nested-loop ((i :upfrom 10 :below 20) (j :downfrom 100 :to 1)) ...)
but then, why bother, if loop has all these features already?