How do you introduce new, predictably named identifiers with suffixes in a macro?
So, I'm writing a simple library for manipulating quaternions. I'm using the simplest representation that could possibly work, a list of components, for the time being, but I'd like to define a simple API that doesn't depend on that representation.
When defining algorithms, I want to refer to each of the components of the quaternion with a predictable name like somesymbol-realpart or somesymbol-i.
I'd like to be able to get the following snippet to work.
(let
((my-quat '(1 2 3 4)))
(with-quaternion my-quat
(assert-equalp 1 my-quat-realpart)
(assert-equalp 2 my-quat-i)
(assert-equalp 3 my-quat-j)
(assert-equalp 4 my-quat-k)))
However, the method I used for producing a symbol with a suffix seems to produce strange case-sensitive symbols with escaped uppercase characters.
(defun add-suffix-to-symbol (sym suffix)
(intern (concatenate 'string "" (string sym) "-" suffix)))
As a result of converting a symbol to a string, it is printed in uppercase... which is a totally valid canonicalization. However, creating a new symbol via intern preserves the case for some reason, so I have to do something like the following to refer to bindings introduced by with-quaternion.
(let
((my-quat '(1 2 3 4)))
(with-quaternion my-quat
(assert-equalp 1 |MY-QUAT-realpart|)
(assert-equalp 2 |MY-QUAT-i|)
(assert-equalp 3 |MY-QUAT-j|)
(assert-equalp 4 |MY-QUAT-k|)))
How do I create a new symbol that's identical to an old one but with a suffix so that it can be used in a macro?
for reference, here is all the code.
(defun assert-equalp (e a)
(assert (equalp e a)))
(defun quat-realpart (q)
(first q))
(defun quat-i (q)
(second q))
(defun quat-j (q)
(third q))
(defun quat-k (q)
(fourth q))
(assert-equalp '1 (quat-realpart '(1 2 3 4)))
(assert-equalp '2 (quat-i '(1 2 3 4)))
(assert-equalp '3 (quat-j '(1 2 3 4)))
(assert-equalp '4 (quat-k '(1 2 3 4)))
(defun add-suffix-to-symbol (sym suffix)
(intern (concatenate 'string "" (string sym) "-" suffix)))
(print (add-suffix-to-symbol 'a "suffix"))
(defgeneric with-quaternion-impl (q-sym body))
(defmethod with-quaternion-impl ((q-sym symbol) body)
(let
((q-realpart (add-suffix-to-symbol q-sym "realpart"))
(q-i (add-suffix-to-symbol q-sym "i"))
(q-j (add-suffix-to-symbol q-sym "j"))
(q-k (add-suffix-to-symbol q-sym "k")))
`(let
((,q-realpart (quat-realpart ,q-sym))
(,q-i (quat-i ,q-sym))
(,q-j (quat-j ,q-sym))
(,q-k (quat-k ,q-sym)))
(progn ,#body))))
(defmacro with-quaternion (q-sym &rest body)
(with-quaternion-impl q-sym body))
(let
((my-quat '(1 2 3 4)))
(with-quaternion my-quat
(assert-equalp 1 |MY-QUAT-realpart|)
(assert-equalp 2 |MY-QUAT-i|)
(assert-equalp 3 |MY-QUAT-j|)
(assert-equalp 4 |MY-QUAT-k|)))
(let
((my-quat '(1 2 3 4)))
(with-quaternion my-quat
(assert-equalp 1 my-quat-realpart)
(assert-equalp 2 my-quat-i)
(assert-equalp 3 my-quat-j)
(assert-equalp 4 my-quat-k)))
When run under clisp, it prints the following symbol, clearly with escaped uppercase characters.
|A-suffix|
and produces the following error message:
*** - PROGN: variable MY-QUAT-REALPART has no value
Symbols in common lisp are interned in uppercase by default. The apparent case insensitivity is because everything you type is converted to uppercase at read/retrieval time, unless you use the special syntax with bar characters |My-case-sensitive-SYMBOL|. my-case-insensitive-symbol and MY-CASE-INSENSITIVE-SYMBOL refer to the same interned symbol, which is stored in all uppercase (though, this being common lisp, it is often possible to change that with command line options and reader macros). The symbol is actually not case insensitive at all, it just appears that way because most of the symbols in your code are uppercased by the reader, unless you specially exempt them from that by surrounding them in bar characters or purposely configuring an environment with unusual reader options.
The net effect of all the above is that if you want to access a macro-generated symbol using the more familiar syntax, be sure to have all components uppercased before it's interned, e.g.:
(add-suffix-to-symbol q-sym "I")
instead of
(add-suffix-to-symbol q-sym "i")
Another option is to pass symbols to be concatenated rather than strings, e.g.
(defun add-suffix-to-symbol (sym suffix)
(intern (concatenate 'string "" (string sym) "-" (string suffix))))
(print (add-suffix-to-symbol 'FOO 'bar)) ; foo-bar
(print (add-suffix-to-symbol 'foo '|bar|)) ; |FOO-bar| because foo is converted to FOO at read time
To add a bit to Joe's answer:
Symbols are a datatype with names, values, a property list and they may be interned in a package (another Lisp data structure).
Symbols preserve their namestring cases
You can create a symbol with its name from a string and also ask for the name of the symbol. A function to make a symbol is, well, make-symbol:
CL-USER 8 > (make-symbol "This is A Symbol!!!***")
#:|This is A Symbol!!!***|
CL-USER 9 > (symbol-name (make-symbol "This is A Symbol!!!***"))
"This is A Symbol!!!***"
As you see the string is used as is and we get it out as it was provided. There is no case conversion.
Escaping a symbol for the reader
To print a symbol with different cases, whitespace and or special characters the symbol is escaped with surrounding | or single \:
CL-USER 11 > '|foo BAR ***# <>|
|foo BAR ***# <>|
CL-USER 12 > '\f\o\o\ BAR\ ***#\ <>
|foo BAR ***# <>|
The reader is by default uppercasing unescaped input for symbol names
The Lisp reader may use functions like find-symbol or intern to find or create symbols. Both can take a string as input and they are also case sensitive:
CL-USER 15 > (let ((symbol '|foo|))
(eq (find-symbol "FOO") symbol))
NIL
But the reader itself (used for example via read and read-from-string) is not case sensitive by default. By default all symbols get uppercased:
CL-USER 21 > (symbol-name 'foo)
"FOO"
CL-USER 22 > (symbol-name 'FOO)
"FOO"
CL-USER 23 > (eq 'foo 'FOO)
T
We can check that both the printer and the reader are using uppercase by default:
CL-USER 35 > *print-case*
:UPCASE
CL-USER 36 > (readtable-case *readtable*)
:UPCASE
When creating symbols in macros we usually want uppercase strings
This means that when one creates symbols with names from strings, then one usually wants uppercase strings as input:
lowercase:
CL-USER 25 > (intern "zippy")
|zippy|
NIL
uppercase:
CL-USER 26 > (intern "ZIPPY")
ZIPPY
NIL
In data we sometimes want mixed case symbols: escape them
Sometimes we want to work with different cases: for example when the case needs to preserved because it is used as data:
CL-USER 27 > (defvar *parents* '(|Eva Luator| |Ben BitDiddle jr.|))
*PARENTS*
CL-USER 28 > *parents*
(|Eva Luator| |Ben BitDiddle jr.|)
Creating uppercased symbol names with format
Often in code one creates symbol names with format - which might be more concise than concatenate. One can then use format control strings, where the text or parts of it are uppercased by using ~:#( and ~):
CL-USER 33 > (format nil "~:#(~a-~a-~a~)" "my" "macro" "name")
"MY-MACRO-NAME"
CL-USER 34 > (intern (format nil "~:#(~a-~a-~a~)" "my" "macro" "name"))
MY-MACRO-NAME
Related
Suppose a function fun in the code below, my goal is evaluating expr2 below.
(defun fun (&key (x nil)) x)
(defparameter expr1 (list 'fun :x 2))
(defparameter expr2 (list 'fun (intern "x" "KEYWORD") 2))
As expected, (eval expr1) gives 2, but (eval expr2) gives an error like
*** - FUN: illegal keyword/value pair :|x|, 2 in argument list.
The allowed keywords are (:X) The following restarts are available: ABORT :R1 Abort main loop
Why does this error occurs? and How can I fix it?
The reason is that normally in Common Lisp every symbol is translated to uppercase letters when read (this is the standard behaviour, and can be changed), so that:
(defun fun (&key (x nil)) x)
(defparameter expr1 (list 'fun :x 2))
is actually read as:
(DEFUN FUN (&KEY (X NIL)) X)
(DEFPARAMETER EXPR1 (LIST 'FUN :X 2))
while intern gets a string as first parameter and does not transform it, so that in your example "x" is interned as the symbol :x, which is different from the symbol :X (and this is the reason of the error). Note that when a symbol with lowercase letters is printed in REPL, it is surrounded by pipe characters (|), like in |x|, so that, when read again, lowercase characters are unchanged:
CL-USER> :x
:X
CL-USER> :|x|
:|x|
CL-USER> (format t "~a" :|x|)
x
NIL
To solve your problem, you can simply write the string directly in uppercase:
(defparameter expr2 (list 'fun (intern "X" "KEYWORD") 2))
and then (eval expr2) works as intended.
Note that \ and | are escape characters in symbols:
? 'foo\xBAR
FOO\xBAR
? '|This is a symbol|
|This is a symbol|
? ':|This is a keyword symbol with spaces and Capital letters!!!|
:|This is a keyword symbol with spaces and Capital letters!!!|
? 'what|? wow |this| also works|?
|WHAT? wow THIS also works?|
I'm building an alphametic solver, and I'd like to make a macro that substitutes numbers into a symbol template.
Here is a self-contained example:
(defparameter *symbol-positions* '(#\H #\T #\S #\R #\A #\U #\E #\O #\W #\N))
(defmacro word-to-number (symbols lst)
`(tonumber (list ,#(loop for symbol in symbols
when (not (eql symbol #\ ))
collect `(nth ,(position symbol *symbol-positions*) ,lst )))))
(defparameter num '(0 1 2 3 4 5 6 7 8 9))
(defparameter east '(#\ #\E #\A #\S #\T))
The following call works:
(word-to-number (#\ #\E #\A #\S #\T) num)
But this one doesn't:
(word-to-number east num) ;=> The value EAST is not of type LIST
Is there a way I can modify the macro to take a variable for the SYMBOLS parameter? ,symbols doesn't work, and neither does `(,#symbols)
When you do:
(word-to-number east num)
The macro expander gets called with the arguments being east and num. They won't be lists and number for you macro. Only for the resulting code will they be evaluated in a contex which yields values.
A macro is syntax transformation. eg.
(cond (p1 c1)
(p2 c2)
(t a))
; ==
(if p1
c1
(if p2
c2
a))
It is regardless if p1 is (< a b) or my-boolean-value. A macro just places the expressions there without having to know what a, b or my-boolean-value is.
So tell me.. How should the expansion look with (word-to-number east num)? Perhaps it shouldn't be a macro at all? eg.
(defun word-to-number (symbols lst)
(tonumber (loop :for symbol :in symbols
:when (not (eql symbol #\ ))
:collect (nth (position symbol *symbol-positions*) lst))))
UPDATE
(defmacro word-to-number (symbols lst)
`(tonumber (loop :for symbol :in ,symbols
:with clst := ,lst
:when (not (eql symbol #\ ))
:collect (nth (position symbol *symbol-positions*) clst))))
You might notice that I'm storing lst in a variable clst and I do it after evaluating symbols. The reason is that when you expect arguments to be evaluated you expect them to be evaluated in order and only once unless repeated evaluation is a feature of the macro like loop does. Eg. this should only print "Oh, happy day" once:
(word-to-number (progn (princ "Oh") east) (progn (princ ", happy day!") num))
It is possible to get value of the symbol unevaluated like this:
(let ((form '(+ 1 2))) `',form)
This expression evaluates to the following:
'(+ 1 2)
Is there some way to do the same thing but without using backquote?
(let ((form '(+ 1 2))) (list 'quote form))
If form is really a constant:
(list 'quote (list '+ '1 '2))
The quotes on 1 and 2 are redundant, since they're literals, but they are informative and are already there in case you replace then with actual expressions.
You can use the longer syntax if you want. Let's see how your form is read and evaluated, step-by-step.
(let ((form '(+ 1 2))) `',form)
Apostrophe is quote:
(let ((form '(+ 1 2))) `(quote ,form))
Backquote/comma (quasiquote) is a shorthand for building data:
(let ((form '(+ 1 2))) (list 'quote form))
Evaluate the let binding, which associates form with its value, literally the list (+ 1 2), inside the lexical environment of this expression:
(list 'quote form)
Te above builds a list made of the quote symbol and the current value bound to form. The above results in a list starting with quote and a sublist, which prints as follows:
(quote (+ 1 2))
... which admits this simpler representation:
'(+ 1 2)
So you can use (list 'quote form) if you prefer, but this is not much different.
(let ((form '(+ 1 2))) `',form)
You ask:
It is possible to get value of the symbol unevaluated...
Actually this not really what it does. By default Common Lisp has local variables using lexical bindings, not symbols having values. Above form computes the value of the variable form, not of the symbol form. The value of form is the list (+ 1 2). With the backquoted quote, you put a list (quote ...) around it. Which gets printed as (QUOTE (+ 1 2)) or '(+ 1 2).
Note that Common Lisp has no way to get from a symbol to the value of a lexical binding. In the source code symbols denote variables, but at runtime we have lexical bindings and not associations from symbols to values.
I often want to output lists and also print their position in the list e.g.
'(a b c) would become "1:A 2:B 3:C"
As FORMAT already supports iterating over a given list, I was wondering whether it also provides some sort of counting directive?
E.g. the FORMAT string could look like this: "~{~#C:~a~}" whereas ~#C would be the counter.
If you want a boring answer, here you go:
(format T "~:{~a:~a ~}" (loop for i from 0 for e in '(x y z) collect (list i e)))
And now for a more interesting one! Similarly to #Renzo's answer, this uses the Tilde directive to achieve its work.
(defvar *count* 0)
(defvar *printer* "~a")
(defun iterate-counting (stream arg c at)
(declare (ignore c))
(let ((*count* (if at -1 0)))
(destructuring-bind (*printer* delimiter &rest args) arg
(format stream (format NIL "~~{~~/iterate-piece/~~^~a~~}" delimiter) args))))
(defun iterate-piece (stream arg &rest dc)
(declare (ignore dc))
(incf *count*)
(format stream *printer* *count* arg))
This uses two special variables to make it both thread-safe and to allow nesting. I won't say that it's handy to use though. The first item of the argument to list has to be a format string that denotes how to print the argument and counter. For such a format list, the first argument is the counter, and the second argument is the actual item to list. You can switch those around if you need to using the asterisk directive. The second item should be a string to print as the delimiter between each item. Finally, the rest of the list has to be the actual items to print.
(format T "~/iterate-counting/" '("~a:~a" " " x y z))
=> 1:X 2:Y 3:Z
(format T "~/iterate-counting/" '("~a:~/iterate-counting/" " " ("~a>~a" "," 0 1 2) ("~a>~a" "," a b c) ("~a>~a" "," x y z)))
=> 1:1>0,2>1,3>2 2:1>A,2>B,3>C 3:1>X,2>Y,3>Z
If you want it to start counting from zero, add an # modifier to the iterate-counting:
(format T "~#/iterate-counting/" '("~a:~a" " " x y z))
=> 0:X 1:Y 2:Z
I wouldn't personally use this, as it's far less than obvious what is going on if you stumble across the directive uninitiated. It would probably be much less confusing for the potential future reader to write a tailored function for this, than trying to ab/use format.
A not so simple but reusable way of producing a numbered list is by using the ~/ directive (Tilde Slash: Call Function) with a user-defined function. For instance:
(let ((position 0))
(defun init-pos(str arg col at)
(declare (ignore str arg col at))
(setf position 0))
(defun with-pos(str arg col at)
(declare (ignore col at))
(format str "~a:~a" (incf position) arg)))
and then write format like this one:
(format nil "~/init-pos/~{~/with-pos/~^ ~}" nil '(a b c))
Note that, as said in a comment, this solution has two limitations:
You cannot use it if you need to format objects in concurrent threads, and
you cannot use it for nested lists.
What is the difference between "set", "setq", and "setf" in Common Lisp?
Originally, in Lisp, there were no lexical variables -- only dynamic ones. And
there was no SETQ or SETF, just the SET function.
What is now written as:
(setf (symbol-value '*foo*) 42)
was written as:
(set (quote *foo*) 42)
which was eventually abbreviavated to SETQ (SET Quoted):
(setq *foo* 42)
Then lexical variables happened, and SETQ came to be used for assignment to them too -- so it was no longer a simple wrapper around SET.
Later, someone invented SETF (SET Field) as a generic way of assigning values to data structures, to mirror the l-values of other languages:
x.car := 42;
would be written as
(setf (car x) 42)
For symmetry and generality, SETF also provided the functionality of SETQ. At this point it would have been correct to say that SETQ was a Low-level primitive, and SETF a high-level operation.
Then symbol macros happened. So that symbol macros could work transparently, it was realized that SETQ would have to act like SETF if the "variable" being assigned to was really a symbol macro:
(defvar *hidden* (cons 42 42))
(define-symbol-macro foo (car *hidden*))
foo => 42
(setq foo 13)
foo => 13
*hidden* => (13 . 42)
So we arrive in the present day: SET and SETQ are atrophied remains of older dialects, and will probably be booted from eventual successors of Common Lisp.
(set ls '(1 2 3 4)) => Error - ls has no value
(set 'ls '(1 2 3 4)) => OK
(setq ls '(1 2 3 4)) => OK - make ls to (quote ls) and then have the usual set
(setf ls '(1 2 3 4)) => OK - same as setq so far BUT
(setf (car ls) 10) => Makes ls '(10 2 3 4) - not duplicated by setq/set
You can use setf in place of set or setq but not vice versa since setf can also set the value of individual elements of a variable if the variable has individual elements. See the exaples below:
All four examples will assign the list (1, 2, 3) to the variable named foo.
(set (quote foo) (list 1 2 3)) ;foo => (1 2 3)
(1 2 3)
(set 'foo '(1 2 3)) ;foo => (1 2 3) same function, simpler expression
(1 2 3)
(setq foo '(1 2 3)) ;foo => (1 2 3) similar function, different syntax
(1 2 3)
(setf foo '(1 2 3)) ;foo => (1 2 3) more capable function
(1 2 3)
setf has the added capability of setting a member of the list in foo to a new value.
foo ;foo => (1 2 3) as defined above
(1 2 3)
(car foo) ;the first item in foo is 1
1
(setf (car foo) 4) ;set or setq will fail since (car foo) is not a symbol
4
foo ;the fist item in foo was set to 4 by setf
(4 2 3)
However, you can define a symbol macro that reprents a single item within foo
(define-symbol-macro foo-car (car foo)) ; assumes FOO => (1 2 3)
FOO-CAR
foo-car ;foo-car is now a symbol for the 1st item in foo
1
(setq foo-car 4) ;set or setq can set the symbol foo-car
4
foo ;Lisp macros are so cool
(4 2 3)
You can use defvar if you have not already defined the variable and do not want to give it a value until later in your code.
(defvar foo2)
(define-symbol-macro foo-car (car foo2))
setq is just like set with a quoted first arg -- (set 'foo '(bar baz)) is just like (setq foo '(bar baz)). setf, on the other hand, is subtle indeed -- it's like an "indirection". I suggest http://www.n-a-n-o.com/lisp/cmucl-tutorials/LISP-tutorial-16.html as a better way to get started understanding it than any answer here can give... in short, though, setf takes the first argument as a "reference", so that e.g. (aref myarray 3) will work (as the first arg to setf) to set an item inside an array.
One can think of SET and SETQ being low-level constructs.
SET can set the value of symbols.
SETQ can set the value of variables.
Then SETF is a macro, which provides many kinds of setting things: symbols, variables, array elements, instance slots, ...
For symbols and variables one can think as if SETF expands into SET and SETQ.
* (macroexpand '(setf (symbol-value 'a) 10))
(SET 'A 10)
* (macroexpand '(setf a 10))
(SETQ A 10)
So SET and SETQ are used to implement some of the functionality of SETF, which is the more general construct. Some of the other answers tell you the slightly more complex story, when we take symbol macros into account.
I would like to add to previous answers that setf is macro that call specific function depending on what was passed as its first argument.
Compare results of macro expansion of setf with different types of arguments:
(macroexpand '(setf a 1))
(macroexpand '(setf (car (list 3 2 1)) 1))
(macroexpand '(setf (aref #(3 2 1) 0) 1))
For some types of arguments "setf function" will be called:
(defstruct strct field)
(macroexpand '(setf (strct-field (make-strct)) 1))