I'm working on a code formatter for Lisp, which is using the reader to read code into S-expression format.
This works fine for plain symbols.
It doesn't work so well for qualified symbols. foo:bar is only readable if the package foo has been defined, but of course as far as the formatter is concerned, it has not, because unlike the compiler, the formatter is only reading the code, not executing it.
How can I tell the reader to either go ahead and automatically create a package foo on the fly, or failing that, don't sweat it, just read foo:bar, not as a symbol per se, but in some unambiguous format I can deal with as a special case?
I believe that you should not use the reader for that, because that is lossy (you lose comments, and anything that gets changed through reader macros, e. g. read-time-values, read-time references etc.).
But if you want, you can automatically create the package and maybe also export the symbol by handling the error, e. g. on SBCL:
(handler-bind ((sb-int:simple-reader-package-error
(lambda (e)
(let ((p (sb-int::package-error-package e)))
(ctypecase p
(string
(make-package p)
(invoke-restart 'retry))
(package
(export (intern (first (simple-condition-format-arguments e)) p) p)
(invoke-restart 'retry)))))))
(with-simple-restart (retry "Retry")
(read-from-string "foo:bar")))
This is a bit hacky, and we have no guarantee that the format of the condition stays like that.
Related
This question is somewhat related to an earlier one on programmatically generating symbol macros. I'm using that function in a convenience macro that throws undefined variable warnings. This macro and function:
(defmacro define-data (d body &optional doc)
(if (and doc (not (stringp doc))) (error "Documentation is not a string"))
`(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
;; Should we have an eval-when (:compile-toplevel) here?
(defparameter ,d ,body ,doc)
(export ',d old-package)
(define-column-names ,d)))
(defun define-column-names (d)
(maphash #'(lambda (key index)
(eval `(cl:define-symbol-macro ,key (cl:aref (columns ,d) ,index))))
(ordered-keys-table (slot-value d 'ordered-keys))))
are intended to be like defparameter, but additionally set up a few niceties for the user by defining:
a package with the name of d
a parameter in the current package with the data that will be sucked in by body
symbol-macros in package d for access to the individual data vectors
If I use defparameter from the REPL, and then call define-column-names, all is well. However when using the macro I get:
; in: DEFINE-COLUMN-NAMES FOO
; (DEFINE-COLUMN-NAMES CL-USER::FOO)
;
; caught WARNING:
; undefined variable: CL-USER::FOO
I suspect that this is because the compiler has no way of knowing that FOO will actually be defined when define-symbol-macro is called. Everything works fine, but I don't want the warning to frighten users, so am thinking of suppressing it. I hate suppressing warnings though, so thought I'd come here for a second opinion.
EDIT: I've marked an answer correct because it does correctly answer the question as asked. For an answer to the problem see my comments.
My answer to the 'when to muffle warnings' question in the title is: if it's your own code then never, under any circumstances. If it is someone else's code, then rewrite it not to warn unless you can't.
As to solving the problem I haven't thought about this hard enough, but the problem is that you definitely want the defparameter to be at top-level so the compiler can see it, and it can't really be if it's inside a let. But you can raise it to toplevel trivially since it depends on nothing inside the let.
I am then pretty certain that you want the rest of the macro to happen at compile time, because you definitely want the symbol-macros available at compile-time. So an attempt at the first macro would be (note I've fixed the handling of the docstring: (defparameter foo 1 nil) is bad):
(defmacro define-data (d body &optional doc)
(when (and doc (not (stringp doc)))
(error "Documentation is not a string"))
`(progn
(defparameter ,d ,body ,#(if doc (list doc) '()))
(eval-when (:compile-toplevel :load-toplevel :execute)
(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
(export ',d old-package)
(define-column-names ,d)))))
As a side note: although I think the fact that programmatically defining symbol macros is hard because CL left that out for some reason, I think I'd personally use some other approach rather than this, because eval is just so horrid. That's just me however: if you want to do this you do need eval I think (it is very rare that this is true!).
I am not sure exactly how define-columns-names works so I replaced it with a stub function that returns d.
Note also that you can use check-type and should try not injecting symbols in generated code, this introduces potential variable capture that can be avoided with gensym.
As far as I know you cannot use eval-when as suggested by your comment (see Issue EVAL-WHEN-NON-TOP-LEVEL Writeup for details).
But I have no warning if I declare the symbol as being special around the call.
(defmacro define-data (d body &optional doc)
(check-type doc (or null string))
(check-type d symbol)
(let ((d-str (string d)))
(alexandria:with-gensyms (old-package)
`(let* ((,old-package *package*)
(*package* (if (find-package ,d-str) ;exists?
(find-package ,d-str) ;yes, return it
(make-package ,d-str)))) ;no, make it
(defparameter ,d ,body ,doc)
(export ',d ,old-package)
(locally (declare (special ,d))
(define-column-names ,d))))))
It is also a bit strange that you expand into a call to define-column-names, which in turns evaluated a form built at runtime. I think it might be possible to do all you want during macroexpansion time, but as said earlier what you are trying to do is a bit unclear to me. What I have in mind is to replace define-column-names by:
,#(expand-column-names-macros d)
... where expand-column-names-macros builds a list of define-symbol-macro forms.
On first look it seems somewhat silly to have a function which returns the name of a symbol, which must be called using the name of that same symbol. i.e. it should already be obvious in the calling context what the function will return. On the other hand there is identity which is sometimes useful (I forget where just now) but I supposed (perhaps wrongly) that symbol-function is there for a reason other than simply to act as a kind of identity.
However, the hyperspec offers a possible hint:
(symbol-name 'temp) => "TEMP"
(symbol-name :start) => "START"
(symbol-name (gensym)) => "G1234" ;for example
I note that :start means get the name of the symbol start from the keyword package, where the keyword package is denoted simply by :.
(:keyword being its longer form, unnecessary to use). Thus, in this case symbol-name plays the role of simply removing the package prefix.
The other thing it might do is, given an implementation is case insensitive, it would get the actual name by removing case in the symbol name supplied.
Is that roughly it or is there any importance to this function I am missing?
One thing I was confused by about symbols (cleared up now) is that symbol-plist does not tell you everything about the symbol (say, whether it holds the value of a special variable or function). Rather, plist is a mainly legacy feature now largely replaced by hashtables. So, a call to symbol-plist is going to return NIL even if one has set a special variable on the symbol.
One final question on that, Paul Graham says in Chapter 8 (p133), that "You can use symbols as data objects and as names for things without understanding how the two are related". Would it be correct say that if we rarely now use plists, that, today, we generally don't use symbols "as data objects" at all, instead, just as names for things (allbeit with the duality in CL of course, i.e. symbol-function and symbol-value simultaneously).
Symbols are objects. You can create them and pass them around programmatically. One of the properties of these objects is their name, which is a string. Symbol-name is the reader (not accessor) for that. That's all.
Symbols are also used in the representation of code. In this case, they are created by the reader and used by the compiler. This is not their only use, however, and the spec makes no such assumptions either. I hope that this addresses the core of your question.
Here is a function that, given a symbol, returns a symbol interned in the same package but with a reversed name:
(defun reverse-symbol (symbol)
(intern (make-symbol (reverse (symbol-name symbol)))
(symbol-package symbol)))
Here is a function that, given a string designator, returns the designated string:
(defun designated-string (string-designator)
(ctypecase string-designator
(string string-designator)
(symbol (symbol-name string-designator))))
You can also do all kinds of shenanigans in macros and compiler macros by inspecting the names of the symbols in the given form and applying some conventions (not that you should…).
Let's assume you want to write some protocol for transmitting bits of structure between two different systems over some serial channel
(defgeneric transmit-object (o stream))
What is the method for symbols going to look like?
(defmethod transmit-object ((o symbol) stream)
... (symbol-name o) ...)
In particular you certainly do not know the name of the symbol at the point where you need to reason about it, and you could not write such a protocol without using symbol-name (or some absolutely appalling hack like printing the symbol to a string and using that).
Regarding symbol-plist, the implementation is completely at liberty to keep all sorts of information on the property list of a symbol. I am not even sure that the implementation is not allowed to do (a cleverer version of):
(defun symbol-value (s)
(let* ((d (load-time-value (cons nil nil)))
(v (get s secret-value-indicator d)))
(when (eq v d)
(error ...))
v))
It may be that this is not allowed, but I am not sure. Certainly it was once fairly common for function definitions to be kept in this way.
It's not so silly when you process symbol names. When you for example build preprocessor - you read some data, convert it to list structures and then some code is processing those list and trigger some additional logic when symbol-name is lets say defun-my-ubermacro. It's exactly what's in Graham quote - you treat symbols as data.
And (in my opinion) it's not correct to say, that when you don't use plists, you generally don't use symbols as data. Plists are only on of many places where it's useful feature. A lot of what macros do is processing symbols as data / names.
On first look it seems somewhat silly to have a function which returns the name of a symbol, which must be called using the name of that same symbol.
That's wrong. symbol-name is called with a symbol and not a name. It returns the name as a string.
I note that :start means get the name of the symbol start from the keyword package, where the keyword package is denoted simply by :. (:keyword being its longer form, unnecessary to use). Thus, in this case symbol-name plays the role of simply removing the package prefix.
No, symbol-name returns the name of the symbol as a string. The keyword symbol is an object of type symbol.
A symbol is a data type and has several cells:
the name, a string
possibly a function
possibly a value
a property list
optionally the home package it is interned in
don't use symbols "as data objects" at all, instead, just as names for things
No, symbols as data objects have many purposes. For example Lisp source code uses many symbols. But it can also appear in all sorts of data.
CL-USER 6 > 'foo
FOO
CL-USER 7 > (type-of 'foo)
SYMBOL
CL-USER 8 > (symbol-name 'foo)
"FOO"
CL-USER 9 > (type-of (symbol-name 'foo))
SIMPLE-BASE-STRING
CL-USER 10 > (mapcar #'symbol-name '(a b c defun foo bar))
("A" "B" "C" "DEFUN" "FOO" "BAR")
CL-USER 11 > (mapcar #'type-of *)
(SIMPLE-BASE-STRING SIMPLE-BASE-STRING SIMPLE-BASE-STRING SIMPLE-BASE-STRING SIMPLE-BASE-STRING SIMPLE-BASE-STRING)
Since you haven't accepted an answer, here is my take.
For most day-to-day programming symbols, and therefore SYMBOL-NAME,
aren't very useful. Mostly they are used for their
unique-ness. However they shine when you are modifying the compiler
with macros.
Here are 3 examples where SYMBOL-NAME is used:
Firstly, LOOP is the CL generic looping construct, it works via
placeholder symbols in a way that some claim is un-lispy, but it
remains very handy, especially if you are stepping things in parallel.
The following 3 forms are equivalent (at run-time):
(loop for i in list collect i)
(loop :for i :in list :collect i)
(loop #:for i #:in list #:collect i)
I, personally, prefer the third form because it makes it really
obvious where the LOOP magic is happening, and also avoids interning
symbols in any package (which is usually harmless, but not
always). That the third works at all requires the existence of
SYMBOL-NAME
Secondly, I don't make much use of CLOS but classes are undeniably
useful. DEFCLASS tends to be too verbose for what I want to do though,
so I often employ a macro that uses implied symbols. So:
(defbean peer ()
(id)
(next
beats))
Becomes:
(defclass peer nil
((id :initarg :id :reader peer-id)
(next :initarg :next :accessor peer-next)
(beats :initarg :beats :accessor peer-beats)))
Saving much typing. Notice how the slot name is converted to a keyword
for the initargs, and how reader and accessor names are created.
Some don't like this sort of macro, and I can see how it might be
problematic if you have a large team with everyone doing this all over
the shop (though M-. and C-c ret are always available), but for
small teams I think this is one of the best reasons to use lisp,
customising it to how you actually want to use it.
Thirdly, my sqlite helper library uses SYMBOL-NAME to generate SQL
statements (along with some more implied symbols) for even more
code-saving:
(defsqlite-table artists
(id :auto-key)
(artist-name :t)
(sort-artist-name :t :n))
Becomes something pretty huge:
(progn
(defparameter +create-artists-sql+
"CREATE TABLE artists (
id INTEGER PRIMARY KEY NOT NULL,
artist_name TEXT NOT NULL,
sort_artist_name TEXT
)")
(defun create-artists-table (pjs-sqlite::db pjs-sqlite::recursive log)
###)
(eval-when (:load-toplevel)
###)
(defun insert-artists
(pjs-sqlite::db artist-name sort-artist-name &key (id nil id-p))
###)
(defun update-artists
(pjs-sqlite::db id
&key (artist-name nil artist-name-p)
(sort-artist-name nil sort-artist-name-p))
###)
(defun delete-artists (pjs-sqlite::db id)
(with-sqlite-statements (pjs-sqlite::db
(pjs-sqlite::stmt
"DELETE FROM artists WHERE id = ?"))
###)))
(I stripped out a lot of code that would be distracting, but it should
be clear how I mapped symbols to sql identifiers)
I hope this gives you some ideas on what sort of things SYMBOL-NAME
can be used for.
My lisp file contains about 50 function and macro definitions. At the head of the file is:
(defpackage :utils
(:use :common-lisp))
(in-package :utils)
; ... ~50 defuns/defmacros follow along with some other code
I want to export all these 50 definitions. But no other symbols.
To write them all out in :export clauses inside defpackage seems very tedious.
I tried getting the necessary symbols by using loop's facility for listing symbols in a package. But both "symbol" and "present-symbol" don't output the needed definition symbols. I could somehow subtract "external-symbols" and inherited symbols. But this seems like the kind of thing everyone would need and there should be an established easy solution. I looked around and haven't found a straightforward answer.
How do I export all the definition symbols without having to write them out individually and manually?
scan symbols
You can scan all symbols and export all that have a function binding (this includes macro names):
(defun export-fbound-symbols (package)
"export symbols with function bindings"
(do-symbols (s package)
(when (fboundp s)
(export s package))))
This is usually not a very good idea: good software engineering practice is to limit the set of public interfaces.
be more selective
You can redefine defun and defmacro so that they automatically export their definienda.
See exporting.lisp in CLISP. E.g.,
(defmacro define-function (name lambda-list &body body)
`(progn
(export ',name)
(defun ,name ,lambda-list ,#body)))
Note that this ignores the possibility of (defun (setf foo) (...) ...).
No, this is not something everyone needs. Usually, you'd export them as you write (and actually need) them.
I'd probably use an ad hoc keyboard macro for this. You could also write a little Lisp function to read your file form by form and output the relevant symbols for copy-paste.
I'm reviving an old LISP program from the early 1980s.
(It's the Nelson-Oppen simplifier, an early proof system.
This version was part of the Ford Pascal-F Verifier,
and was running in Franz LISP in 1982.) Here's
the entire program:
https://github.com/John-Nagle/pasv/tree/master/src/CPC4
I'm converting the code to run under clisp on Linux,
and need some advice. Most of the problems are with
macros.
HUNKSHELL
Hunkshell was a 1970s Stanford SAIL hack to support records with named fields in LISP. I think I've converted this OK; it seems to work.
https://github.com/John-Nagle/pasv/blob/master/src/CPC4/hunkshell.l
The original macro generated more macros as record update functions.
I'm generating defuns. Is there any reason to generate macros?
By the way, look at what I wrote for "CONCAT". Is there a better way
to do that?
DEFMAC
More old SAIL macros, to make macro definition easier before defmacro became part of the language.
https://github.com/John-Nagle/pasv/blob/master/src/CPC4/defmac.l
I've been struggling with "defunobj". Here's my CL version, partly converted:
; This macro works just like defun, except that both the value and the
; function-binding of the symbol being defined are set to the function
; being defined. Therefore, after (defunobj f ...), (f ...) calls the
; defined function, whereas f evaluates to the function itself.
;
(defmacro defunobj (fname args &rest b)
`(progn
(defun ,fname ,args ,b)
;;;;(declare (special ,fname)) ;;;; ***declare not allowed here
(setq ,fname (getd ',fname))))
If I made that declare a proclaim, would that work right?
And what replaces getd to get a function pointer?
SPECIAL
There are lots of (DECLARE (SPECIAL FOO)) declarations at top
level in this code. That's not allowed in CL. Is it
appropriate to use (PROCLAIM (SPECIAL FOO)) instead?
Concat
Essentially correct, but indentation is broken (like everywhere else - I suggest Emacs to fix it).
Also, you don't need the values there.
defunobj
I suggest defparameter instead
of setq. Generally speaking, before setting a variable (with, e.g., setq) one should establish it (with, e.g., let or defvar).
fdefinition is what
you are looking for instead of getd.
I also don't think you are using backquote right:
(defmacro defunobj (fname &body body)
`(progn
(defun ,fname ,#body)
(defparameter ,fname (fdefinition ',fname))))
Special
I think defvar and defparameter
are better than proclaim
special.
PS. Are you aware of the CR.se site?
The other day (perhaps yesterday) I was quite perplexed about this #+nil read-time conditional found in https://github.com/billstclair/defperson/blob/master/defperson.lisp#L289.
After some deep thinking I came to the conclusion that this is very lispy way of commenting out code. Can someone confirm this?
Perhaps my assumptions are completely wrong. Anyway, thanks in advance.
Yes, it is a lispy way of commenting code, but you shouldn't leave this out in production code.
A better alternative is #+(or).
It only takes one more character, it takes the same key presses if you use Emacs paredit or some other mode that automatically inserts the closing parenthesis, and it's not subject to the existence of the symbol :nil in *features*.
See CLHS 2.4.8.17 Sharpsign Plus
To conditionalize reading expressions from input, Common Lisp uses feature expressions.
In this case it has been used to comment out a form.
It's a part of the reader. #+ looks if the next item, usually as a keyword symbol with the same name, is a member of the list *features*. If yes, the then next item is read as normal, if not, it is skipped.. Usually :NIL is not a member of that list, so the item is skipped. Thus it hides the expression from Lisp. There might have been a Lisp implementation, where this would not work : NIL, New Implementation of Lisp. It might have had the symbol :NIL on the *features* list, to indicate the name of the implementation.
Features like NIL are by default read in the keyword package:
#+NIL -> looks for :NIL in cl:*features*
#+CL:NIL -> looks for CL:NIL in cl:*features*
Example
(let ((string1 "#+nil foo bar")) ; string to read from
(print (read-from-string string1)) ; read from the string
(let ((*features* (cons :nil *features*))) ; add :NIL to *features*
(print (read-from-string string1))) ; read from the string
(values)) ; return no values
It prints:
BAR
FOO
Note that Common Lisp has other ways to comment out forms:
; (sin 3) should we use that?
#| (sin 3) should we use that?
(cos 3) or this? |#