Implicitly defined variables in Lisp and symbol tables - common-lisp

Question

Suppose one introduces variables in a fresh Common Lisp
at the REPL and one types: (setq q 2).
I understand from these columns that this variable q is
not defined following the Common Lisp standard and
depends upon the implementation.
My question is: what is the easiest way or test to be
sure what it is exactly?
I read in one source that q is then automatically an implicitly defined
global dynamic variable and is then equivalent to
(defpar q 2).
In relation with this question. Seasoned Lisp
programmers talk much about the symbol tables. I do
not find in e.g. Seibel how to find out what is in
those tables. Can one access these tables? Do
debuggers support accessing these tables in a non
standard way?

Answer 1

Using SETQ rather than DEFPARAMETER is likely to make a global variable, but not a special variable. That will cause annoying debugging later. Don't use SETQ to define variables.
An example with lengthy code snippets. I'm using non-standard SBCL for this.
Let's define a package with two variables, one defined with DEFPARAMETER and one set with SETQ.
CL-USER> (defpackage :foo (:use :cl))
#<PACKAGE "FOO">
CL-USER> (in-package :foo)
#<PACKAGE "FOO">
FOO> (defparameter q 2)
Q
FOO> (setq w 2)
; in: SETQ W
; (SETQ FOO::W 2)
;
; caught WARNING:
; undefined variable: W
;
; compilation unit finished
; Undefined variable:
; W
; caught 1 WARNING condition
2 (2 bits, #x2, #o2, #b10)
FOO> q
2 (2 bits, #x2, #o2, #b10)
FOO> w
2 (2 bits, #x2, #o2, #b10)
The warning message already tells us that SBCL doesn't like the SETQ option, but the variable seems to work. Let's try to DESCRIBE the variables:
FOO> (describe 'q)
FOO::Q
[symbol]
Q names a special variable:
Value: 2
; No values
FOO> (describe 'w)
FOO::W
[symbol]
W names an undefined variable:
Value: 2
; No values
This says that Q is a special variable, while W is an undefined variable.
FOO> (sb-cltl2:variable-information 'q)
:SPECIAL
NIL
NIL
FOO> (sb-cltl2:variable-information 'w)
NIL
NIL
NIL
This also confirms that W is not a special variable like Q is. So what does this mean? Let's define a function that uses those variables:
FOO> (defun foobar ()
(format t "~&Q: ~a~%W: ~a~%" q w))
; in: DEFUN FOOBAR
; (FORMAT T "~&Q: ~a~%W: ~a~%" FOO::Q FOO::W)
;
; caught WARNING:
; undefined variable: W
;
; compilation unit finished
; Undefined variable:
; W
; caught 1 WARNING condition
FOOBAR
FOO> (foobar)
Q: 2
W: 2
NIL
Again we get warnings about W, but the code still seems to work. Let's try to shadow the variables.
FOO> (defun quux ()
(let ((q 100)
(w 100))
(foobar)))
; in: DEFUN QUUX
; (LET ((FOO::Q 100) (FOO::W 100))
; (FOO::FOOBAR))
;
; caught STYLE-WARNING:
; The variable W is defined but never used.
;
; compilation unit finished
; caught 1 STYLE-WARNING condition
QUUX
FOO> (quux)
Q: 100
W: 2
NIL
Now we notice that since W is not special, you can't shadow it. Also
FOO> (sb-introspect:who-binds 'q)
((QUUX
. #S(SB-INTROSPECT:DEFINITION-SOURCE
:PATHNAME NIL
:FORM-PATH (0 3 2)
:FORM-NUMBER 0
:CHARACTER-OFFSET 0
:FILE-WRITE-DATE NIL
:PLIST NIL
:DESCRIPTION NIL)))
FOO> (sb-introspect:who-binds 'w)
NIL
We can't see who binds the variable. Or who sets either:
FOO> (defun qwerty ()
(setf w 1000
q 1000))
; in: DEFUN QWERTY
; (SETF FOO::W 1000
; FOO::Q 1000)
; --> PROGN SETF
; ==>
; (SETQ FOO::W 1000)
;
; caught WARNING:
; undefined variable: W
;
; compilation unit finished
; Undefined variable:
; W
; caught 1 WARNING condition
QWERTY
FOO> (qwerty)
1000 (10 bits, #x3E8)
FOO> (sb-introspect:who-sets 'q)
((QWERTY
. #S(SB-INTROSPECT:DEFINITION-SOURCE
:PATHNAME NIL
:FORM-PATH (0 3 2)
:FORM-NUMBER 0
:CHARACTER-OFFSET 0
:FILE-WRITE-DATE NIL
:PLIST NIL
:DESCRIPTION NIL)))
FOO> (sb-introspect:who-sets 'w)
NIL
Since you also asked about symbol tables, the easiest way to see what's in it is to INSPECT a package. You can do that with your IDE (in Slime C-c I) or by calling the function directly:
FOO> (inspect (find-package :foo))
The object is a STRUCTURE-OBJECT of type PACKAGE.
0. %NAME: "FOO"
1. %NICKNAMES: NIL
2. %USE-LIST: (#<PACKAGE "COMMON-LISP">)
3. TABLES: #(#<SB-INT:PACKAGE-HASHTABLE
(978+0)/1973 [2.270 words/sym,load=49.6%] {100001A483}>)
4. MRU-TABLE-INDEX: 0
5. %USED-BY-LIST: NIL
6. INTERNAL-SYMBOLS: #<SB-INT:PACKAGE-HASHTABLE (7+0)/17 [2.732 words/sym,load=41.2%] {1006D60AE3}>
7. EXTERNAL-SYMBOLS: #<SB-INT:PACKAGE-HASHTABLE (0+0)/3 [load=0.0%] {1006D60B13}>
8. %SHADOWING-SYMBOLS: NIL
9. DOC-STRING: NIL
10. LOCK: NIL
11. %IMPLEMENTATION-PACKAGES: (#<PACKAGE "FOO">)
12. SOURCE-LOCATION: #S(SB-C:DEFINITION-SOURCE-LOCATION
:NAMESTRING NIL
:TOPLEVEL-FORM-NUMBER NIL
:FORM-NUMBER NIL
:PLIST NIL)
13. %LOCAL-NICKNAMES: NIL
14. %LOCALLY-NICKNAMED-BY: NIL
> 6
The object is a STRUCTURE-OBJECT of type SB-INT:PACKAGE-HASHTABLE.
0. CELLS: #(FOOBAR 0 QUUX ? 0 0 0 E W 0 Q QWERTY 0 0 0 0 0
#(21 0 98 59 0 0 0 223 135 0 193 37 0 0 0 0 0))
1. SIZE: 12
2. FREE: 5
3. DELETED: 0
> 0
The object is a VECTOR of length 18.
0. FOOBAR
1. 0
2. QUUX
3. ?
4. 0
5. 0
6. 0
7. E
8. W
9. 0
10. Q
11. QWERTY
12. 0
13. 0
14. 0
15. 0
16. 0
17. #(21 0 98 59 0 0 0 223 135 0 193 37 0 0 0 0 0)
> 8
The object is a SYMBOL.
0. Name: "W"
1. Package: #<PACKAGE "FOO">
2. Value: 1000
3. Function: "unbound"
4. Plist: NIL
> u
The object is a VECTOR of length 18.
0. FOOBAR
1. 0
2. QUUX
3. ?
4. 0
5. 0
6. 0
7. E
8. W
9. 0
10. Q
11. QWERTY
12. U
13. 0
14. 0
15. 0
16. 0
17. #(21 0 98 59 0 0 0 223 135 0 193 37 201 0 0 0 0)
> 10
The object is a SYMBOL.
0. Name: "Q"
1. Package: #<PACKAGE "FOO">
2. Value: 1000
3. Function: "unbound"
4. Plist: NIL
> q
You could also use DO-SYMBOLS to loop over symbols in a package.
FOO> (do-symbols (symbol)
(when (and (boundp symbol)
(eq (symbol-package symbol) *package*))
(format t "~&~a~% Value: ~a~% Info: ~a~% Who sets: ~a~% ~
Who binds: ~a~% Plist: ~a~% Documentation: ~a~%~%"
symbol
(symbol-value symbol)
(multiple-value-list
(sb-cltl2:variable-information symbol))
(sb-introspect:who-sets symbol)
(sb-introspect:who-binds symbol)
(symbol-plist symbol)
(documentation symbol 'variable))))
Q
Value: 1000
Info: (SPECIAL NIL NIL)
Who sets: ((QWERTY
. #S(SB-INTROSPECT:DEFINITION-SOURCE
:PATHNAME NIL
:FORM-PATH (0 3 2)
:FORM-NUMBER 0
:CHARACTER-OFFSET 0
:FILE-WRITE-DATE NIL
:PLIST NIL
:DESCRIPTION NIL)))
Who binds: ((QUUX
. #S(SB-INTROSPECT:DEFINITION-SOURCE
:PATHNAME NIL
:FORM-PATH (0 3 2)
:FORM-NUMBER 0
:CHARACTER-OFFSET 0
:FILE-WRITE-DATE NIL
:PLIST NIL
:DESCRIPTION NIL)))
Plist: NIL
Documentation: NIL
W
Value: 1000
Info: (NIL NIL NIL)
Who sets: NIL
Who binds: NIL
Plist: NIL
Documentation: NIL