Define a constant array of struct known at compilation-time - common-lisp

Question

In my program I have constant strings, the values are known at compilation time. For each offset there are currently 2 associated strings. I first wrote the following code:
(eval-when (:compile-toplevel :load-toplevel :execute) ;; BLOCK-1
(defstruct test-struct
str-1
str-2))
(eval-when (:compile-toplevel) ;; BLOCK-2
(defparameter +GLOBAL-VECTOR-CONSTANT+ nil) ;; ITEM-1
(let ((vector (make-array 10
:initial-element (make-test-struct)
:element-type 'test-struct)))
(setf (test-struct-str-1 (aref vector 0)) "test-0-1")
(setf (test-struct-str-2 (aref vector 0)) "test-0-2")
(setf +GLOBAL-VECTOR-CONSTANT+ vector)))
(format t "[~A]~%" (test-struct-str-1 (elt +GLOBAL-VECTOR-CONSTANT+ 0)))
(format t "[~A]~%" (test-struct-str-2 (elt +GLOBAL-VECTOR-CONSTANT+ 0)))
This seems to work as it returns the following:
[test-2-1]
[test-2-2]
In BLOCK-1 the struct containing the data is defined, for compile-time, load-time and execute-time. In BLOCK-2, the code which create a vector and sets the values is executed, at compile-time.
But I have the following concerns:
This code seems unnecessary verbose
The strings are stored in a structure
I need to manually set the offset of each values ((aref vector 0), (aref vector 1), etc).
When I set ITEM-1 inside BLOCK-1 instead of BLOCK-2 I get an error in SBCL which I don't understand
What is the idiomatic way to define complex constants in Common Lisp?

Answer 1

It's not really clear what you want to do from your question.
First important note: your code is seriously broken. It's broken because you define +global-vector-constant+ only at compile time but refer to it later than that. If you compile this file and then load that compiled file into a cold image you will get errors.
It is absolutely critical when dealing with things like this to make sure that your code will compile in a cold Lisp. One of the classic problems with resident environments (which CL isn't really, compared to the way Interlisp-D was for instance) is to end up with systems which you can't cold build: I'm pretty sure I worked for several years with an Interlisp-D sysout that no-one knew how to cold build any more.
If what you want is an object (an array, for instance) whose initial value is computed at compile time and then treated as a literal, then the answer to that is, in general, a macro: macros are exactly functions which do their work at compile time, and so a macro can expand to a literal. In addition it must be the case that the object you want to be a literal is externalizable (which means 'can be dumped in compiled files') and anything involved in it is known about at compile time. Instances of some classes are externalizable by default, those of some other classes can be made externalizable by user code, and some are not externalizable at all (for instance functions).
In quite a lot of simple cases, like the one you gave, if I understand it, you don't really need a macro, and in fact you can almost always get away without one, although it may make your code easier to understand if you do use one.
Here is a simple case: many arrays are externalizable if their elements are
(defparameter *my-strings*
#(("0-l" . "0-r")
("1-l" . "1-r")))
This means that *my-strings* will be bound to a literal array of conses of strings.
A more interesting case is when the elements are, for instance structures. Well, structures are also externalizable, so we can do that. And in fact it's quite possible, still, to avoid a macro, although it now becomes a bit noisy.
(eval-when (:compile-toplevel :load-toplevel :execute)
(defstruct foo
l
r))
(defparameter *my-strings*
#(#s(foo :l "0-l" :r "0-r")
#s(foo :l "1-l" :r "1-r")))
Note that the following won't work:
(defstruct foo
l
r)
(defparameter *my-strings*
#(#s(foo :l "0-l" :r "0-r")
#s(foo :l "1-l" :r "1-r")))
It won't work because, at compile time, you are trying to externalize instances of a structure which is not yet defined (but it probably will work if the Lisp is not cold, and you might even be able to reload the compiled file you made that way). Again, in this case you can avoid the eval-when in a larger system by ensuring that the file which defines the foo structure is compiled and loaded before the file with the defparameter is loaded.
And even in more complex cases you can escape using a macro. For instance for many sorts of objects which are normally not externalizable you can teach the system how to externalize them, and then splice the object in as a literal using #.:
(eval-when (:compile-toplevel :load-toplevel :execute)
;; Again, this would be in its own file in a bigger system
(defclass string-table-wrapper ()
((strings)
(nstrings :initform 0)))
(defmethod initialize-instance :after ((w string-table-wrapper)
&key (strings '()))
(let ((l (length strings)))
(when l
(with-slots ((s strings) (n nstrings)) w
(setf s (make-array l :initial-contents strings)
n l)))))
(defmethod make-load-form ((w string-table-wrapper) &optional environment)
(make-load-form-saving-slots w :slot-names '(strings nstrings)
:environment environment))
) ;eval-when
(defgeneric get-string (from n)
(:method ((from string-table-wrapper) (n fixnum))
(with-slots (strings nstrings) from
(assert (< -1 n nstrings )
(n)
"bad index")
(aref strings n))))
(defparameter *my-strings*
#.(make-instance 'string-table-wrapper
:strings '("foo" "bar")))
Note that, of course, although the value of *my-strings* is a literal, code ran to reconstruct this object at load-time. But that is always the case: it's just that in this case you had to define what code needed to run. Instead of using make-load-form-saving-slots you could have done this yourself, for instance by something like this:
(defmethod make-load-form ((w string-table-wrapper) &optional environment)
(declare (ignore environment))
(if (slot-boundp w 'strings)
(values
`(make-instance ',(class-of w))
`(setf (slot-value ,w 'strings)
',(slot-value w 'strings)
(slot-value ,w 'nstrings)
,(slot-value w 'nstrtrings)))
`(make-instance ',(class-of w))))
But make-load-form-saving-slots is much easier.
Here is an example where a macro does perhaps least make reading the code easier.
Let's assume you have a function which reads an array of strings from a file, for instance this:
(defun file-lines->svector (file)
;; Needs CL-PPCRE
(with-open-file (in file)
(loop
with ltw = (load-time-value
(create-scanner '(:alternation
(:sequence
:start-anchor
(:greedy-repetition 1 nil
:whitespace-char-class))
(:sequence
(:greedy-repetition 1 nil
:whitespace-char-class)
:end-anchor)))
t)
for nlines upfrom 0
for line = (read-line in nil)
while line
collect (regex-replace-all ltw line "") into lines
finally (return (make-array nlines :initial-contents lines)))))
Then, if this function is available at macroexpansion time, you could write this macro:
(defmacro file-strings-literal (file)
(check-type file (or string pathname) "pathname designator")
(file-lines->svector file))
And now we can create a literal vector of strings:
(defparameter *fl* (file-strings-literal "/tmp/x"))
However you could perfectly well instead do this:
(defparameter *fl* #.(file-lines->svector "/tmp/x"))
Which will do the same thing, but slightly earlier (at read time, rather than at macroexpansion/compile time). So this is gaining nothing really.
But you could also do this:
(defmacro define-stringtable (name file &optional (doc nil docp))
`(defparameter ,name ,(file-lines->svector file)
,#(if docp (list doc) nil)))
And now your code reads like
(define-stringtable *st* "my-stringtable.dat")
And that actually is a significant improvement.
Finally note that in file-lines->svector that load-time-value is used to create the scanner exactly once, at load time, which is a related trick.

Answer 2

First of all, your let code can be simplified to
(defparameter +global-vector-constant+
(let ((vector ...))
...
vector))
Second, you can also do
(defparameter +global-vector-constant+
(make-array 10 :element-type 'test-struct :initial-content
(cons (make-test-struct :str-1 "test-0-1" :str-2 "test-0-2")
(loop :repeat 9 :collect (make-test-struct)))))
Note that the benefit of :element-type 'test-struct is generally limited to code self-documentation (see upgraded-array-element-type)