Sounds deceptively easy. This doesn't work:
~/.sbclrc
(load #P"~/in-package.lisp")
~/in-package.lisp
(in-package 'stored-package)
The package change only applies to the file in-package.lisp itself.
Try different approach: store just the name.
(defmacro recall-package (&optional (filename #p"~/lisp-package.lisp"))
"IN-PACKAGE the contents of the FILENAME"
(let ((p (car (uiop:read-file-lines filename))))
`(in-package ,p)))
This works, but only from ~/.sbclrc. Files which it LOADs expand the macro within their own context, and so it doesn't work.
SBCL reads it's .sbclrc like this:
(restart-case
(flet ((process-init-file (kind specified-pathname default-function)
(awhen (or specified-pathname (funcall default-function))
(with-open-file (stream (if specified-pathname
(parse-native-namestring it)
(pathname it))
:if-does-not-exist nil)
(cond (stream
(sb-fasl::call-with-load-bindings
(lambda (stream kind) (load-as-source stream :context kind))
stream kind stream))
(specified-pathname
(cerror "Ignore missing init file"
"The specified ~A file ~A was not found."
kind specified-pathname)))))))
(unless no-sysinit
(process-init-file "sysinit" sysinit *sysinit-pathname-function*))
(unless no-userinit
(process-init-file "userinit" userinit *userinit-pathname-function*))
Using these fancy sb-fasl::call-with-load-bindings and sb-int:load-as-source yields similar results to the above.
You can't do this with load, because
load binds *readtable* and *package* to the values they held before loading the file.
Function load
This means that any changes made to the values of these variables within a file being loaded are local to the file. That's almost always a good thing. In particular it means that there is no way at all (or no portable way: if you had access to the guts of the dynamic binding mechanism of the implementation this might not be true) that any changes made to the current package (ie the dynamic value of *package*) can ever propagate up through calls to load.
If all you want to do is set the package based on some name in a file, then this is relatively easy, with something like the below:
(defpackage :empty-package
(:use))
(defun set-package-from-file (f)
(let ((pn
(with-standard-io-syntax
;; EP just in case the file does somehow manage to smash
;; *package*)
(let* ((ep (find-package :empty-package))
(*package* ep)
(*read-eval* nil))
(unwind-protect
(with-open-file (in f)
(string (read in)))
;; Clean up EP to avoid leakage
(do-symbols (s ep)
(unintern s ep)))))))
(let ((p (find-package pn)))
(unless p (error "no package ~A" pn))
(setf *package* p))))
This is probably both overly-protective and thus will contain some horrible unexpected bug which I should have thought about (I know it's not safe against interning symbols in other packages). However the idea is that the file contains a single string-designator which should be the package name.
If you had time on your hands you could fairly easily write a version of load which would not rebind *package* &c, and which would work for source files. I think you can't portably write one which would work for FASL files.
Here's one reason why the behaviour the language specifies is the right behaviour: it makes compilation a lot easier. Consider a file which contains:
(in-package ...)
(defun foo (...) ...)
(load ...)
(defun bar (...)
(foo ...)
...)
If *package* could propagate up through load then compiling this file would be, at best, interesting.
Related
Is there any standard way in Common Lisp (+common libraries today) how to put default settings for not-yet-loaded libraries/systems (that are from public sources and cannot be reasonably modified) in the config file such as .sbclrc?
For example, I want to have set
(setf quickproject:*template-directory* "~/projects/template/"
quickproject:*author* "my name <etcetc>")
when Quickproject is loaded. (Obviously, there is an additional complication with the relevant package not being defined at that point of time, but this I can handle.)
Something along the lines of emacs's (with)-eval-after-load.
I am interested primarily in sbcl and quicklisp/asdf libraries, but implementation independent solution is preferred.
I tried to find in existing libraries and out of the box functionalities of SBCL require and of asdf, without success.
I considered and rejected abusing sbcl's extensions on trace.
The best I was able to do so far is define method for asdf:operate, something like (for illustration, not really used)
(defvar *after-system-hooks* (make-hash-table :test 'equal))
(defmethod asdf:operate :after ((op asdf:load-op) (component asdf:system) &key)
(let* ((name (asdf:component-name component)))
(dolist (hook (gethash name *after-system-hooks*))
(funcall hook))
(setf (gethash name *after-system-hooks*) nil)))
(defun intern-uninterned (code package)
(typecase code
(cons (cons (intern-uninterned (car code) package)
(intern-uninterned (cdr code) package)))
(symbol (if (symbol-package code) code (intern (symbol-name code) package)))
(t code)))
(defmacro with-eval-after-load ((name package) &body body)
`(let ((fn (lambda () (eval (intern-uninterned '(progn ,#body) ',package)))))
(if (member ,name (asdf:already-loaded-systems) :test 'equal)
(funcall fn)
(push fn (gethash ,name *after-system-hooks*)))))
and then write something like
(with-eval-after-load ("quickproject" :quickproject)
(setf
#:*template-directory* "~/projects/template/"
#:*author* "My Name <etcetc>"))
However, this is overwriting method on standard component dispatched on standard classes, so it does not seem safe and future-proof. And I do not see how to define new classes to specialize in such a way that it would work with existing systems and tooling.
(And it has eval, but it does not bother me much here)
You are on the right track with asdf:operate. But instead of specializing on asdf:system in general, use EQL Specializer:
(defmethod asdf:operate :after ((op asdf:load-op) (component (eql (asdf:find-system "quickproject"))) &key)
... )
This method only gets called after loading Quickproject.
I would also get rid of extra abstractions and just do want I want to do in the method itself. There is uiop:symbol-call to call a function which isn't loaded yet. But I wouldn't mind using eval with a fixed string.
To use this method (and similar custom forms) from multiple implementations, just put this in a shared-init.lisp file and then load this file in each implementation's init file.
As for common libraries for settings and configs for your own code, I have used https://github.com/Shinmera/ubiquitous before, which worked without any issues.
This question is somewhat related to an earlier one on programmatically generating symbol macros. I'm using that function in a convenience macro that throws undefined variable warnings. This macro and function:
(defmacro define-data (d body &optional doc)
(if (and doc (not (stringp doc))) (error "Documentation is not a string"))
`(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
;; Should we have an eval-when (:compile-toplevel) here?
(defparameter ,d ,body ,doc)
(export ',d old-package)
(define-column-names ,d)))
(defun define-column-names (d)
(maphash #'(lambda (key index)
(eval `(cl:define-symbol-macro ,key (cl:aref (columns ,d) ,index))))
(ordered-keys-table (slot-value d 'ordered-keys))))
are intended to be like defparameter, but additionally set up a few niceties for the user by defining:
a package with the name of d
a parameter in the current package with the data that will be sucked in by body
symbol-macros in package d for access to the individual data vectors
If I use defparameter from the REPL, and then call define-column-names, all is well. However when using the macro I get:
; in: DEFINE-COLUMN-NAMES FOO
; (DEFINE-COLUMN-NAMES CL-USER::FOO)
;
; caught WARNING:
; undefined variable: CL-USER::FOO
I suspect that this is because the compiler has no way of knowing that FOO will actually be defined when define-symbol-macro is called. Everything works fine, but I don't want the warning to frighten users, so am thinking of suppressing it. I hate suppressing warnings though, so thought I'd come here for a second opinion.
EDIT: I've marked an answer correct because it does correctly answer the question as asked. For an answer to the problem see my comments.
My answer to the 'when to muffle warnings' question in the title is: if it's your own code then never, under any circumstances. If it is someone else's code, then rewrite it not to warn unless you can't.
As to solving the problem I haven't thought about this hard enough, but the problem is that you definitely want the defparameter to be at top-level so the compiler can see it, and it can't really be if it's inside a let. But you can raise it to toplevel trivially since it depends on nothing inside the let.
I am then pretty certain that you want the rest of the macro to happen at compile time, because you definitely want the symbol-macros available at compile-time. So an attempt at the first macro would be (note I've fixed the handling of the docstring: (defparameter foo 1 nil) is bad):
(defmacro define-data (d body &optional doc)
(when (and doc (not (stringp doc)))
(error "Documentation is not a string"))
`(progn
(defparameter ,d ,body ,#(if doc (list doc) '()))
(eval-when (:compile-toplevel :load-toplevel :execute)
(let* ((d-str (string ',d))
(old-package *package*)
(*package* (if (find-package d-str) ;exists?
(find-package d-str) ;yes, return it
(make-package d-str)))) ;no, make it
(export ',d old-package)
(define-column-names ,d)))))
As a side note: although I think the fact that programmatically defining symbol macros is hard because CL left that out for some reason, I think I'd personally use some other approach rather than this, because eval is just so horrid. That's just me however: if you want to do this you do need eval I think (it is very rare that this is true!).
I am not sure exactly how define-columns-names works so I replaced it with a stub function that returns d.
Note also that you can use check-type and should try not injecting symbols in generated code, this introduces potential variable capture that can be avoided with gensym.
As far as I know you cannot use eval-when as suggested by your comment (see Issue EVAL-WHEN-NON-TOP-LEVEL Writeup for details).
But I have no warning if I declare the symbol as being special around the call.
(defmacro define-data (d body &optional doc)
(check-type doc (or null string))
(check-type d symbol)
(let ((d-str (string d)))
(alexandria:with-gensyms (old-package)
`(let* ((,old-package *package*)
(*package* (if (find-package ,d-str) ;exists?
(find-package ,d-str) ;yes, return it
(make-package ,d-str)))) ;no, make it
(defparameter ,d ,body ,doc)
(export ',d ,old-package)
(locally (declare (special ,d))
(define-column-names ,d))))))
It is also a bit strange that you expand into a call to define-column-names, which in turns evaluated a form built at runtime. I think it might be possible to do all you want during macroexpansion time, but as said earlier what you are trying to do is a bit unclear to me. What I have in mind is to replace define-column-names by:
,#(expand-column-names-macros d)
... where expand-column-names-macros builds a list of define-symbol-macro forms.
While trying to work through cl-sdl2-tutorial, I'm having trouble loading a bitmap due to a wrong current working directory.
I'd like to get a proper solution to the problem using relative path names.
A minimal working example:
Having modified the code of example two from above mentioned tutorial.
(defpackage #:sdl2-tutorial-2
(:use :common-lisp)
(:export :main))
(in-package :sdl2-tutorial-2)
(defparameter *screen-width* 640)
(defparameter *screen-height* 480)
(defmacro with-window-surface ((window surface) &body body)
`(sdl2:with-init (:video)
(sdl2:with-window (,window
:title "SDL2 Tutorial"
:w *screen-width*
:h *screen-height*
:flags '(:shown))
(let ((,surface (sdl2:get-window-surface ,window)))
,#body))))
(defun main(&key (delay 2000))
(format t " cwd: ~a, ~% dpd: ~a, ~& e-p: ~a, ~% pf: ~a, ~& load: ~a"
(sb-posix:getcwd)
*default-pathname-defaults*
(uiop:file-exists-p "hello_world.bmp")
(probe-file "hello_world.bmp")
(sdl2:load-bmp "hello_world.bmp"))
(with-window-surface (window screen-surface)
(let ((image (sdl2:load-bmp "hello_world.bmp")))
(break "1 here with ~a~%" image)
(setf image (sdl2:load-bmp "hello_world.bmp"))
(break "2 here with ~a~%" image)
(break "3 error: ~a~%" (sdl2-ffi.functions:sdl-get-error))
(sdl2:blit-surface image
nil
screen-surface
nil)
(sdl2:update-window window)
(sdl2:with-event-loop (:method :poll)
(:quit () t)
(:idle ()
(sdl2:delay delay))))))
Before compiling above code and running (main), I changed working directory in the REPL, via:
(sb-posix:chdir (truename "/test/cl-sdl2-tutorial/2/"))
(setf *default-pathname-defaults* (truename "/test/cl-sdl2-tutorial/2/"))
The above code prints, as expected, when running (main) in the REPL:
SDL2-TUTORIAL-2> (sdl2-tutorial-2:main)
0: (SDL2-TUTORIAL-2:MAIN)
cwd: /test/cl-sdl2-tutorial/2,
dpd: /test/cl-sdl2-tutorial/2/,
e-p: /test/cl-sdl2-tutorial/2/hello_world.bmp,
pf: /test/cl-sdl2-tutorial/2/hello_world.bmp,
load: #<SDL-SURFACE {#X7F5CBC018DD0}>
Problem:
The bitmap can not be found and therefore not loaded.
Calls to (sdl2:load-bmp "hello_world.bmp") always return a a zero pointer (#<SDL-SURFACE {#X00000000}>) and breakpoint 3 states:
3 error: Couldn't open /home/jue/hello_world.bmp
but evaling (sdl2:load-bmp "hello_world.bmp") during a break from breakpoints 1 or 2 or 3, is successful and continuing (main) displays the picture.
Questions:
Why is sdl2:load-bmp using the "wrong" working directory and why is it using the "correct" working directory during breakpoints?
How to make sdl2:load-bmp use the wanted working directory (instead of "/home/jue/") when using relative paths?
Remarks
(I'm using current released versions of sbcl, Emacs, sly on a Linux machine, if that matters. I'm only intermediate experienced with Common Lisp and its development environment, but advanced at elisp)
I suspect but don't know that the problem is that the sdl2 library is doing fanciness with threads, with the result that the working directory isn't what you think.
The solution to this in my experience is never to let the implementation second-guess you like that. In any case where there's some interface which says "do something to a file" give it an absolute pathname so it has no chance to do any thinking of its own. Do something like.
(defparameter *where-my-bitmaps-live* (merge-pathnames
(pathname "lib/bitmaps/")
(user-homedir-pathname)))
...
(defun load-a-bitmap (name)
(load-bmp (merge-pathnames (pathname name) *where-my-bitmaps-live*)))
And now it really has no excuse.
(You want to check that the above pathname-mergery is actually right: it seems to be for me but I forget the details of pathname rules every time I look away for more than a few minutes).
Thanks to Common Lisp's powerful macro system, I can write lots of code template to generate functions avoid writing redundant code manually. What's more, it can generate different code based on configure file, so I can implement many kinds of feature just apply different configure file.
However, I have no idea how to deliver the project (It's a library):
In my opinion, maybe every config file corresponds to a package?
For example, there is a common lisp file common.lisp, it generate different functions based different configure file in compile-time.
It reads a.conf in compile-time and generate functions for PackageA and reads config b.conf in compile-time for PackageB. But in-place statement must specify only one package, the common.lisp can't both in Package A and B.
By the way, I still can't find out a proper method to get the configure path of project (So I can read and use it in compile-time to generate functions) I have tried *load-truename* for it points to the cache path which contains .fasl file on SBCLv2.0.1. But it looks like the staic files are not contained in it, so it doesn't works.
For macro-expansion the thing you care about is compile time, not load time, and the variables you want are therefore *compile-file-pathname* &/or *compile-file-truename*. ASDF likes to stash compiled files (and hence the files being loaded) somewhere known to it, which you can turn off (I do) but defaultly they end up somewhere far from their sources.
Here's an example macro which should (I have not really tested it) let you enable debugging output on a per-file basis. In real life it would be better to cache the read of the config file/s but this is mildly fiddly to get right.
(declaim (inline mutter))
(defun mutter (format &rest arguments)
(declare (ignore format arguments))
(values))
(defmacro maybe-debugging (&body forms)
(let ((config-file (and *compile-file-truename*
(make-pathname :name "debug"
:type "cf"
:defaults *compile-file-truename*))))
(multiple-value-bind (debugging cond)
(if (and config-file (probe-file config-file))
(ignore-errors
(with-standard-io-syntax
(let ((*read-eval* nil))
(with-open-file (in config-file)
(values (assoc (pathname-name *compile-file-truename*)
(read in)
:test #'string-equal)
nil)))))
(values nil nil))
(when cond
(warn "bogons reading ~A for ~A: ~A"
config-file *compile-file-truename* cond))
(if debugging
`(flet ((mutter (format &rest arguments)
(apply #'format *debug-io* format arguments)))
,#forms)
`(progn
,#forms)))))
For the single-source-file-resulting-in-multiple-object-files you could do something like this (note this repeats a variant of the above code):
(eval-when (:load-toplevel :compile-toplevel :execute)
(defvar *package-compilation-configuration*
nil
"Compile-time configuration for a package")
(defun package-config-value (key &optional (default nil))
(getf *package-compilation-configuration* key default)))
(declaim (inline mutter))
(defun mutter (format &rest args)
(declare (ignore format args))
(values))
(defmacro with-muttering (&body forms)
(if (package-config-value ':mutter)
`(flet ((mutter (fmt &rest args)
(apply #'format *debug-io* fmt args)))
,#forms)
`(progn
,#forms)))
(defun compile-file-for-package (file package &rest kws
&key (output-file nil output-file-p)
&allow-other-keys)
(with-muttering
(let* ((sf-pathname (pathname file))
(package-file (make-pathname :name (string package)
:type "cf"
:defaults sf-pathname))
(the-output-file
(if output-file-p
output-file
(compile-file-pathname
(make-pathname :name (format nil "~A-~A"
(pathname-name sf-pathname)
package)
:defaults sf-pathname))))
(*package-compilation-configuration*
(if (probe-file package-file)
(with-standard-io-syntax
(mutter "~&Compile ~A -> ~A using ~A~%"
sf-pathname the-output-file package-file)
(let ((*read-eval* nil))
(with-open-file (in package-file)
(read in))))
(progn
(mutter "~&Compile ~A -> ~A (no package)~%"
sf-pathname the-output-file)
nil))))
(apply #'compile-file file
:output-file the-output-file
kws))))
Then (compile-file-for-package "x.lisp" "y") will compile x.lisp having read configuration for package "y".
To use something like this in anger you would need to integrate it with ASDF and I don't know how to do that.
An alternative idea is just to use symlinks for the source files, and have the filename-dependent configuration depend on the symlink name, not the target name.
For my case:
project-a.asd:
(asdf:defsystem #:project-a
:components ((:static-file "my-config-file.conf")
(:static-file "common.lisp") ; shared common lisp file
(:file "project-a-package")
(:file "project-a-setup")
;; other components
)
)
project-a-setup.lisp:
(in-package #:project-a)
(eval-when (:compile-toplevel)
(defvar *mypackage* (find-package 'project-a))
(defvar *source-home* (path:dirname *compile-file-truename*))
;; read configure file
(defparameter *myconf*
(with-open-file (stream (merge-pathnames *source-home* #P"my-config-file.conf"))
(read stream)))
)
(load (merge-pathnames *source-home* #P"common.lisp"))
common.lisp:
(let ((*package* *mypackage*))
;; intern symbol
)
I want to provide additional information to the user during a restart-case for missing input files.
Is there a portable way to print out the files in the current directory?
Please take a look at File System Concepts, specifically at directory.
E.g.,
use (directory "~/*") to get the list of files in your home,
use (directory "~/*/") to get the list of directories in your home,
and (directory "~/**/*") to get the list of all files in all subdirectoris in your home
Note that directory may take implementation-specific arguments like :full.
If you are using asdf (you should!), you might want to take a look at uiop which comes with it.
PS. If your implementation does not support ~ in pathname to refer to the home directory, you can use user-homedir-pathname instead.
The reference to uiop was the hint I needed. This is what I use now in my test case and which works perfectly for me:
(defun file-is-missing ()
(let* ((cwd (uiop:getcwd))
(files (uiop:directory-files cwd)))
(format t "~&Current directory: ~a~%" cwd)
(dolist (file files)
(format t "~&~3T~a" file))
(format t "~&Provide new file: ")
(list (read-line))))
(defun read-file (file)
(restart-case
(with-open-file (s file :direction :input :if-does-not-exist :error)
(do ((l (read-line s) (read-line s nil 'eof)))
((eq l 'eof) "Reached end of file.")
(format t "~&*** ~A~%" l)))
(New-File (filename)
:interactive file-is-missing
(read-file filename))))