How do I change the path that .fas files are placed in? By lisp code and not configuration file.
I am doing like this:
(asdf:disable-output-translations)
(push *default-pathname-defaults* asdf:*central-registry*)
(setq asdf:*central-registry*
(append asdf:*central-registry*
(directory (merge-pathnames
(make-pathname :directory '(:relative :wild))
*default-pathname-defaults*))))
With ECL, it generates the .fas and .o files in the source code directory being compiled, but I don't want it to be in the project directory, which is on a path that I go through, to better organize myself here.
Something like:
(asdf:set-path-compiled-files "/home/noloop/lisp-armv7a-libs/builds")
asdf:*central-registry* is for the location of SOURCE, there isn't a single location for compiled results because asdf2 explicitly handles you using multiple implementations, which have incompatible fasl formats (this is a point of improvement in asdf2 vs the previous state of affairs).
According to [1] (asdf:initialize-output-translations list) can be used to set the translation programmatically, and (asdf:ensure-output-translations) can be used to see the current value
As mentioned in the DSL, you must include one and only one of :inherit-configuration or :ignore-inherited-configuration.
It's not clear if you also need to set one of :enable-user-cache or :disable-cache but choosing one seems sensible.
As stated in [2] the order matters.
On my machine, with sbcl:
cl-user> (asdf::initialize-output-translations '(:output-translations (t #p"/tmp/asdf/") :disable-cache :ignore-inherited-configuration))
Gives:
((#P"/home/peter/Programming/unix-built/sbcl/lib/sbcl/**/*.*" T)
(#P"/tmp/asdf/" T) (T #P"/tmp/asdf/") (T T)
(#P"/home/peter/.cache/common-lisp/sbcl-2.0.0.104-b5a4454ca-linux-x64/**/*.*"
T)
(T
#P"/home/peter/.cache/common-lisp/sbcl-2.0.0.104-b5a4454ca-linux-x64/**/*.*"))
And with that I see FASLs in /tmp/asdf when I try and load a system.
1: http://soc.if.usp.br/manual/cl-asdf/asdf/Controlling-where-ASDF-saves-compiled-files.html#Controlling-where-ASDF-saves-compiled-files
2: https://gitlab.common-lisp.net/asdf/asdf/issues/22
Related
I use a macro called use-db and change-to-path in a lot of my projects.
(defmacro use-db (db project-name &body query)
`(progn
(change-to-path ,project-name)
(clsql:connect ,db :database-type :sqlite3)
(unwind-protect (progn ,#query)
(clsql:disconnect :database ,db))))
(defmacro change-to-path (project-name)
`(uiop:chdir (merge-pathnames (write-to-string ,project-name) "/Users/vince/quicklisp/local-projects/")))
As you can see, change to path changes the repl dir. Then use-db reads the sqlite db from the root of the project folder. This works fine.
However, when I try to read a db as part of a route (easy-routes + hunchentoot):
(defroute test-file ("/results" :method :post)
()
(get-one-col-as-list-db #'first "firstname"))
(defun get-one-col-as-list-db (fn tablename)
(ki:use-db "new.db" 'custom-sender
(mapcar fn
(clsql:query
(concatenate 'string "Select * from " tablename)))))
I get the following error:
Error in SB-POSIX:CHDIR: No such file or directory (2)
without using the change-to-path macro, clsql function would note find the database.
Is there a better way to use sqlite (from the file system) and manage the pathing for specific db files?
To change the current working directory, temporarily, use uiop:with-current-directory:
(uiop:with-current-directory ("/tmp/")
(uiop:getcwd))
(edit) chdir has a side effect: it effectively changes the current working directory (uiop:getcwd), while with-current-directory doesn't, or it does but only temporarily for the code under it. For your sanity you want to avoid chdir's side effets (or I do, I want to work at a project root). It could have an effect on bare-bones Hunchentoot, if it looks at a www/ directory under the project root, but in your case I'm not sure.
Your use-db mimics a well-known pattern: get a resource, and be sure to release it in case of errors. These macros are often named "with-something", just like "with-current-directory". Does CLSQL have such a macro? It does: with-database.
Why is your change-to-path a macro? It can be a function that concatenates strings and calls chdir.
Inside use-db, "project-name" is an unknown variable. Look at the compiler warnings:
; caught WARNING:
; undefined variable: COMMON-LISP-USER::PROJECT-NAME
How do you use the two anyways?
Error in SB-POSIX:CHDIR: No such file or directory (2) […] without using the change-to-path macro
what is the directory it tries to go to? You'd have a tip on how things are not working.
I try to never rely on the process working directory, because from Common Lisp I am much happier using *default-pathname-defaults* (no side effects!). This is a special variable with a long name, and what I want is to use merge-pathnames in most of the cases, so I have this macro:
(defmacro with-path (pathname &body body)
`(let ((*default-pathname-defaults* (merge-pathnames ,pathname)))
,#body))
It is just a local binding of a special variable so unwinding is trivial, and it comes with all the benefits of CL pathnames (and some of its drawbacks but that's fine).
On the Posix side of things, programs want a native namestring so I also use this a lot:
(defun fullname (pathname)
(osicat:native-namestring (merge-pathnames pathname)))
I do also have a concept of projects in my environment so it looks as follows. Note that I try to carry the special variable until the last moment, until it needs to be given to other processes, like:
(within-project projects:my-project
(with-path "build/"
(run "make")))
Here run grabs the current *default-pathname-defaults* for its :directory argument (using sb-ext:run-program but this is a detail).
Since it is a pathname, some components can be wild too, if that makes sense in your use case. For example, you can avoid giving a ".db" suffix if you prefer:
(defmacro using-db ((name db) &body body)
`(with-path #P"*.db"
(with-database (,name (list (fullname ,db)) :database-type :sqlite3)
,#body)))
For example, here I enter the "/tmp/" directory and open database "test.db":
(with-path "/tmp/"
(using-db (database "test")
database)))
#<CLSQL-SQLITE3:SQLITE3-DATABASE /tmp/test.db CLOSED {100DD6CE63}>
Edit: with respect to the following code:
(uiop:chdir
(merge-pathnames (write-to-string project-name)
"/Users/vince/quicklisp/local-projects/")))
Note that you can use ASDF to access files relative to systems:
(asdf:system-relative-pathname :cl-ppcre "ppcre.lisp")
#P"..../quicklisp/software/cl-ppcre-20220220-git/ppcre.lisp"
This is example.lisp:
(defun f (x)
(* x 2))
(defconstant +n+ (f 1))
When I compile try to compile the file using (compile-file "example.lisp"), I get an error when compiling the constant: The function COMMON-LISP-USER::F is undefined. How can this be? I have already defined the function f before using it to define the constant. How do I define a constant whose value is the return value of a function?
I am using SBCL 2.0.1.
From DEFCONSTANT:
[...] users must ensure that the initial-value can be evaluated at compile time (regardless of whether or not references to name appear in the file) and that it always evaluates to the same value.
But COMPILE-FILE and 3.2.3 File Compilation only require the DEFUN form to establish that F exists, not that is may be evaluated during compilation.
You need to LOAD the resulting file for the definition to be available in your environment (e.g. Slime compiles a file and loads the result). Likewise, when you evaluate forms one at a time in the REPL, the definition becomes available right away, as-if a small compile-load cycle was done.
You have two main options:
wrap the defun in an eval-when form
(eval-when (:compile-toplevel :load-toplevel :execute)
(defun f (x) (* x 2)))
put the two forms in distinct files, and have the second one depends in the system (in the defsystem sense) on the first one:
(defsystem "foo"
:components ((:file "define-f")
(:file "define-constant"))
:serial t)
I have a directory name (as string) with a tilde: ~/projects.
I want to get its fullpath: /home/user/projects. How do I do that ?
The goal is to pass it to uiop:run-program, that doesn't seem to do the right thing©.
With this answer: How to translate (make-pathname :directory '(:absolute :home "directoryiwant") into absolute path
(merge-pathnames
(make-pathname
:directory '(:relative "~/projects"))
(user-homedir-pathname))
#P"/home/me/~/projects/"
=> WRONG
Thank you.
edit I'll share more context.
I wanted to run a program through uiop:launch-program. I had a user-defined list of directories such as ~/projects. Using it as is created the ./~/projects directory instead of /home/user/projects.
truename doesn't work if the directory doesn't exist.
On SBCL, (namestring "~/doesntexist") returns also its tilde.
merge-pathnames didn't work, still the tilde problem.
Feeding ensure-directories-exist with this result created a directory named ~.
Given the answers, I had no choice but to adapt the logic to expand the directory name of a directory we actually want to exist.
;; Create a directory
;; Ensure its name (string) ends with a slash.
(setf mydir
(str:concat (string-right-trim (list #\/) mydir)
"/"))
(ensure-directories-exist base)
Then I could use its truename.
General remarks about ~
Your Lisp implementation may or may not support tilde syntax.
If it does (e.g. CCL, ABCL, CLISP, ECL, LispWorks), then truename would consistently expand to a filename:
(truename "~/projects")
=> /home/user/projects
If your implementation doesn't, or if you want to code portably, you have to merge relatively to (user-homedir-pathname):
(truename (merge-pathnames #p"projects" (user-homedir-pathname)))
=> /home/user/projects
Note that the tilde, if it is supported, seems to only be supported for strings used as pathnames, and not in directory components; (:relative "~") does not work as you would expect, and refers to a directory literaly named "~".
Instead, at least for SBCL, the appropriate directory is (:absolute :home), or, if you want to refer to another user, you can wrap the component in a list:
(make-pathname :directory '(:absolute (:home "root")))
=> #P"~root/"
Notice how it only works if the :home form is just after :absolute, it doesn't work otherwise (see Home Directory Specifiers).
Expanding to non-existent pathnames
truename would require that the thing exists?
Yes, if you want to build the absolute path to a file that does not exist (yet), then you need to call truename on the part that exists, and merge with that.
In your case, that would be (truename "~/"), which is the same as (user-homedir-pathname).
As pointed out by Rainer Joswig, calling namestring on implementations other than SBCL returns an expanded pathname, translating ~ as /home/user. In SBCL you have to call sb-ext:native-namestring to obtain the same effect.
In other words, in order to expand to a filename that does not necessarily exist, you could write the following portability layer:
(defun expand-file-name (pathname)
(check-type pathname pathname)
(block nil
#+(or lispworks clozure cmu clisp ccl armedbear ecl)
(return (namestring pathname))
#+sbcl
(return (native-namestring pathname))
#+(not (or sbcl lispworks clozure cmu clisp ccl armedbear ecl))
(let ((expanded (namestring pathname)))
(prog1 expanded
(assert (not find #\~ expanded) ()
"Tilde not supported")))))
See also https://github.com/xach/tilde/blob/master/tilde.lisp for inspiration if your Lisp doesn't support the syntax.
There is a native-namestring function in uiop, which should be available in all implementations:
(uiop:native-namestring "~/projects")
=> /home/user/projects
Anselm Farber's solution, involving uiop:native-namestring breaks on some pathnames that don't have native-namestrings, like the following:
(uiop:native-namestring "~/Music/[Video] performance.mp4")
==>
The pathname #P"~/Music/[Video] performance.mp4"
does not have a native namestring because
of the :NAME component #<SB-IMPL::PATTERN (:CHARACTER-SET
. "Video")
" performance">.
[Condition of type SB-KERNEL:NO-NATIVE-NAMESTRING-ERROR]
Here is a direct solution that only uses pathname- functions:
(defun expand-user-homedir (f)
(let ((d (pathname-directory f)))
(if (and (eql (car d) :absolute)
(eql (cadr d) :home))
(make-pathname :directory (append (pathname-directory (user-homedir-pathname))
(cddr d))
:name (pathname-name f)
:type (pathname-type f))
f)))
I would like to make a single CGI program in Common Lisp, that reads a cookie.
It is clear to me that I can send a Cookie by sending HTTP headers. Yet, I would like to know how to read a cookie on a server, through CGI program, from the client who is accessing the program.
Based on Kaz answer, we need to get an environment variable. But I won't refer to the cl-cookbook[1], now we have this solution that comes with asdf:
(uiop:getenv)
Also the Osicat library (os, files and directories) has (environment-variable name) for posix systems, incl. windows.
[1] the cl-cookbook on sourceforge is old and sometimes outdated. There is a copy on github, maintained (well, a bit) and editable: https://lispcookbook.github.io/cl-cookbook/os.html (https://github.com/LispCookbook/cl-cookbook/).
edit: the cookbook on github has been edited accordingly and should be deployed soon on https://lispcookbook.github.io/cl-cookbook/os.html
The CGI mechanism passes the cookie using the HTTP_COOKIE environment variable. ANSI Common Lisp doesn't have any API for accessing POSIX and Windows system environment variables. Implementation-specific functions, or else an implementation's FFI, is used to achieve the equivalent of C's getenv.
The Common Lisp Cookbook suggests this:
* (defun my-getenv (name &optional default)
#+CMU
(let ((x (assoc name ext:*environment-list*
:test #'string=)))
(if x (cdr x) default))
#-CMU
(or
#+Allegro (sys:getenv name)
#+CLISP (ext:getenv name)
#+ECL (si:getenv name)
#+SBCL (sb-unix::posix-getenv name)
#+LISPWORKS (lispworks:environment-variable name)
default))
MY-GETENV
* (my-getenv "HOME")
"/home/edi"
* (my-getenv "HOM")
NIL
* (my-getenv "HOM" "huh?")
"huh?"
Before using, I'd slightly modify this to:
(or #+CMU (let ((x ...)) (if ...))
#+Allegro ...
#+CLISP
...
default)
Following this question: Strange symbols in filespec when calling load I tried my luck with pathnames, but, as you see, failed. Below is an example of the error, which I cannot explain:
This code does not work:
(defun test-process-imgae-raw ()
(cl-gd:with-image-from-file
(test #P"digit-recognition:digit-7.png")
(process-image-raw test)))
Neither does this:
(defun test-process-imgae-raw ()
(cl-gd:with-image-from-file
(test "digit-recognition:digit-7.png")
(process-image-raw test)))
But this code does:
(defun test-process-imgae-raw ()
(cl-gd:with-image-from-file
(test (translate-logical-pathname "digit-recognition:digit-7.png"))
(process-image-raw test)))
And so does this:
(defun test-process-imgae-raw ()
(cl-gd:with-image-from-file
(test (translate-logical-pathname #P"digit-recognition:digit-7.png"))
(process-image-raw test)))
Here's the "translator":
(setf (logical-pathname-translations "DIGIT-RECOGNITION")
`(("**;*.*" "/home/wvxvw/Projects/digit-recognition/**/*.*")))
And here's the error I'm getting:
Pathname components from SOURCE and FROM args to TRANSLATE-PATHNAME
did not match:
:NEWEST NIL
[Condition of type SIMPLE-ERROR]
Restarts:
0: [RETRY] Retry SLIME REPL evaluation request.
1: [*ABORT] Return to SLIME's top level.
2: [ABORT] Abort thread (#<THREAD "repl-thread" RUNNING {1003800113}>)
Backtrace:
0: (SB-IMPL::DIDNT-MATCH-ERROR :NEWEST NIL)
1: (SB-IMPL::TRANSLATE-COMPONENT :NEWEST NIL :NEWEST T)
2: (TRANSLATE-PATHNAME #P"DIGIT-RECOGNITION:DIGIT-7.PNG.NEWEST" #P"DIGIT-RECOGNITION:**;*.*" #P"/home/wvxvw/Projects/digit-recognition/**/*.*")
3: (TRANSLATE-LOGICAL-PATHNAME #P"DIGIT-RECOGNITION:DIGIT-7.PNG.NEWEST")
4: (SB-IMPL::QUERY-FILE-SYSTEM #P"DIGIT-RECOGNITION:DIGIT-7.PNG" :TRUENAME NIL)
5: (PROBE-FILE #P"DIGIT-RECOGNITION:DIGIT-7.PNG")
6: (CREATE-IMAGE-FROM-FILE #<unavailable argument> NIL)
7: (TEST-PROCESS-IMGAE-RAW)
I'm trying to read the Hyperspec section on translate-pathname, but I can make absolutely no sense of what it says, neither from the examples it shows. Let alone it, I can't even understand how there can possibly be an error if you transform a string by whatever rules you put in place, so far it's only one way transformation...
I'm trying to read SBCL sources for this function, but they are really lengthy, and trying to figure out the problem this way is taking huge amounts of time.
tl;dr How is it even possible that translate-logical-pathname called from user's code will produce something different to what is produced from that function if called from system code? This is not only non-portable, this is just outright broken.
EDIT:
Adding one more asterisk to the pattern on the left side, but not on the right solved this. But the purpose or logic of why is this necessary is beyond me.
I.e.
(setf (logical-pathname-translations "DIGIT-RECOGNITION")
`(("**;*.*.*" "/home/wvxvw/Projects/digit-recognition/**/*.*")))
This allows pathnames like digit-recognition:foo.bar.newest to succeed, just like digit-recognition:foo.bar but why is that asterisk a requirement flies beyond me. Also, why is the system function feels entitled to change the pathname to something else of what it was given?.. But just not to get you confused, with-image-from-file will only work with the path already expanded by translate-logical-pathname, it won't work otherwise.
EDIT2:
OK, it seems like this is the problem with cl-gd, instead of trying to expand the file name, it takes it literally. This code taken from create-image-from-file probably best answers my question:
(when (pathnamep file-name)
(setq file-name
#+:cmu (ext:unix-namestring file-name)
#-:cmu (namestring file-name)))
(with-foreign-object (err :int)
(with-cstring (c-file-name file-name)
(let ((image (ecase %type
((:jpg :jpeg)
(gd-image-create-from-jpeg-file c-file-name err))
I.e. instead of doing (namestring file-name) it has to do (namestring (trnaslate-logical-pathname file-name)). Duh...
Another way is to use TRUENAME, which returns the real file name. Normally this would not make a difference.
Image a file system with file versions (like the file systems of VMS, ...). If you have a logical pathname foo:bar;baz.png.newest, then it might translate to, say, /myfiles/images/baz.png~newest (again, just assume that it has version numbers). This still is not a real physical file. If such a Lisp system tries to open the file, it has to look into the file system to actually determine the newest file. That might be /myfiles/images/baz.png~42.
So, if you want to pass real physical filenames to external tools (like a C library), it might not be sufficient to expand the logical pathname, but it might be necessary to compute the truename - the real physical file.
The ability to deal with file versions comes from a time when file versions where quite common (see Versioning file system) with operating systems like ITS, VMS or the various Lisp Machine operating systems.
The main practical problem for this is that there is no common test suite for pathname operations for the various CL implementations and thus implementations differ in a lot of subtle details (especially when you need to deal with different file systems from different operating systems). Plus real file systems have complications - for example file names in Mac OS X use a special unicode encoding when dealing with Umlauts.