I am having trouble using the asdf build tools with common lisp. Here is my fractals.asd file:
(defpackage :fractals
(:use :cl :asdf :cl-opengl :cl-glu :cl-glut)
(:export :frac-tree :draw-tree))
(in-package :fractals)
;----------------------------------------------------------
(defsystem fractals
:name "fractals"
:version "0.0.0"
:serial t
:components ((:file "frac-tree")
(:file "fractal-lab")))
Both the ffiles fract-tree.lisp and fractal-lab.lisp have the statement (in-package :fractals) at the very beginning of the file. However, I am automatically getting an error saying The name "CL-OPENGL" does not designate any package. I don't understand why this is wrong. Furthermore, if I don't include these libraries, then I have to manually require them myself... I created a directory called: ~/.config/common-lisp/source-registry.conf.d/ where I placed a file called fractals2.conf which contains the following:
(:directory "~/lisp_proj/fractals2/")
This is the directory of my fractals.asd file as shown above. Apparently, this is supposed to tell asdf where my user-defined systems are located. I followed this tutorial.
In summary, how can I get asdf to find my user-defined systems so that I do not have to manually load them?
Thanks for all the help!
You need to declare your dependencies in the defsystem with a :depends-on clause. The :use clause of the defpackage only makes names of already loaded packages available, but won't load them.
BTW, your .asd file should not include the defpackage of the main package. While it makes sense to declare a special package to run the defsystems in there, regular packages should be declared in .lisp files.
EDIT: This may also solve your second problem. If not, it's probably because ASDF doesn't automatically expand the ~ character in paths. In that case, replace it with the actual path of your home directory.
1- Looks like you were using the undefined fractals:defsystem instead of asdf:defsystem
2- the ASDF DSL accepts the directive (:directory (:home "lisp_proj/fractals_2"))
3- Starting with 3.1.2, you can put everything under ~/common-lisp/
4- For God's Sake, RTFM!
Related
Lets say i define a macro
(defmacro foo(x)
(print x))
Now i want to be able to always load this macro in all my lisp files in future where should i save this macro?
Every CL implementation has an init file it calls on start. For SBCL that is ~/.sbclrc. Usually also quicklisp has some setup code in there if you installed quicklisp.
So you could add a line:
(load #P"path/to/your/macro/file.lisp")
And it'll get loaded each time you start SBCL.
You want to have a library.
A library can contain any number of definitions, not only macros.
The de facto standard way to define systems (which is a more general term for libraries, frameworks, and applications — units of software) is to use ASDF.
Let's say that you put your macro into a lisp file:
;;;; my-util.lisp
(in-package cl-user)
(defpackage my-util
(:use cl))
(in-package my-util)
(defmacro foo (x)
`(whatever ,x etc))
Then you put that under a directory that is known to ASDF. One useful directory for that is ~/common-lisp/, so let's use ~/common-lisp/my-util/. In the same directory, put the system definition file:
;;;; my-util.asd
(in-package asdf-user)
(defsystem "my-util"
:components ((:file "my-util")))
Now you can load this utility into any lisp interaction, e. g. on the repl:
CL-USER> (asdf:load-system "my-util")
Or in a different system:
;;;; my-application.asd
(in-package asdf-user)
(defsystem "my-application"
:depends-on ("my-util")
...)
In the case of utility libraries, you often want to use their package so that you don't need to package-qualify the symbols.
If you want things that only work on your computer (like shortcuts for your REPL use or one-off scripts), you can sometimes get away with adding things to your implementation's init file.
I've built a library that wraps custom C code and thinking about the best way to build the shared library as part of the ASDF load. The makefile is conditionalised for various OSs, so it could be as simple as uiop:run-program ..., but I thought I'd ask here if there were a more standard idiom for this.
Because the C code is specific to this application, it won't be available through a package manager and must be built specifically for each users machine. I'm fine with documenting a manual build, but if I can smooth things for the user I will. I notice that Python seems to have some kind of automated way of building libs for their CFFI and wonder if there's something for CL.
For an answer to my own question: there seems to be neither a de-facto way of doing this (based on a search of github asd files), nor a definitive method in the ASDF best practices, though there are some ideas to be gleaned from that document.
I'll put my implementation out as a suggested idiom for this use case, along with some possible alternatives. Hopefully some of the ASDF experts here will correct any misunderstandings.
;; Define a makefile as a type of source file for the system
(defclass makefile (source-file) ((type :initform "m")))
;; tell ASDF how to compile it
(defmethod perform ((o load-op) (c makefile)) t)
(defmethod perform ((o compile-op) (c makefile))
(let* ((lib-dir (system-relative-pathname "cephes" "scipy-cephes"))
(lib (make-pathname :directory `(:relative ,(namestring lib-dir))
:name "libmd"
:type #+unix "so" #+(or windows win32) "dll"))
(built (probe-file (namestring lib))))
(if built
(format *error-output* "Library ~S exists, skipping build" lib)
(format *error-output* "Building ~S~%" lib))
(unless built
(run-program (format nil "cd ~S && make" (namestring lib-dir)) :output t))))
(defsystem "cephes"
:description "Wrapper for the Cephes Mathematical Library"
:version (:read-file-form "version.sexp")
:license "MS-PL"
:depends-on ("cffi")
:serial t
:components ((:module "libmd"
:components ((:makefile "makefile")))
(:file "package")
(:file "init")
(:file "cephes")))
This works fine, on both MS Windows and UNIX. Adding a method to perform seems to be the most common method on github.
An alternative might be to use a build-op, as described in building a system. The description
Some systems offer operations that are neither loading in the current
image, nor testing. Whichever operation a system is meant to be used
with, you may use it with:
(asdf:make :foobar)
This will invoke build-op, which in turn will depend on the
build-operation for the system, if defined, or load-op if not.
Therefore, for usual Lisp systems that want you to load them, the
above will be equivalent to (asdf:load-system :foobar), but for other
Lisp systems, e.g. one that creates a shell command-line executable,
(asdf:make ...) will do the Right Thing™, whatever that Right Thing™
is.
suggest to me that this is rather close to the idea of building a C library, and it would map nicely to the mental model of using a makefile and the asdf:make command. I didn't find too many examples in the wild of this being used though and technically we are loading the C lib into the existing image.
Another point that could be reconsidered is the detection of an existing shared library to avoid the rebuild. make will avoid recompiling if the shared library exists, but will still call the linker again. This causes errors because it can't write to the shared library when it's in use, at least on MS Windows. The ASDF example used Lisp code to detect the existence of the library and avoiding recompilation, but an alternative might be to use output-files.
The ASDF docs are a bit muddled on the purpose of output-files and there are no examples that make their intentions clear, but in the manual section on creating new operations we have:
output-files If your perform method has any output, you must define a
method for this function. for ASDF to determine where the outputs of
performing operation lie.
which suggests that defining the shared library (libmd.so or libmd.dll) is the recommended way to avoid a recompilation if the output-files already exists.
Finally, the C library could be considered a secondary system, cephes/libmd in this case, and added to the :depends-on clause in the main system. The section on other secondary systems demonstrates building an executable this way, with build-op. Except for the fact that this is building an executable and hard-codes ".exe" it seems to map well onto the use case:
To build an executable, define a system as follows (in this case, it's
a secondary system, but it could also be a primary system). You will
be able to create an executable file foobar-command by evaluating
(asdf:make :foobar/executable):
(defsystem "foobar/executable"
:build-operation program-op
:build-pathname "foobar-command" ;; shell name
:entry-point "foobar::start-foobar" ;; thunk
:depends-on ("foobar")
:components ((:file "main")))
The build-pathname gives the name of the executable; a .exe type will
be automatically added on Windows.
I didn't use this method because the secondary system would look almost exactly like the primary one does now, but would be slightly less understandable.
I'm reading the book Practical Common Lisp and I'm at chapter 25 now. For each chapter I create a folder in my book's folder, containing the chapter's system definition file, package definition file and the code. In chapter 25, I'll need the package I built in chapter 24, so the package definition for chapter 25 will look like this:
(defpackage :chapter-25-package (:use :common-lisp :chapter-24-package))
But chapter-24-package isn't in the Chapter 25 system. Can I use it in this package without having to include it in the system? Note: They are in separate folders within a same folder.
This is my system definition. There are links to all the necessary files in chapter 24 system inside chapter 25 folder. If any of them aren't necessary, please let me know.
(defpackage #:chapter-25-asd (:use :cl :asdf) (:export :chapter-25-system))
(in-package :chapter-25-asd)
(defsystem chapter-25-system
:name 'chapter-25-system
:components
((:file "chapter-25-package" :depends-on ("chapter-24-package"))
(:file "chapter-25" :depends-on ("chapter-25-package")))
:depends-on ("chapter-24-system"))
This is the error I get:
Component "chapter-24-package" not found, required by
#<CL-SOURCE-FILE "chapter-24-system" "chapter-24">
[Condition of type ASDF/FIND-COMPONENT:MISSING-DEPENDENCY]
Edit: This is the second question I make so I'm sorry if there is any bad practice. I appreciate if you let me know.
In short, no, but you can depend on the other system.
In order to depend on a different system, use the :depends-on key in the system definition.
chapter-25.asd:
(defsystem "chapter-25"
:depends-on ("chapter-24")
:components ((:file "package")
(:file "chapter-25" :depends-on ("package"))))
Assuming that a system "chapter-24" is defined in a file "chapter-24.asd" where ASDF can find it, this makes ASDF ensure that the system "chapter-24" is loaded before the system "chapter-25" is loaded.
In the code of the system "chapter-25", you can then assume that packages defined in the system "chapter-24" are loaded and can be referenced, e. g. through the :use option of defpackage:
(defpackage #:chapter-25
(:use (#:cl #:chapter-24)))
In short, yes, you can depend on the other system. Just use "package inferred" ASDF system.
In this case, each lisp file will be considered a separate subsystem and can depend on a file from the same or another system. And dependencies will be inferred from defpackage forms.
Here is an example of such a system.
These are imports from different files of another system.
Let's say we have the following prerequisites:
(quickproject:make-project "lib1")
(quickproject:make-project "lib2")
;;;; Modify lib1/package.lisp like that:
(defpackage #:lib1
(:use #:cl)
(:export #:symbol1))
;;;; Modify lib2/package.lisp like that:
(defpackage #:lib1
(:use #:cl)
(:export #:symbol2))
(quickproject:make-project "project1")
;;;; Edit the project1's asdf:defsystem so that it has the following dependencies:
:depends-on (:lib1 :lib2)
Load project1 and inspect the lib1 package and you will note that you have both symbol1 and symbol2 there.
Implications:
Different dependencies can pollute the packages of other dependencies
Worse - one dependency can redefine symbol definitions of another dependency and break the system.
I can probably solve my current problem by adopting a package naming convention like the one used in Java (with long delimited names). However, this appears to be a problem that is supposed to be solved as a community. We need a general purpose solution that will apply to all CL libraries. What would that be?
I am 90% sure the answer is in this paragraph of the asdf documentation , but I seem unable to grok it.
I am wondering if I am able to have source files that do not end in ".lisp" as file components. For example, usually I have something like
(asdf:defsystem #:hash-bang-lang
:serial t
:depends-on (#:named-readtables)
:components ((:file "package")
(:file "hash-bang-lang")))
As you can see the :file components do not specify the extension. I am wondering if I can load lisp files that do not end in ".lisp".
The reason I want this is a little odd but has no impact on the question so feel free to skip the next bit.
I have some file "test.mylang" and it starts with the following
(in-package :hash-bang-lang)
(in-readtable hash-bang)
#!mylang
..rest of file..
The #!mylang is a reader macro that gives my #'mylang function control of parsing the rest of the file. So if I had a parser for python I would use #!python and my #'python function will take up the parsing.
In those cases I would like to have the .mylang or .py extension so editors know instantly how to highlight the code, even though it's going to be loaded as lisp anyway.
Thanks folks
The :file component type forces the file extension to .lisp. If you need some other extension, you need a new component type.
(defpackage #:mylang-system
(:use #:cl #:asdf))
(in-package #:mylang-system)
(defclass mylang-file (cl-source-file)
((type :initform "mylang")))
(defsystem test-ext
:encoding :utf-8
:components ((:file "package")
(:mylang-file "hash-bang-lang")))