I just started learning Common Lisp a few weeks ago, so sorry if this is an obvious question. How can I load modules programmatically? I have a directory of "tasks", each a Lisp program, and would like to import each and run a specific function that they all contain.
I found out a way to iterate over a directory (via UIOP:DIRECTORY-FILES). But I'm stuck trying to figure out a way to load a module "as" a specific name, as in Python. That would allow me to load "module-1.lisp" as mod and then load "module-2.lisp" as mod in a loop.
Pseudocode:
for path in directory
(load path as mod)
(mod:function)
If there is a better way to achieve what I am trying to do, feel free to say so! Thanks in advance for any help!
Quick summary of packages and systems
"Loading as" is not something meaningful in Common Lisp, because loading is not the same as defining a new system or package. In Python, a file is a module etc. but not in Lisp.
Lisp has a concept of packages, which are namespaces. They are used to organize symbols (symbols are first-class values). In other words, they use, export or maybe shadow symbols from other packages and that's it. When you evaluate (in-package pack), you can write all accessible symbols from pack directly, like my-fun, otherwise you need to fully qualify symbols, as some-package:their-func.
There are extensions that allows you to import a package with a different name, but this is not standard (see https://github.com/phoe/trivial-package-local-nicknames)
Loading a Lisp script is not necessarily the same as defining a package, it depends on what belongs in the file (if it as defpackage forms).
When you want to organize your source code in Lisp, you define a system, using ASDF. A system list all its dependencies, its components (files) and the dependencies between their components. That's how you describe in which order files should be loaded, compiled, tested, etc.
Packages and systems are independant but in small systems you often have the the same name for a system and the unique package it defines. For larger systems there might be multiple packages.
See 21. Programming in the Large: Packages and Symbols.
Your question
Each of your file needs to define a package, to avoid polluting a single namespace, for example:
(defpackage :common-name.01
(:use :cl :utils)
(:export #:run-me))
(in-package :common-name.01)
;; some code
But once they are all loaded, either using your approach or by defining a proper ASDF system, you want to be able to access all the RUN-ME functions, in all the packages.
You can write some introspective code that lists packages, etc. but I think a better approach would be to have a way for each of your file to declare that their RUN-ME functions should be registered in your framework, like test frameworks do.
For example, at the end of your files, you could write:
(provide-function 'run-me)
This assumes that e.g. your utils package defines and exports a function named provide-function that stores values in a central registry.
For example, in utils.lisp:
(defpackage :utils
(:use :cl)
(:export #:provide-function
#:run-all))
(in-package :utils)
(defvar *all-interesting-functions* nil)
(defun provide-function (f)
(pushnew f *all-interesting-functions* :test #'eql))
And when you want to run all the functions, you can iterate over this list:
(defun run-all ()
(mapcar #'funcall *all-interesting-functions*))
Related
I need to know how to create a library in Julia and where I must keep it in order to call it later. I come from C and matlab, it seems there is no documentation about pratical programming in Julia.
Thanks
If you are new to Julia, you will find it helpful to realize that Julia has two mechanisms for loading code. Stating you "need to know how to create a library in Julia" would imply you most likely will want to create a Julia module docs and possibly a packagedocs. But the first method listed below may also be useful to you.
The two methods to load code in Julia are:
1. Code inclusion via the include("file_path_relative_to_call_or_pwd.jl")docs
The expression include("source.jl") causes the contents of the file source.jl to be evaluated in the global scope of the module where the include call occurs.
Regarding where the "source.jl" file is searched for:
The included path, source.jl, is interpreted relative to the file where the include call occurs. This makes it simple to relocate a subtree of source files. In the REPL, included paths are interpreted relative to the current working directory, pwd().
Including a file is an easy way to pull code from one file into another one. However, the variables, functions, etc. defined in the included file become part of the current namespace. On the other hand, a module provides its own distinct namespace.
2. Package loading via import X or using Xdocs
The import mechanism allows you to load a package—i.e. an independent, reusable collection of Julia code, wrapped in a module—and makes the resulting module available by the name X inside of the importing module.
Regarding the difference between these two methods of code loading:
Code inclusion is quite straightforward: it simply parses and evaluates a source file in the context of the caller. Package loading is built on top of code inclusion and is quite a bit more complex.
Regarding where Julia searches for module files, see docs summary:
The global variable LOAD_PATH contains the directories Julia searches for modules when calling require. It can be extended using push!:
push!(LOAD_PATH, "/Path/To/My/Module/")
Putting this statement in the file ~/.julia/config/startup.jl will extend LOAD_PATH on every Julia startup. Alternatively, the module load path can be extended by defining the environment variable JULIA_LOAD_PATH.
For one of the simplest examples of a Julia module, see Example.jl
module Example
export hello, domath
hello(who::String) = "Hello, $who"
domath(x::Number) = x + 5
end
and for the Example package, see here.
Side Note There is also a planned (future) library capability similar to what you may have used with other languages. See docs:
Library (future work): a compiled binary dependency (not written in Julia) packaged to be used by a Julia project. These are currently typically built in- place by a deps/build.jl script in a project’s source tree, but in the future we plan to make libraries first-class entities directly installed and upgraded by the package manager.
Upon using (make-package 'test) (in-package test) in SBCL and CCL implementations, I've noticed that SBCL requires (cl:defun foo () (...)) or (cl:describe <symbol name here>) while CCL does not require any colons or double colons to use built-in symbols. It is my understanding that external symbols must be accessed with one colon, even if they are built-in. However CCL seems to work differently in this respect.
This confuses me somewhat regarding the use of external symbols. Are external symbols supposed to be available without any colons or is CCL just using/importing/inheriting automatically for convenience-sake?
Also, are there any more of these small but significant differences between the implementations regarding symbols and packages?
The ANSI CL standard does not define which packages are used when you create a package
At some point in time, SBCL deviated from common practice, but still follows ANSI CL standards.
Using packages
Using packages in some other package means making their symbols available in that package.
You can get a package's use list by calling the function package-use-list.
It is undefined in the ANSI Common Lisp standard which packages a new package uses by default and also if it uses any at all.
Different common practice in implementations
There are now two common practices in implementations:
use COMMON-LISP and some implementation specific packages. CCL does that.
Example in CCL:
? (package-use-list (make-package "FOOBAR"))
(#<Package "CCL"> #<Package "COMMON-LISP">)
LispWorks:
CL-USER 17 > (package-use-list (make-package "FOOBAR"))
(#<The COMMON-LISP package, 0/4 internal, 978/1024 external>
#<The HARLEQUIN-COMMON-LISP package, 0/4 internal, 365/512 external>
#<The LISPWORKS package, 0/4 internal, 226/256 external>)
use no package. SBCL does that. If you want a new package to use the package COMMON-LISP, then you have to explicitly request that.
Example in SBCL:
* (package-use-list (make-package "FOOBAR"))
NIL
ABCL:
CL-USER(1): (package-use-list (make-package "FOOBAR"))
NIL
Writing portable code
Thus in SBCL and thus in portable Common Lisp you need to tell Lisp which packages should be used. To get the COMMON-LISP package used and only that package, you need to write:
(make-package "FOO" :use '("COMMON-LISP"))
Background
The original idea in the first Common Lisp was that one could write (in-package "FOO") at the REPL and the package was created with sensible defaults and one was directly in that package. The defaults were usually the package for the language (at that time called "LISP") and packages for common extensions (for example with CLOS+MOP, threads, ...).
Later Common Lisp was changed so that IN-PACKAGE did not create a package and it is defined that upon package creation it is undefined which packages are used and it is not required to use any package upon package creation. The SBCL maintainers then thought: instead of supporting common practice (which is not mentioned in the standard), provide a more neutral and predictable behaviour of using no package upon package creation.
Other differences
Most other differences in package systems in Common Lisp are around extensions to the standard. Examples:
hierarchical/nested packages
package prefixes to whole forms (not just symbols)
A bigger and incompatible change provided by some implementations as an option:
lowercase of all existing symbols and lowercasing reader. The standard defines symbols to be internally uppercase by default.
Undefined:
garbage collection of otherwise unreferenced, but interned, symbols
Trying to finally start using asdf for my lisp doodles, I tried setting up a simple example. The files are
; contents of example.asd
(asdf:defsystem "example"
:name "example"
:depends-on ("foo")
:components ((:file "example")))
and
; contents of example.lisp
(defpackage :example
(:use :cl :asdf :foo))
(in-package :example)
(where "foo" is actually "cl-wav", but the problem persists with any of the packages I have installed locally).
Now, even though running
(asdf:load-system :foo)
works, when I try to evaluate
(asdf:make :example)
I get this error message:
The name "FOO" does not designate any package.
What am I doing wrong?
For context, my asdf-version is "3.1.5", the package "foo" is installed with (the latest version of) quicklisp, and all of this takes place in SBCL 1.3.20.
ASDF systems are a different thing from packages. A system is simply a way to group a bunch of files together in a single application or a library, which can be easily compiled, loaded, tested or installed with Quicklisp. There may be multiple packages in a single system (or even none, although that would be strange). Usually libraries have a "main" package with the same name as the system, but that isn't mandatory.
In this case, the system cl-wav defines a package named WAV, so you need to use that in your package definition. It might have been better for the library author to name the package CL-WAV with WAV as a nickname, but they didn't do so.
(defpackage :example
(:use :cl :asdf :wav))
I've set up Quicklisp to run whenever SBCL runs, and added the following line to the top of my file that I'm trying to use the priority-queue library in (as suggested in the answer to my earlier question, Priority queue for Common Lisp?). However, when I try to use it, I get errors from SBCL, saying that the functions from priority-queue are not defined! What am I missing?
For reference, I tried to write something like this:
(ql:quickload "priority-queue")
(defparameter *heap* (make-pqueue #'<))
And I get an error saying that make-pqueue is not defined.
In common lisp, anything that's named (a variable, a function, a macro) is attached to a symbol. In this case, you have a function which is attached to the symbol make-pqueue. Symbols are separated from each other using packages. This keeps collisions to a minimum and also allows for things like internal variables/functions that aren't exported by the package.
Sounds like you need to do one of three things:
Use the package name before the function: (priority-queue:make-pqueue #'<). This method is good if you want people reading your source to know exactly what code is being run. however, it can get cumbersome if you call the package many times.
Use the priority-queue package in the current package you're in:
(use-package :priority-queue)
(make-pqueue #'<)
What this does is import every exported symbol from the priority-queue package into the current package you're in (most likely cl-user). While this is good for testing, you generally want to create your own package. See next item.
Define your own package that uses priority-queue:
(defpackage :queue-test (:use :cl :priority-queue))
(in-package :queue-test)
(make-pqueue #'<)
Defining your own packages seems like a lot of work at first, but you'll start to like the separation you get, especially if you start integrating different pieces of your code together.
I use SLIME/SBCL/Emacs and Quicklisp on Ubuntu Raring. I have the function defined below. I would like to add to my Lisp library's top .lisp file, i.e., the one every other one depends on, so that I can use it in all the functions I write with my library by just adding (update-swank) to a function instead of having to add the entire function below to each piece of code that uses it.
(defun update-swank ()
"Grabs SWANK connections and tells it to handle requests.
Call this every loop in the main loop of your program"
(continuable
(let ((connection (or swank::*emacs-connection*
(swank::default-connection))))
(when connection
(swank::handle-requests connection t)))))
When I do this and restart emacs, loading my library in the process because I have the asdf:load-op in my .sbclrc file, I get a
READ error during COMPILE-FILE:
;
; Package SWANK does not exist.
in inferior lisp, and SLIME is stuck polling because the library doesn't load because in my current setup SLIME/SBCL doesn't know what swank is at the time the .lisp file that update-swank is in is loaded. I tried adding (in-package :swank) to the file that update-swank is in, but got
The name "SWANK" does not designate any package.
in inferior lisp when my library is loaded at emacs startup.
I searched through CEPL (where i got update-swank from https://github.com/cbaggers/cepl/blob/master/cepl-utils.lisp) and then copied what the creator of CEPL did and exported the function in my packages.lisp. I made sure the function was added like he did on line 20 of cepl-utils here https://github.com/cbaggers/cepl/blob/master/cepl-utils.lisp......I load my library btw with
(asdf:operate 'asdf:load-op :cl-test)
(in-package #:cl-test)
in my .sbclrc file which i assume is loaded before my .emacs file loads slime at emacs start up (i have (slime) in my .emacs file) ...I just tested removing the adsf:load-op and in-package and from my .sbclrc and running the asdf:load-op after slime/swank has been loaded and what i've been trying to do here worked with no error....but i would like to be able load my library automatically at emacs startup and the way i usually do that is by adding the asdf:load-op to my .sbclrc....If someone could tell me another way to load my library automatically at emacs startup after swank has been loaded that would answer this ?
If there is not a package defined in a running Lisp, then this package can't be used. First you need to define the package and then you can read symbols from that package. Packages are not created when CL tries to read a symbol from an unknown package.
If (find-package "FOO") returns NIL, then you can not read a symbol like FOO::BAR.
Two solutions to that are:
execute the package definition before a symbol from that package is read
remove the symbols from the source code:
Example:
foo::*bar*
replace with
(symbol-value (find-symbol "*BAR*" "FOO"))
Above finds the symbol at runtime and retrieves the symbol value.
Also:
(foo::bar :baz t)
replace with
(funcall (symbol-function (find-symbol "BAR" "FOO")) :baz t)
Above finds the symbol at runtime, retrieves the function and calls it.
This was made to sit inside the main loop of the game/realtime-demo as that was what was blocking the repl from updating. Luckily we are in control of the loop so that is easy. Now you are wanting to use this in your library (probably with opencv) so you need to identify what is blocking the update, this is generally some kind of 'main loop' if you have access to the main loop then call update-swank from in there.
If the main loop is being controlled by a foreign library then perhaps you can put it in a function that gets called every 'loop'. All you really need is it to be called often enough that the repl feels responsive.
Other than that you can try changing the settings of swank to run on a separate thread though I have no experience doing that so I cant tell you how well that would work.
You can see it inside the main loop in the (run-demo) function of this example
Also I talk about it in this video. Though albeit perhaps not in enough detail :)
The fact you are getting "swank does not exist" is very odd. It suggest that swank is not loaded, but if you are using slime then swank must be there!
p.s. Remember that this is for use with Slime or Slim, which means you are using them with emacs or vim. If you are not using Slime+emacs or Slim+vim this function will not work!
[EDIT]
Ok so I duplicated your issue by putting (ql:quickload :cepl) at the end of quicklisp's setup.lisp file. This gave me the 'Package SWANK does not exist'. One quick way to address this is to specify swank as a dependency in your project's asd file. So for example:
(asdf:defsystem #:cepl
:serial t
:depends-on (#:cl-opengl
#:swank ;;<---HERE
#:lbm-sdl
#:varjo
#:cl-utilities
#:cl-ppcre
#:symbol-munger
#:classimp
#:temporary-file
#:md5)
:components ((:file "package")
(:file "maths/base-maths")
(:file "base-macros")
;; (:file "base-lispbuilder")
(:file "cepl-utils")
Hope it helps :)