Is there a way of compiling UFFI dependent lisp programs on Embeddable Common Lisp without modifying them?
For example when i try to load CLSQL via Quicklisp i get the below error:
> (ql:quickload "clsql")
Condition of type: LOAD-SYSTEM-DEFINITION-ERROR
Error while trying to load definition for system clsql from pathname /home/thorin/quicklisp/dists/quicklisp/software/clsql-20120107-git/clsql.asd: Cannot find the external symbol GETENV in #<"FFI" package>.
Available restarts:
1. (ABORT) Give up on "clsql"
2. (RESTART-TOPLEVEL) Go back to Top-Level REPL.
Broken at SI:BYTECODES. [Evaluation of: (QUICKLISP-CLIENT:QUICKLOAD "clsql")]
More interestingly when i issue the command second time without quitting i got following error:
> (ql:quickload "clsql")
To load "clsql":
Load 1 ASDF system:
clsql
; Loading "clsql"
[package cmucl-compat]............................
[package clsql-sys]...............................
[package clsql]...................................
[package clsql-user]
FINALIZE-INHERITANCE
;;; Error:
;;; in file kmr-mop.lisp, position 1682
;;; at (FINALIZE-INHERITANCE (FIND-CLASS '%SLOT-ORDER-TEST-CLASS))
;;; * The form (FINALIZE-INHERITANCE (FIND-CLASS '%SLOT-ORDER-TEST-CLASS)) was not evaluated successfully.
;;; Error detected:
;;; The function FINALIZE-INHERITANCE is undefined.
Condition of type: COMPILE-ERROR
Error while invoking #<compile-op (:VERBOSE NIL) 000000000497ae70> on #<cl-source-file "clsql" "sql" "base" "kmr-mop">
Maybe this is something to do with Quicklisp.
ECL's ffi interface is compatible with UFFI's. http://ecls.sourceforge.net/new-manual/
Related
This compiles fine on ccl, but fails with a circular ref. error on SBCL:
kp.asd:
(in-package :asdf)
(defsystem kp
:components
((:module "utils"
:components
((:file "utils")
))
))
(load-system :kp)
------------------------
utils.lisp:
(defpackage :utils)
(in-package :utils)
(defvar *kp-version-utime* (get-universal-time))
------------------------
Error reported by SBCL:
debugger invoked on a LOAD-SYSTEM-DEFINITION-ERROR in thread #<THREAD "main thread" RUNNING {10005E85B3}>: Error while trying to load definition for \
system kp from pathname /var/www/ai/insights/kp.asd: Circular dependency: ((#<DEFINE-OP > . #<SYSTEM "kp">) (#<LOAD-OP > . #<SYSTEM "kp">) (#<LOAD-OP\
> . #<MODULE "kp" "utils">) (#<LOAD-OP > . #<CL-SOURCE-FILE "kp" "utils" "utils">) (#<PREPARE-OP > . #<CL-SOURCE-FILE "kp" "utils" "utils">) (#<PREP\
ARE-OP > . #<MODULE "kp" "utils">) (#<PREPARE-OP > . #<SYSTEM "kp">))
(It looks like your post is mostly code; please add some more details.)
(load-system :kp)
You should not have this in an ASDF declaration. Better not consider a system definition as a Lisp program, only as a declarative way of stating your dependencies.
If, during the declaration of the system, you also need to load another system, then that system becomes a dependency. What happens most likely is that SBCL consider the file as a whole when evaluating it, and while loading kp, you ask it to load kp, which is indeed a circular dependency. Maybe CCL silently consider the file already loaded, I don't know.
If you remove it, entering (ql:quickload :kp) in the REPL works perfectly fine. Also, the (in-package :asdf) is useless (and if provided, it should be asdf-user)
Your .asd is invalid and deserves to lose.
Do you have an old CCL with ASDF 3.2 or earlier? Then it fails to detect the bug in your .asd. ASDF 3.3 correctly detects circular dependencies in the loading of .asd files.
I'm running LispWorks 7.1 on OSX (macOS High Sierra). I sometimes encounter a Quicklisp loading bug (I say sometimes because when I restart LispWorks I sometimes don't see this error again).
Example:
CL-USER 1 > (ql:quickload "iterate")
To load "iterate":
Load 1 ASDF system:
asdf
Install 1 Quicklisp release:
iterate
Error: Error detected during deflate decompression: Corrupted Data detected during decompression: Incorrect huffman code (1E55) in huffman decode!
1 (abort) Give up on "iterate"
2 Return to top loop level 0.
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
Any idea what's going on?
I've verified I have a recent client:
CL-USER 6 > (ql:update-client)
The most up-to-date client, version 2017-03-06, is already installed.
T
Also, the backtrace, in case it helps:
CL-USER 12 : 1 > :b
Call to ERROR
Call to QL-GUNZIPPER::DECODE-HUFFMAN-BLOCK
Call to QL-GUNZIPPER::DECODE-BLOCK
Call to QL-GUNZIPPER::INFLATE-STREAM
Call to QL-GUNZIPPER::INFLATE-GZIP-STREAM
Call to QL-GUNZIPPER:GUNZIP
Call to (METHOD QL-DIST:INSTALL (QL-DIST:RELEASE))
Call to (METHOD QL-DIST:ENSURE-INSTALLED (T))
Call to MAP
Call to QUICKLISP-CLIENT::APPLY-LOAD-STRATEGY
Call to QUICKLISP-CLIENT::AUTOLOAD-SYSTEM-AND-DEPENDENCIES
Call to (METHOD QUICKLISP-CLIENT:QUICKLOAD (T))
Call to CLOS::NEXT-METHOD-CALL-2
Call to QL-DIST::CALL-WITH-CONSISTENT-DISTS
Call to CLOS::GENERIC-FUNCTION-NON-DISCRIMINATOR
Call to LET
Call to EVAL
Call to CAPI::CAPI-TOP-LEVEL-FUNCTION
Call to CAPI::INTERACTIVE-PANE-TOP-LOOP
Call to MP::PROCESS-SG-FUNCTION
As Rainer Joswig pointed out, I just had to (duh!) apply the latest patches.
For future reference, this involves
Downloading the patches listed here
Copying them to the appropriate private-patches directory (e.g. /Applications/LispWorks\ 7.1\ \(64-bit\)/Library/lib/7-1-0-0/private-patches for me)
Modifying load.lisp in the directory accordingly (in this case, adding (load-one-private-patch "lisp-memory-copy-32-chunks" :system64) and
(load-one-private-patch "replace-i-vectors" :system))
Restarting Lispworks
I put failing.asd
(in-package :asdf-user)
(defsystem "failing"
:description "some code destined to fail"
:version "0.1"
:author "me"
:components ((:file "package")))
and package.lisp
(defpackage :failing
(:export :foo :bar))
(in-package :failing)
(defun foo () 42)
(defmacro bar ()
(let ((x (foo)))
`(print ,x)))
(bar)
into ~/quicklisp/local-projects/failing. Using Clozure CL with Quicklisp installed, I run
(ql:quickload :failing)
which gives me
To load "failing":
Load 1 ASDF system:
failing
; Loading "failing"
[package failing]
> Error: Undefined function FOO called with arguments () .
> While executing: BAR, in process listener(1).
> Type :GO to continue, :POP to abort, :R for a list of available restarts.
> If continued: Retry applying FOO to NIL.
> Type :? for other options.
It seems I can't call a function from a macro inside the package. Why not?
That can only happen when the file gets compiled before loaded. Generally it has nothing to do with ASDF (which is a tool to manage file dependencies and to compile/load code) or packages (which are namespaces and are not in any way related to ASDF).
It has to do with how file compilation works in Common Lisp:
The file compiler sees the function foo and compiles it -> the code for it gets written to a file. It does not (!) load the code into the compile-time environment.
The file compiler then sees the macro bar and compiles it -> the code gets written to a file. It does (!) load the code into the compile-time environment.
The file compiler then sees the macro form (bar) and wants to expand it. It calls the macro function bar. Which calls foo, which is undefined, since it is not in the compile-time environment.
Solutions:
put the function definition in a separate file in the ASDF system and compile/load it earlier.
put the function inside the macro as a local function
put (EVAL-WHEN (:COMPILE-TOPLEVEL :LOAD-TOPLEVEL :EXECUTE) ...) around the function definition. It causes the definition to be executed at compile-time.
Remember: the file compiler needs to know macro functions -> otherwise it would not be able to macroexpand code. Ordinary functions just get compiled, but not loaded at compile time, during compilation of a file.
Here's a strange situation. I have this code:
(eval-when (:compile-toplevel :load-toplevel :execute)
(ql:quickload "cffi-grovel")
(setf cffi-grovel::*cc* "mpicc")) ; <--- this is the line it complains about.
Which I belive has to load cffi-grovel package before setting cffi-grovel::*cc* variable. When this form is executed from SLIME it works, but it doesn't work when loaded directly by SBCL, here's the output:
$ sbcl --noinfo
* (ql:quickload "cl-mpi")
debugger invoked on a LOAD-SYSTEM-DEFINITION-ERROR in thread
#<THREAD "main thread" RUNNING {10029C0E43}>:
Error while trying to load definition for system cl-mpi from pathname
/home/wvxvw/quicklisp/local-projects/cl-mpi/cl-mpi.asd:
READ error during COMPILE-FILE:
Package CFFI-GROVEL does not exist.
Line: 6, Column: 25, File-Position: 264
< restarts ... >
* (ql:quickload "cffi-grovel")
To load "cffi-grovel":
Load 1 ASDF system:
cffi-grovel
; Loading "cffi-grovel"
..
("cffi-grovel")
* (ql:quickload "cl-mpi")
To load "cffi-grovel":
Load 1 ASDF system:
cffi-grovel
; Loading "cffi-grovel"
To load "cl-mpi":
Load 1 ASDF system:
cl-mpi
; Loading "cl-mpi"
; mpicc -m64 ...
; ...
.
("cl-mpi")
Why does it fail the first time?
PS. I also tried #.cffi-grovel::*cc* instead - same result.
It fails because Lisp reads every form before executing it. And when it reads it, package cffi-grovel indeed does not exist yet, because cffi-grovel is loaded at execution time (whatever it means for form wrapped with eval-when).
Try spliting the eval-when form into two eval-whens: ql:quickload and setf. Or write something like this:
(setf (symbol-value (find-symbol "*CC*" "CFFI-GROVEL"))
"mpicc")
monoid gives a good answer. Here is a slightly shorter form.
Find the symbol at runtime, when the package actually exists. Note the use of SET, not SETF here:
(set (find-symbol "*CC*" "CFFI-GROVEL") "mpicc")
To make it more robust, one would need to check that FIND-SYMBOL actually finds the symbol.
The following function also gives you meaningful error messages and restarts:
(defun set-runtime-symbol (name package value)
(assert (find-package package) (package))
(assert (find-symbol name package) (name))
(set (find-symbol name package) value))
Alternatively use two EVAL-WHEN statements, instead of one.
OK, finally figured it I can do it like so:
(setf #.(intern "cffi-grovel::*cc*") "mpicc")
But I'm not sure how much this is fail-safe.
Doing a SWIG tutorial, and using the example.c, example.i as they provided there. I generated lisp file with swig -cffi example.i.
But when I run test.lisp with SBCL, i get a complaint about undefined alien function, as well as complaints when compiling the example.lisp itself. I'm pretty sure I still have to compile my example.c into a library and then somehow tell SBCL to load it! But the docs are very scant on this, except for this.
Can somebody tell me how to do this or is there a better way than SWIG to automatically generate CFFI bindings from C/C++ sources??
sbcl output:
...
;
; caught STYLE-WARNING:
; Undefined alien: "fact"
;
; compilation unit finished
; caught 1 STYLE-WARNING condition
;
; caught STYLE-WARNING:
; Undefined alien: "my_mod"
...
test.lisp
;call C functions defined in example.c
(require :cffi)
;;;(require :example "example.lisp")
(load "example.lisp")
(fact 2)
(quit)
First, you need to compile the C library. Do something like:
gcc -shared example.c -o libexample.so
Of course, for a complex existing library compilation could be considerably more complex -- if you're wrapping an existing library, it probably comes with some sort of Makefile to help you build it.
Then, in Lisp, use CFFI to define and load the library. This seems to be the main part that you're missing.
(cffi:define-foreign-library libexample
(t (:default "libexample"))) ;; note no .so suffix here
(cffi:use-foreign-library libexample)
This part:
(t (:default "libexample"))
is a conditional which you can use to give different loading instructions for different platforms. (t ...) is the catchall option, much like with COND. You can find the exact syntax in the documentation for define-foreign-library.
You would now normally use cffi:defcfun and so on to define the functions in the library. This is what the SWIG-generated file does for you, so load it:
(load "example.lisp")
You can now call the functions as normal Lisp functions:
(fact 5)
=> 120