Doing a SWIG tutorial, and using the example.c, example.i as they provided there. I generated lisp file with swig -cffi example.i.
But when I run test.lisp with SBCL, i get a complaint about undefined alien function, as well as complaints when compiling the example.lisp itself. I'm pretty sure I still have to compile my example.c into a library and then somehow tell SBCL to load it! But the docs are very scant on this, except for this.
Can somebody tell me how to do this or is there a better way than SWIG to automatically generate CFFI bindings from C/C++ sources??
sbcl output:
...
;
; caught STYLE-WARNING:
; Undefined alien: "fact"
;
; compilation unit finished
; caught 1 STYLE-WARNING condition
;
; caught STYLE-WARNING:
; Undefined alien: "my_mod"
...
test.lisp
;call C functions defined in example.c
(require :cffi)
;;;(require :example "example.lisp")
(load "example.lisp")
(fact 2)
(quit)
First, you need to compile the C library. Do something like:
gcc -shared example.c -o libexample.so
Of course, for a complex existing library compilation could be considerably more complex -- if you're wrapping an existing library, it probably comes with some sort of Makefile to help you build it.
Then, in Lisp, use CFFI to define and load the library. This seems to be the main part that you're missing.
(cffi:define-foreign-library libexample
(t (:default "libexample"))) ;; note no .so suffix here
(cffi:use-foreign-library libexample)
This part:
(t (:default "libexample"))
is a conditional which you can use to give different loading instructions for different platforms. (t ...) is the catchall option, much like with COND. You can find the exact syntax in the documentation for define-foreign-library.
You would now normally use cffi:defcfun and so on to define the functions in the library. This is what the SWIG-generated file does for you, so load it:
(load "example.lisp")
You can now call the functions as normal Lisp functions:
(fact 5)
=> 120
Related
I've built a library that wraps custom C code and thinking about the best way to build the shared library as part of the ASDF load. The makefile is conditionalised for various OSs, so it could be as simple as uiop:run-program ..., but I thought I'd ask here if there were a more standard idiom for this.
Because the C code is specific to this application, it won't be available through a package manager and must be built specifically for each users machine. I'm fine with documenting a manual build, but if I can smooth things for the user I will. I notice that Python seems to have some kind of automated way of building libs for their CFFI and wonder if there's something for CL.
For an answer to my own question: there seems to be neither a de-facto way of doing this (based on a search of github asd files), nor a definitive method in the ASDF best practices, though there are some ideas to be gleaned from that document.
I'll put my implementation out as a suggested idiom for this use case, along with some possible alternatives. Hopefully some of the ASDF experts here will correct any misunderstandings.
;; Define a makefile as a type of source file for the system
(defclass makefile (source-file) ((type :initform "m")))
;; tell ASDF how to compile it
(defmethod perform ((o load-op) (c makefile)) t)
(defmethod perform ((o compile-op) (c makefile))
(let* ((lib-dir (system-relative-pathname "cephes" "scipy-cephes"))
(lib (make-pathname :directory `(:relative ,(namestring lib-dir))
:name "libmd"
:type #+unix "so" #+(or windows win32) "dll"))
(built (probe-file (namestring lib))))
(if built
(format *error-output* "Library ~S exists, skipping build" lib)
(format *error-output* "Building ~S~%" lib))
(unless built
(run-program (format nil "cd ~S && make" (namestring lib-dir)) :output t))))
(defsystem "cephes"
:description "Wrapper for the Cephes Mathematical Library"
:version (:read-file-form "version.sexp")
:license "MS-PL"
:depends-on ("cffi")
:serial t
:components ((:module "libmd"
:components ((:makefile "makefile")))
(:file "package")
(:file "init")
(:file "cephes")))
This works fine, on both MS Windows and UNIX. Adding a method to perform seems to be the most common method on github.
An alternative might be to use a build-op, as described in building a system. The description
Some systems offer operations that are neither loading in the current
image, nor testing. Whichever operation a system is meant to be used
with, you may use it with:
(asdf:make :foobar)
This will invoke build-op, which in turn will depend on the
build-operation for the system, if defined, or load-op if not.
Therefore, for usual Lisp systems that want you to load them, the
above will be equivalent to (asdf:load-system :foobar), but for other
Lisp systems, e.g. one that creates a shell command-line executable,
(asdf:make ...) will do the Right Thing™, whatever that Right Thing™
is.
suggest to me that this is rather close to the idea of building a C library, and it would map nicely to the mental model of using a makefile and the asdf:make command. I didn't find too many examples in the wild of this being used though and technically we are loading the C lib into the existing image.
Another point that could be reconsidered is the detection of an existing shared library to avoid the rebuild. make will avoid recompiling if the shared library exists, but will still call the linker again. This causes errors because it can't write to the shared library when it's in use, at least on MS Windows. The ASDF example used Lisp code to detect the existence of the library and avoiding recompilation, but an alternative might be to use output-files.
The ASDF docs are a bit muddled on the purpose of output-files and there are no examples that make their intentions clear, but in the manual section on creating new operations we have:
output-files If your perform method has any output, you must define a
method for this function. for ASDF to determine where the outputs of
performing operation lie.
which suggests that defining the shared library (libmd.so or libmd.dll) is the recommended way to avoid a recompilation if the output-files already exists.
Finally, the C library could be considered a secondary system, cephes/libmd in this case, and added to the :depends-on clause in the main system. The section on other secondary systems demonstrates building an executable this way, with build-op. Except for the fact that this is building an executable and hard-codes ".exe" it seems to map well onto the use case:
To build an executable, define a system as follows (in this case, it's
a secondary system, but it could also be a primary system). You will
be able to create an executable file foobar-command by evaluating
(asdf:make :foobar/executable):
(defsystem "foobar/executable"
:build-operation program-op
:build-pathname "foobar-command" ;; shell name
:entry-point "foobar::start-foobar" ;; thunk
:depends-on ("foobar")
:components ((:file "main")))
The build-pathname gives the name of the executable; a .exe type will
be automatically added on Windows.
I didn't use this method because the secondary system would look almost exactly like the primary one does now, but would be slightly less understandable.
I would like to use some ps macros in a .parenscript file. The macros are in a library that will be loaded with quicklisp. I am using sigil to compile the .parenscript file.
I have tried this at the top of the parenscript file:
(lisp
(progn
(ql:quickload 'paren6)
(use-package :paren6)))
but the macro package does not get "used".
Loading the libraries from the command line works:
>sigil --eval "(progn (ql:quickload 'paren6) (use-package :paren6))" sample.parenscript
But it feels clunky, and sigil needs a small hack to prevent it from dumping the output from quickload into the javascript output - indicating that no one else is doing things this way.
I am using sigil because I prefer to build .parenscript -> .js from the command line. Alternatives to sigil will be considered.
What is the best way to indicate a quicklisp dependency for a .parenscript file?
Tracked this one down: Sigil sets *package* to ps except when evaluating lisp forms, where it's left as common-lisp-user so your use-package was affecting the wrong package, and your symbols went wonky.
Try (use-package 'paren6 'ps)
I am trying to get live-coding to work in lisp. i have the file t.cl which contains only this line: (loop(write(- 2 1))). Now, when i run the file in bash with sbcl --load t.cl --eval '(quit)', it runs the line, but when I try to edit the file in another terminal and save it while it runs, nothing changes ..
Why your example fails
When running sbcl --load t.cl --eval '(quit)' in a shell, what this does is spin-up a SBCL Lisp image in a process, compile the file and run it. You then modify the file and save it to your disk. This last action is of no concern to the already running SBCL process, which has already compiled the previous file. SBCL read the file once when you asked it to, once it has the compiled instructions to run, it has no reason to look at the file again unless you explicitly ask it to.
A 'live' example with Emacs+SLIME
In order to perform 'live' changes to your running program, you must interact with the already running Lisp image. This is easily doable with Emacs+Slime. You can, for example, have a loop like so:
(defun foo (x) (+ x 3))
(dotimes (it 20)
(format t "~A~%" (foo it))
(sleep 1))
and then recompile foo during execution within the REPL with a new definition:
(defun foo (x) (+ x 100))
Another thread will be used to recompile the function. The new function will be used for future calls as soon as its compilation is finished.
The output in the REPL will look like:
3
4
5
CL-USER> (defun foo (x) (+ x 100))
WARNING: redefining COMMON-LISP-USER::FOO in DEFUN
FOO
103
104
105
...
This would also work with the new definition of foo being compiled from another file as opposed to being entered directly in the REPL.
Working from the system shell
While you can already use the example above for development purposes, you might want to interact with a running SBCL Lisp image from the shell. I am not aware of how to do that. For your exact example, you want to get SBCL to reload eventual files that you have modified. A brief look at the SBCL manual doesn't seem to provide ways to pipe lisp code to an already running SBCL process.
Here's a strange situation. I have this code:
(eval-when (:compile-toplevel :load-toplevel :execute)
(ql:quickload "cffi-grovel")
(setf cffi-grovel::*cc* "mpicc")) ; <--- this is the line it complains about.
Which I belive has to load cffi-grovel package before setting cffi-grovel::*cc* variable. When this form is executed from SLIME it works, but it doesn't work when loaded directly by SBCL, here's the output:
$ sbcl --noinfo
* (ql:quickload "cl-mpi")
debugger invoked on a LOAD-SYSTEM-DEFINITION-ERROR in thread
#<THREAD "main thread" RUNNING {10029C0E43}>:
Error while trying to load definition for system cl-mpi from pathname
/home/wvxvw/quicklisp/local-projects/cl-mpi/cl-mpi.asd:
READ error during COMPILE-FILE:
Package CFFI-GROVEL does not exist.
Line: 6, Column: 25, File-Position: 264
< restarts ... >
* (ql:quickload "cffi-grovel")
To load "cffi-grovel":
Load 1 ASDF system:
cffi-grovel
; Loading "cffi-grovel"
..
("cffi-grovel")
* (ql:quickload "cl-mpi")
To load "cffi-grovel":
Load 1 ASDF system:
cffi-grovel
; Loading "cffi-grovel"
To load "cl-mpi":
Load 1 ASDF system:
cl-mpi
; Loading "cl-mpi"
; mpicc -m64 ...
; ...
.
("cl-mpi")
Why does it fail the first time?
PS. I also tried #.cffi-grovel::*cc* instead - same result.
It fails because Lisp reads every form before executing it. And when it reads it, package cffi-grovel indeed does not exist yet, because cffi-grovel is loaded at execution time (whatever it means for form wrapped with eval-when).
Try spliting the eval-when form into two eval-whens: ql:quickload and setf. Or write something like this:
(setf (symbol-value (find-symbol "*CC*" "CFFI-GROVEL"))
"mpicc")
monoid gives a good answer. Here is a slightly shorter form.
Find the symbol at runtime, when the package actually exists. Note the use of SET, not SETF here:
(set (find-symbol "*CC*" "CFFI-GROVEL") "mpicc")
To make it more robust, one would need to check that FIND-SYMBOL actually finds the symbol.
The following function also gives you meaningful error messages and restarts:
(defun set-runtime-symbol (name package value)
(assert (find-package package) (package))
(assert (find-symbol name package) (name))
(set (find-symbol name package) value))
Alternatively use two EVAL-WHEN statements, instead of one.
OK, finally figured it I can do it like so:
(setf #.(intern "cffi-grovel::*cc*") "mpicc")
But I'm not sure how much this is fail-safe.
I've build some toy C++ library to quickly create a Qt window from Lisp. I know that common-qt exists, I'm just trying to learn how to use cffi.
Right now, I have 4 binded functions :
create-application : create a QApplication and return a pointer
create-window : create a QMainWindow and return a poiner
show : show the window specified as argument
exec : Qt exec function
Here is a lisp code that work perfectly :
(defctype t-app :pointer)
(defctype t-window :pointer)
(defcfun (create-application "create_application" ) t-app)
(defcfun (exec "exec") :void (app t-app))
(defcfun (create-window-aalt "create_window_aalt") t-window)
(defcfun (show "show") :void (o t-window))
(defparameter a (create-application))
(defparameter w (create-window-aalt))
(show w)
(exec a)
But if I use LET or LET*...I have a memory fault !
(let* ((a (create-application)) (w (create-window-aalt)))
(show w)
(exec a))
CORRUPTION WARNING in SBCL pid 1312(tid 140737353860992):
Memory fault at a556508 (pc=0x7ffff659b7f1, sp=0x7ffff2bbe688)
The integrity of this image is possibly compromised.
Exiting.
Does someone know why ?
I am using SBCL :
env LD_LIBRARY_PATH=`pwd` \
env LD_PRELOAD=/usr/lib/libQtGui.so.4 \
sbcl --script aalt.lisp
Thanks.
I would suggest you do the following:
Since you are writing library C++ and using its symbols from Lisp, make sure that you use extern "C" declarations - those are needed to ensure that C++ compiler does not mangle names.
Check that your library works in a C (not C++) application. This will ensure that the library itself is working (e.g., it does not throw C++ exceptions).
UPD:
I've tried to run your code and had the same crash. The problem seems to be in your create_application function. I've attached my fixed version of this function at http://paste.lisp.org/display/138049.
Concretely, there are 2 problems:
create_application allocated v on stack. Subsequent code (i.e., the let binding) overwrites it.
argv should be NULL-terminated. I.e., it should contain argc+1 elements - the last element is NULL. (Qt seems to not use this, but it is good habit to write code according to specs).
In your case, the stack allocation is the problem - it seems that let binding overwrites the value of v on stack, crashing SBCL. Using malloc or new to allocate argv on heap fixes this issue.