It might seem simple, but I can't get it to work. I simply need to read a file where the contents are just one big list
(a b c d)
. . . as is . . . into a list in my program. I have
(let ((ardplst nil))
...
(with-open-file (in ardpfile :direction :input :if-does-not-exist nil)
(when in
(read-sequence ardplst in))
(format t "~a" ardplst))
But it's not working. I get NIL. What am I doing wrong?
What does read-sequence do? It reads some elements from the stream, typically characters (but it depends on the element-type of the stream) and destructively insert them into the input sequence. So, you would collect characters #\(, then #\a, then #\Space, then #\b, etc. However, reading stops as soon as you reach the end of your sequence: with your empty list, that means immediately (you are supposed to pass a buffer, e.g. a vector). In you case, read-sequence returns 0.
The reason you get nil is because your last expression is format, which in the above code outputs to the standard output (because of t) and returns nil. You could have used print, which returns the printed object.
I don't understand why you are explicitely using :if-does-not-exist nil. Are you sure you want to silently skip the task if the file cannot be opened? What if the list you read is empty? You should probably let an error be signaled in case the file is not found.
I would use read while disabling read-time evaluation:
(with-open-file (in my-file)
(let* ((*read-eval* nil)
(list (read in)))
(prog1 list
(check-type list list))))
Note that the default :direction is :input. In my opinion it does not hurt to omit this argument here, though sometimes it can be more readable to write it explicitely.
Related
I don't understand why this code behaves differently in different implementations:
(format t "asdf")
(setq var (read))
In CLISP it behaves as would be expected, with the prompt printed followed by the read, but in SBCL it reads, then outputs. I read a bit on the internet and changed it:
(format t "asdf")
(force-output t)
(setq var (read))
This, again, works fine in CLISP, but in SBCL it still reads, then outputs. I even tried separating it into another function:
(defun output (string)
(format t string)
(force-output t))
(output "asdf")
(setq var (read))
And it still reads, then outputs. Am I not using force-output correctly or is this just an idiosyncrasy of SBCL?
You need to use FINISH-OUTPUT.
In systems with buffered output streams, some output remains in the output buffer until the output buffer is full (then it will be automatically written to the destination) or the output buffer is explicity emptied.
Common Lisp has three functions for that:
FINISH-OUTPUT, attempts to ensure that all output is done and THEN returns.
FORCE-OUTPUT, starts the remaining output, but IMMEDIATELY returns and does NOT wait for all output being done.
CLEAR-OUTPUT, tries to delete any pending output.
Also the T in FORCE-OUTPUT and FORMAT are unfortunately not the same.
force-output / finish-output: T is *terminal-io* and NIL is *standard-output*
FORMAT: T is *standard-output*
this should work:
(format t "asdf")
(finish-output nil) ; note the NIL
(setq var (read))
How do you embed control characters (ie " or ' or :) in cl-dbi queries for sqlite3?
currently using a make-string-output/get-output-stream-string to build a variable that contains quoted data (json). Now, I want to be able to store the data in a sqlite3 db, but I'm obviously building the string wrong, because I get an error
DB Error: unrecognized token: ":" (Code: ERROR)
how do I escape characters in cl-dbi to pass them through to sqlite3?
EDIT - here's a brief passage of the JSON data that I'm trying to store (as text) in the sqlite3 db:
{
"type": "artificial",
Update: GAH! Took me a day to find an errant : in the prepared query string. :/
As far as I can make out, what you are trying to do is generate, in Lisp, some string which contains valid SQL, which itself contains an SQL literal string.
First of all this is an instance of an antipattern I call 'language in a string'. I keep thinking I have a big diatribe about this to point people at but it seems I haven't. Suffice it to say it's kind of the antithesis of what Lisp people have tried to achieve for more than 60 years and it's why we have SQL injection attacks and a lot of the other crap that afflicts us. But that battle is long lost and all we can do is try to avoid actually drowning in the sea of mud and rotting bits of people that now litters the battlefield.
So, to do this you need to be able to do two things.
You need to be able to generate an SQL literal string from a sequence of characters (or from a string). This means you need to know the syntax of literal SQL strings and in particular what characters are legal in them and how you express characters which are not.
You need to be able to interpolate this string into a CL string.
The second of these is trivial: this is what format's ~A directive does. Or if you want to get fancy you could use cl-interpol.
For the first, I don't know the syntax of SQL literal strings, but I will give an example which assumes the following simple rules:
literal strings are delimited by " characters;
the character \ escapes the following character to remove it of any special significance;
all other characters are allowed (this is almost certainly wrong).
Well, there are lots of ways of doing this, all of which involve walking along the sequence of characters looking for the ones that need to be escaped. Here is something reasonably horrible and quick which I wrote. It needs a macro called nlet which is Scheme's 'named let' construct, and it assumes TRO in the implementation (if your implementation does not do this, get one that does).
(defmacro nlet (name bindings &body forms)
"named let"
(multiple-value-bind (vars vals) (values (mapcar (lambda (b)
(etypecase b
(symbol b)
(cons (first b))))
bindings)
(mapcar (lambda (b)
(etypecase b
(symbol 'nil)
(cons
(unless (null (cddr b))
(error "bad binding ~A" b))
(second b))))
bindings))
`(labels ((,name ,vars ,#forms))
(,name ,#vals))))
(defun ->sql-string (seq)
;; turn SEQ (a sequence of characters) into a string representing an
;; SQL literal string (perhaps)
(nlet loupe ((tail (coerce seq 'list))
(accum '()))
(if (null tail)
(coerce (cons #\" (nreverse (cons #\" accum))) 'string)
(destructuring-bind (first . rest) tail
(loupe rest
(case first
((#\\ #\")
(append (list first #\\) accum))
(otherwise
(cons first accum))))))))
So now:
> (->sql-string "foo")
"\"foo\""
> (->sql-string '(#\f #\\ #\o #\" #\o))
"\"f\\\\o\\\"o\""
This is made ugly by the Lisp printer, but (see above) we can see what the strings actually are:
> (format t "~&select x from y where x.y = ~A;~%"
(->sql-string '(#\f #\\ #\o #\" #\o)))
select x from y where x.y = "f\\o\"o";
nil
And you can see that the SQL literal string obeys the rules I set out above.
Before using anything like this check what the rules are, because if you get them wrong you are possibly open to SQL injection attacks.
I was hoping to experiment with cl-async to run a series of external programs with a large combinations of command line arguments. However, I can't figure out how to read the stdout of the processes launched with as:spawn.
I would typically use uiop which makes it easy to capture the process output:
(let ((p (uiop:launch-program ... :output :stream)))
(do-something-else-until-p-is-done)
(format t "~a~%" (read-line (uiop:process-info-output p))))
I've tried both :output :pipe and :output :stream options to as:spawn and executing (as:process-output process-object) in my exit-callback shows the appropriate pipe or async-stream objects but I can't figure out how to read from them.
Can anyone with experience with this library tell how to accomplish this?
So you go to your repl and type:
CL-USER> (documentation 'as:spawn 'function)
And you read whatever comes out (or put your point on the symbol and hit C-c C-d f). If you read it you’ll see that the format for the :input, etc arguments is either :pipe, (:pipe args...), :stream, or (:stream args...) (or some other options). And that :stream behaves similarly to :pipe but gives output of a different type and that for details of args one should look at PIPE-CONNECT so you go and look up the documentation for that. Well it tells you what the options are but it isn’t very useful. What’s the documentation/description of PIPE or STREAM? Well it turns out that pipe is a class and a subclass of STREAMISH. What about PROCESS that’s a class too and it has slots (and accessors) for things like PROCESS-OUTPUT. So what is a good plan for how to figure out what to do next? Here’s a suggestion:
Spawn a long running process (like cat foo.txt -) with :output :stream :input :pipe say
Inspect the result (C-c C-v TAB)
Hopefully it’s an instance of PROCESS. What is it’s output? Inspect that
Hopefully the output is a Gray stream (ASYNC-STREAM). Get it into your repl and see what happens if you try to read from it?
And what about the input? See what type that has and what you can do with it
The above is all speculation. I’ve not tried running any of this but you should. Alternatively go look at the source code for the library. It’s already on your computer and if you can’t find it it’s on GitHub. There are only about half a dozen source files and they’re all small. Just read them and see what you can learn. Or go to the symbol you want to know about and hit M-. to jump straight to its definition. Then read the code. Then see if you can figure out what to do.
I found the answer in the test suite. The output stream can only be processed asynchronously via a read call-back. The following is simple example for posterity
(as:start-event-loop
(lambda ()
(let ((bytes (make-array 0 :element-type '(unsigned-byte 8))))
(as:spawn "./test.sh" '()
:exit-cb (lambda (proc exit-status term-signal)
(declare (ignore proc exit-status term-signal))
(format t "proc output:~%~a"
(babel:octets-to-string bytes)))
:output (list :stream
:read-cb (lambda (pipe stream)
(declare (ignore pipe))
(let ((buf (make-array 128 :element-type '(unsigned-byte 8))))
(loop for n = (read-sequence buf stream)
while (plusp n) do
(setf bytes
(concatenate '(vector (unsigned-byte 8))
bytes
(subseq buf 0 n)))))))))))
with
$ cat test.sh
#!/bin/bash
sleep_time=$((1+$RANDOM%10))
echo "Process $$ will sleep for $sleep_time"
sleep $sleep_time
echo "Process $$ exiting"
yields the expected output
Is there a way, in common lisp, to receive a user input, say "foo", and defvar a global variable *foo*?
For example (which does NOT work):
(defun global-name (s)
"Takes s and changes it to *s*"
(concatenate 'string "*" s "*"))
(defun add-global-var (var)
"defvars a global variable and adds it to *global-list*"
(let ((var-name (global-name var)))
(defvar var-name var)
(push var-name *global-list*)))
; Used like this:
(add-global-var "myvar")
In this case, the var-name is a string, and will not work with defvar.
Déjà vu... I asked these kinds of questions 20+ years ago ;-)
Your question
Yes, you can do that (but no, you do not want to!)
(defun add-global-var (var-name &optional (package *package*))
(let ((var (intern var-name package)))
(proclaim `(special ,var))
(push var *global-list*)))
Please see
proclaim
intern
*package*
Alternatively, you can use a macro as the other answer suggests - in
fact, symbol creation at macroexpansion time (which is part of
compilation) is a very common thing,
cf. gensym.
Your problem
There is little reason to do this though.
Global variables created at run time were not available at compile time
and are, therefore, pretty useless.
Why do you want to do this?
If you want to map strings to values, you are much better off using an
equal hash table.
If you want to integrate with read,
you should call it while binding
*package*
to your internal temp package and then use
symbol-value
to store and retrieve values.
You will use intern to
map "variable names" to the symbols.
This is most likely a XY problem since it's very unusual to need to make a variable with a name made up in runtime. It's very common in compile time, but not runtime. #coredump has already covered compile time macros if that is what you are after.
Here is how you do it though:
(defun add-global-var (var)
"defvars a global variable and adds it to *global-list*"
(let ((var-name (intern (string-upcase (global-name var)))))
(set var-name var)
(push var-name *global-list*)))
set is deprecated, but I doubt it will ever be removed. Implementations might not be able to run as fast though since this is like messing with internals.
Since the names are not from source you you have no good use for the bidnings. because of this I would rather use a hash:
(defvar *bindings* (make-hash-table :test #'eq))
(defun add-binding (var)
(let ((var-name (intern (string-upcase (global-name var)))))
(setf (gethash var-name *bindings*) var)
*bindings*))
A reason to do this is as a part of your own little interpreter symbol table or something. You don't need a list of them since you can get all the keys from the hash as well as get the bound values.
Yes, with a macro:
(defvar *global-list* nil)
I changed global-name so that it also accepts symbols, to avoid thinking about whether the string should be upcased or not. With a symbol, the case is given by readtable-case (you can use uninterned symbols if you want to avoid polluting packages).
(defun global-name (name)
(check-type name (or string symbol))
(intern
(concatenate 'string "*" (string name) "*")))
I named the macro defvar*:
(defmacro defvar* (name)
`(push
(defvar ,(global-name name) ',name)
*global-list*))
Tests:
CL-USER> (defvar* #:foo)
(*FOO*)
CL-USER> (defvar* #:bar)
(*BAR* *FOO*)
Note:
You can also add an optional package argument like in #sds's answer, that's better.
Consider this question. Here the basic problem is the code:
(progv '(op arg) '(1+ 1)
(eval '(op arg)))
The problem here is that progv binds the value to the variable as variable's symbol-value not symbol-function. But, that's obvious because we didn't explicitly suggest which values are functions.
The Plan
So, in order to solve this problem, I thought of manually dynamically binding the variables, to their values based on the type of values. If the values are fboundp then they should be bound to the symbol-function of the variable. A restriction, is that match-if can't be a macro. It has to be a function, because it is called by a funcall.
Macro : functioner:
(defmacro functioner (var val)
`(if (and (symbolp ',val)
(fboundp ',val))
(setf (symbol-function ',var) #',val)
(setf ,var ,val)))
Function: match-if:
(defun match-if (pattern input bindings)
(eval `(and (let ,(mapcar #'(lambda (x) (list (car x))) bindings)
(declare (special ,# (mapcar #'car bindings)))
(loop for i in ',bindings
do (eval `(functioner ,(first i) ,(rest i))))
(eval (second (first ,pattern))))
(pat-match (rest ,pattern) ,input ,bindings))))
Here, the let part declares all the variables lexically (supposedly). Then declare declares them special. Then functioner binds the variables and their values aptly. Then the code in the pattern is evaluated. If the code part is true, then only the pattern-matcher function pat-match is invoked.
The Problem
The problem is that in the function, all it's arguments are evaluated. Thus bindings in the let and declare parts will be replaced by something like :
((v1 . val1)(v2 . val2)(v3 . val3))
not
'((v1 . val1)(v2 . val2)(v3 . val3))
So, it's treated as code, not a list.
So, I'm stuck here. And macros won't help me on this one.
Any help appreciated.
Not the answer you are looking for, but PROGV is a special operator; it is granted the ability to modify the dynamic bindings of variables at runtime; AFAIK, you can't simply hack it to operate on "dynamic function bindings".
The point of progv is to use list of symbols and values that are evaluated, meaning that you can generate symbols at runtime and bind them dynamically to the corresponding values.
You might be able to find a solution with eval but note that if you macroexpand into (eval ...), then you loose the surrounding lexical context, which is generally not what you want ("eval" operates on the null lexical environment). I speculate that you could also have a custom code walker which works on top-level forms but reorganizes them, when it finds your special operator, to bring the context back in, producing something like (eval '(let (...) ...)).