I'm trying to launch node, write JS and read back the answer. Why does the following code not work but hangs at the read-line? echo "console.log(1+1)" | node works fine.
(defvar node-process (uiop:launch-program '("/usr/local/bin/node")
:input :stream
:output :stream
:error-output :stream))
(defvar node-stream (make-two-way-stream (uiop:process-info-output node-process)
(uiop:process-info-input node-process)))
(write-line "console.log(1+1)" node-stream)
(force-output node-stream)
(format t "~s~%" (read-line node-stream))
It appears that node only emit output after EOF is read when in non-interactive mode, so you need to close the input stream before reading.
Add
(close (two-way-stream-input-stream node-stream))
after force-output and before read-line.
Related
I am trying to read a series of web pages with CLISP, if they exist, but I don't understand how open-http works to skip non existing web pages.
I have the following:
(dolist (word '(a b c))
(with-open-stream (stream (ext:open-http
(format nil
"https://en.wikipedia.org/wiki/~a.html"
word)
:if-does-not-exist nil))
(when stream
(print word))))
I want to simply skip a web-page if it doesn't exist, but CLISP seems to hang and returns an "Invalid argument" error.
Could anyone explain how the argument :if-does-not-exist works and/or provide examples of how to use open-http. Thanks!
It does work for me:
(with-open-stream (stream (ext:open-http
"http://stackoverflow.com/questions/234242424242"
:if-does-not-exist nil))
(format t "~&Stream: ~A~%" stream))
Output:
;; connecting to "http://stackoverflow.com/questions/234242424242"...connected...HTTP/1.1 404 Not Found
;; HTML source of Page not found
Stream: NIL
NIL
There is a delay to get the connection, but it works.
If the page does exist:
[7]> (with-open-stream (stream (ext:open-http
"http://stackoverflow.com/questions/36003343/clisp-open-http-example"
:if-does-not-exist nil))
(format t "~&Stream: ~A~%" stream))
;; connecting to "http://stackoverflow.com/questions/36003343/clisp-open-http-example"...connected...HTTP/1.1 200 OK
Stream: #<IO INPUT-BUFFERED SOCKET-STREAM CHARACTER stackoverflow.com:80>
NIL
With Wikipedia I couldn't make it work since Wikipedia.org re-directs it to HTTPS and EXT:OPEN-HTTP neither can handle HTTPS directly, nor it can handle redirects:
Here if HTTPS is used directly:
[10]> (with-open-stream (stream (ext:open-http
"https://en.wikipedia.org/wiki/Common_Lisp"
:if-does-not-exist nil))
(format t "~&Stream: ~A~%" stream))
*** - OPEN-HTTP: "https://en.wikipedia.org/wiki/Common_Lisp" is not an HTTP URL
The following restarts are available:
ABORT :R1 Abort main loop
Break 1 [11]> :r1
If "https" is replaced by "http", CLISP doesn't construct a proper address:
[12]> (with-open-stream (stream (ext:open-http
"http://en.wikipedia.org/wiki/Common_Lisp"
:if-does-not-exist nil))
(format t "~&Stream: ~A~%" stream))
;; connecting to "http://en.wikipedia.org/wiki/Common_Lisp"...connected...HTTP/1.1 301 TLS Redirect --> "https://en.wikipedia.org/wiki/Common_Lisp"
;; connecting to "http://en.wikipedia.orghttps://en.wikipedia.org/wiki/Common_Lisp"...
*** - PARSE-INTEGER: substring "" does not have integer syntax at position 0
The following restarts are available:
ABORT :R1 Abort main loop
Break 1 [13]>
I have a text file with one sentence per line. I would like to lemmatize the worlds in each line using hunspell (-s option). Since I want to have the lemmas of each line separately, it wouldn't make sense to submit the whole text file to hunspell. I do need to send one line after another and have the hunspell output for each line.
Following the answers from How to process input and output streams in Steel Bank Common Lisp?, I was able to send the whole text file for hunspell one line after another but I was not able to capture the output of hunspell for each line. How interact with the process sending the line and reading the output before send another line?
My current code to read the whole text file is
(defun parse-spell-sb (file-in)
(with-open-file (in file-in)
(let ((p (sb-ext:run-program "/opt/local/bin/hunspell" (list "-i" "UTF-8" "-s" "-d" "pt_BR")
:input in :output :stream :wait nil)))
(when p
(unwind-protect
(with-open-stream (o (process-output p))
(loop
:for line := (read-line o nil nil)
:while line
:collect line))
(process-close p))))))
Once more, this code give me the output of hunspell for the whole text file. I would like to have the output of hunspell for each input line separately.
Any idea?
I suppose you have a buffering problem with the program you want to run. For example:
(defun program-stream (program &optional args)
(let ((process (sb-ext:run-program program args
:input :stream
:output :stream
:wait nil
:search t)))
(when process
(make-two-way-stream (sb-ext:process-output process)
(sb-ext:process-input process)))))
Now, on my system, this will work with cat:
CL-USER> (defparameter *stream* (program-stream "cat"))
*STREAM*
CL-USER> (format *stream* "foo bar baz~%")
NIL
CL-USER> (finish-output *stream*) ; will hang without this
NIL
CL-USER> (read-line *stream*)
"foo bar baz"
NIL
CL-USER> (close *stream*)
T
Notice the finish-output – without this, the read will hang. (There's also force-output.)
Python in interactive mode will work, too:
CL-USER> (defparameter *stream* (program-stream "python" '("-i")))
*STREAM*
CL-USER> (loop while (read-char-no-hang *stream*)) ; skip startup message
NIL
CL-USER> (format *stream* "1+2~%")
NIL
CL-USER> (finish-output *stream*)
NIL
CL-USER> (read-line *stream*)
"3"
NIL
CL-USER> (close *stream*)
T
But if you try this without the -i option (or similar options like -u), you'll probably be out of luck, because of the buffering going on. For example, on my system, reading from tr will hang:
CL-USER> (defparameter *stream* (program-stream "tr" '("a-z" "A-Z")))
*STREAM*
CL-USER> (format *stream* "foo bar baz~%")
NIL
CL-USER> (finish-output *stream*)
NIL
CL-USER> (read-line *stream*) ; hangs
; Evaluation aborted on NIL.
CL-USER> (read-char-no-hang *stream*)
NIL
CL-USER> (close *stream*)
T
Since tr doesn't provide a switch to turn off buffering, we'll wrap the call with a pty wrapper (in this case unbuffer from expect):
CL-USER> (defparameter *stream* (program-stream "unbuffer"
'("-p" "tr" "a-z" "A-Z")))
*STREAM*
CL-USER> (format *stream* "foo bar baz~%")
NIL
CL-USER> (finish-output *stream*)
NIL
CL-USER> (read-line *stream*)
"FOO BAR BAZ
"
NIL
CL-USER> (close *stream*)
T
So, long story short: Try using finish-output on the stream before reading. If that doesn't work, check for command line options preventing buffering. If it still doesn't work, you could try wrapping the programm in some kind of pty-wrapper.
I am trying to download and save a PDF but it fails while writing with an EOF-Error.
What would be the correct way of doing this?
(with-open-file (file "/home/*/test.pdf"
:direction :io
:if-does-not-exist :create
:if-exists :supersede
:element-type '(unsigned-byte 8))
(let ((input (drakma:http-request "http://www.fractalconcept.com/ex.pdf"
:want-stream t)))
(awhile (read-byte input)
(write-byte it file))
(close input)))
The solution was that I forgot to use the two optional parameters of read-byte.
The correct way would be to set eof-error-p and eof-value to nil:
(with-open-file (file "/home/*/test.pdf"
:direction :output
:if-does-not-exist :create
:if-exists :supersede
:element-type '(unsigned-byte 8))
(let ((input (drakma:http-request "http://www.fractalconcept.com/ex.pdf"
:want-stream t)))
(awhile (read-byte input nil nil)
(write-byte it file))
(close input)))
(ql:quickload "trivial-download")
(trivial-download:download URL FILE)
; Does pretty much exactly what your code does but in bigger chunks and can show a progress bar.
; QuickLisp can be found at http://QuickLisp.org.
AWHILE is found in package ARNESI.
In REPL, we can get its output normally:
(shell "head -3 ~/misc.lisp")
(asdf:make-build 'stumpwm :type :program :monolithic t
:move-here "."
:name-suffix ""
However, we get nil from (shell "head -3 ~/misc.lisp") in slime.
Sincerely!
I don't know which Lisp implementation you're using, but normally you should need a package to run shell command like:
trivial-shell
inferior-shell
clesh (build on top of trivial-shell)
use #'run-shell-command instead of shell:
(let ((str (ext:run-shell-command cmd :output :stream)))
(loop for line = (read-line str nil)
until (null line)
do (print line)))
it works now.
I am writing a common lisp program that needs to handle the output of a command. However, when I try to use the results in another function, I only get a NIL as return-value.
Here is the function I use to run commands:
(defun run-command (command &optional arguments)
(with-open-stream (pipe
(ext:run-program command :arguments arguments
:output :stream :wait nil))
(loop
:for line = (read-line pipe nil nil)
:while line :collect line)))
Which, when run by itself gives:
CL-USER> (run-command "ls" '("-l" "/tmp/test"))
("-rw-r--r-- 1 petergil petergil 0 2011-06-23 22:02 /tmp/test")
However, when I run it through a function, only NIL is returned:
(defun sh-ls (filename)
(run-command "ls" '( "-l" filename)))
CL-USER> (sh-ls "/tmp/test")
NIL
How can I use the results in my functions?
Try this:
(defun sh-ls (filename)
(run-command "ls" (list "-l" filename)))
The '("-l" filename) is quoting the list, AND the symbol 'filename', rather than evaluating filename.
You can also use backquote ` before sexpr and , before filename to evaluate it:
(defun sh-ls (filename)
(run-command "ls" `("-l" ,filename)))