I'm using the following code to grab the url http://brandonhsiao.com/essays.html:
(defparameter *new-line* '(#\Return #\Newline))
(defun read-url (host port path)
(let ((sock (usocket:socket-connect host port)))
(format (usocket:socket-stream sock) "~A"
(concatenate 'string
"GET " path " HTTP/1.1" *new-line*
"Connection: close" *new-line* *new-line*))
(force-output (usocket:socket-stream sock))
(do ((line
(read-line (usocket:socket-stream sock) nil)
(read-line (usocket:socket-stream sock) nil))
(all ""))
((not line) all)
(setf all (concatenate 'string all line *new-line*)))))
(print (read-url "brandonhsiao.com" 80 "/essays.html"))
This gives me a 400 Bad Request error, but when I visit http://brandonhsiao.com/essays.html with Firefox, everything is fine. What am I doing wrong?
It looks like I needed to include the host.
(defparameter *new-line* '(#\Return #\Newline))
(defun read-url (host port path)
(let ((sock (usocket:socket-connect host port)))
(format (usocket:socket-stream sock) "~A"
(concatenate 'string
"GET " path " HTTP/1.1" *new-line*
"Host: " host *new-line* *new-line*))
(force-output (usocket:socket-stream sock))
(do ((line
(read-line (usocket:socket-stream sock) nil)
(read-line (usocket:socket-stream sock) nil))
(all ""))
((not line) all)
(setf all (concatenate 'string all line " ")))))
Related
I've found this example from the Common Lisp Cookbook which shows how to start a TCP server with usocket.
The example creates a socket object and establishes a connection, and then writes to the socket. In case of error, the socket write is wrapped in an unwind-protect which will close the socket so it can be reused. I've rewritten the example to cause the error, but when I run this multiple times, I get a USOCKET:ADDRESS-IN-USE-ERROR. The behavior is the same if I remove the socket-close function calls.
(load "~/quicklisp/setup.lisp")
(ql:quickload "usocket")
(let* ((socket (usocket:socket-listen "localhost" 8080))
(connection (usocket:socket-accept socket)))
(unwind-protect
(progn
(error "error 1"))
(progn
(usocket:socket-close connection)
(usocket:socket-close socket)
(print "Error clean up"))))
Unhandled USOCKET:ADDRESS-IN-USE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
{10005E85B3}>:
Condition USOCKET:ADDRESS-IN-USE-ERROR was signalled.
Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {10005E85B3}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<USOCKET:ADDRESS-IN-USE-ERROR {1003FDC063}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<USOCKET:ADDRESS-IN-USE-ERROR {1003FDC063}>)
2: (INVOKE-DEBUGGER #<USOCKET:ADDRESS-IN-USE-ERROR {1003FDC063}>)
3: (ERROR #<USOCKET:ADDRESS-IN-USE-ERROR {1003FDC063}>)
4: (USOCKET:SOCKET-LISTEN "localhost" 8080 :REUSEADDRESS NIL :REUSE-ADDRESS NIL :BACKLOG 5 :ELEMENT-TYPE CHARACTER)
5: ((LAMBDA NIL :IN "/home/sam/test/serve.lisp"))
6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (LET* ((SOCKET (USOCKET:SOCKET-LISTEN "localhost" 8080)) (CONNECTION (USOCKET:SOCKET-ACCEPT SOCKET))) (UNWIND-PROTECT (PROGN (ERROR "error 1")) (PROGN (USOCKET:SOCKET-CLOSE CONNECTION) (USOCKET:SOCKET-CLOSE SOCKET) (PRINT "Error clean up")))) #<NULL-LEXENV>)
7: (EVAL-TLF (LET* ((SOCKET (USOCKET:SOCKET-LISTEN "localhost" 8080)) (CONNECTION (USOCKET:SOCKET-ACCEPT SOCKET))) (UNWIND-PROTECT (PROGN (ERROR "error 1")) (PROGN (USOCKET:SOCKET-CLOSE CONNECTION) (USOCKET:SOCKET-CLOSE SOCKET) (PRINT "Error clean up")))) 2 NIL)
8: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (LET* ((SOCKET (USOCKET:SOCKET-LISTEN "localhost" 8080)) (CONNECTION (USOCKET:SOCKET-ACCEPT SOCKET))) (UNWIND-PROTECT (PROGN (ERROR "error 1")) (PROGN (USOCKET:SOCKET-CLOSE CONNECTION) (USOCKET:SOCKET-CLOSE SOCKET) (PRINT "Error clean up")))) 2)
9: ((LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) (LET* ((SOCKET (USOCKET:SOCKET-LISTEN "localhost" 8080)) (CONNECTION (USOCKET:SOCKET-ACCEPT SOCKET))) (UNWIND-PROTECT (PROGN (ERROR "error 1")) (PROGN (USOCKET:SOCKET-CLOSE CONNECTION) (USOCKET:SOCKET-CLOSE SOCKET) (PRINT "Error clean up")))) :CURRENT-INDEX 2)
10: (SB-C::%DO-FORMS-FROM-INFO #<CLOSURE (LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) {1001B7128B}> #<SB-C::SOURCE-INFO {1001B71243}> SB-C::INPUT-ERROR-IN-LOAD)
11: (SB-INT:LOAD-AS-SOURCE #<SB-SYS:FD-STREAM for "file /home/sam/test/serve.lisp" {1001B66763}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
12: ((FLET SB-FASL::THUNK :IN LOAD))
13: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<CLOSURE (FLET SB-FASL::THUNK :IN LOAD) {7FFFF63E769B}> #<SB-SYS:FD-STREAM for "file /home/sam/test/serve.lisp" {1001B66763}>)
14: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<SB-SYS:FD-STREAM for "file /home/sam/test/serve.lisp" {1001B66763}> NIL)
15: (LOAD #<SB-SYS:FD-STREAM for "file /home/sam/test/serve.lisp" {1001B66763}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :DEFAULT)
16: ((FLET SB-IMPL::LOAD-SCRIPT :IN SB-IMPL::PROCESS-SCRIPT) #<SB-SYS:FD-STREAM for "file /home/sam/test/serve.lisp" {1001B66763}>)
17: ((FLET SB-UNIX::BODY :IN SB-IMPL::PROCESS-SCRIPT))
18: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-IMPL::PROCESS-SCRIPT))
19: (SB-IMPL::PROCESS-SCRIPT "serve.lisp")
20: (SB-IMPL::TOPLEVEL-INIT)
21: ((FLET SB-UNIX::BODY :IN SAVE-LISP-AND-DIE))
22: ((FLET "WITHOUT-INTERRUPTS-BODY-36" :IN SAVE-LISP-AND-DIE))
23: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))
unhandled condition in --disable-debugger mode, quitting
The reason you get this is because of the nature of the TCP protocol: the connection is in a state called TIME-WAIT in the TCP state machine described by RFC793. A diagram of the state machine is on page 23 of RFC793.
The interesting bit of the state machine is when one end (who I'll call 'you') wants to close the connection – this is known as an 'active close', and in this case it's what you are initiating by the socket-close calls. I'll call the other end 'them'. The normal sequence of events for an active close is:
you send them a FIN packet;
they ACK your FIN and send in turn a FIN;
you ACK their FIN.
Now it's important to remember that any of these packets (their ACK and FIN are the same packet usually and I think always) can get lost, and the state machine needs to recover from this.
There's one particularly interesting packet which is the last ACK: it's particularly interesting because it's the last packet ever sent, which means that you have no way of knowing whether it reached them.
So consider the situation from both ends
From their end: they've sent a FIN and are waiting for your ACK for it. Now:
either the ACK arrives in due course, in which case they know everything is wound up and they can dismantle everything involved in the connection.
or, after waiting for the prescribed time, the ACK doesn't arrive, so they must assume that either their FIN was lost or your ACK to their FIN was lost, and so they must resend the FIN and go back to waiting for the ACK.
From your end: you've got their FIN and have sent the last ACK. But you have no idea whether that ACK ever got to them. So you wait for a prescribed time in order to give them a chance to realise that the ACK did not get their and resend their FIN. During this wait you can't dismantle the connection, because at any point you may get another FIN.
This waiting state is known as TIME-WAIT, and during it the endpoint of the connection can't be reused. And this is the problem you are seeing.
You need to sit in TIME-WAIT for twice the maximum segment lifetime (MSL): the MSL is how long a packet can be expected to sit in the network.
There are other waiting-states which can occur prior to TIME-WAIT of course, if earlier packets get lost. But TIME-WAIT is the only one which always occurs.
TIME-WAIT is often called TIME_WAIT due to languages which can't handle hyphens in names.
I am having some issues in enabling CORS on hunchentoot:
(hunchentoot:define-easy-handler (one-api :uri *one-endpoint*) ()
(when (boundp '*acceptor*)
(setf (hunchentoot:header-out "Access-Control-Allow-Origin") "*")
(setf (hunchentoot:header-out "Accept") "*/*")
(setf (hunchentoot:header-out "Access-Control-Allow-Headers") "Content-Type, Accept, Origin")
(setf (hunchentoot:header-out "Access-Control-Allow-Methods") "POST, GET, OPTIONS, PUT, DELETE")
(setf (hunchentoot:header-out "Access-Control-Allow-Origin") "*")
(setf (hunchentoot:content-type*) "text/html"))
(let* ((raw-data (hunchentoot:raw-post-data :force-text t)))
(funcall callback raw-data))))
But still not work, anything that I am doing wrong?
The following worked for me:
(setf (header-out "Access-Control-Allow-Origin") "*")
(setf (header-out "Access-Control-Allow-Methods") "POST,GET,OPTIONS,DELETE,PUT")
(setf (header-out "Access-Control-Max-Age") 1000)
(setf (header-out "Access-Control-Allow-Headers") "x-requested-with, Content-Encoding, Content-Type, origin, authorization, accept, client-security-token")
(setf (header-out "Content-Type") "text/json")
The binary of my web-app built on CI fails to run, and succeeds if built locally. What could cause those differences, what would cause Hunchentoot to fail with this message ?
Here are the different stacktraces:
<INFO> [14:42:11] weblocks/server server.lisp (start) -
Starting weblocks WEBLOCKS/SERVER::PORT: 4000
WEBLOCKS/SERVER::SERVER-TYPE: :HUNCHENTOOT DEBUG: T
<INFO> [14:42:11] weblocks/server server.lisp (start-server) -
Starting webserver on WEBLOCKS/SERVER::INTERFACE: "localhost"
WEBLOCKS/SERVER::PORT: 4000 DEBUG: T
Unhandled SIMPLE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
{10005C85B3}>:
:HUNCHENTOOT is unknown handler.
Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {10005C85B3}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<SIMPLE-ERROR "~S is unknown handler." {100495FEC3}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<SIMPLE-ERROR "~S is unknown handler." {100495FEC3}>)
2: (INVOKE-DEBUGGER #<SIMPLE-ERROR "~S is unknown handler." {100495FEC3}>)
3: (UIOP/IMAGE:HANDLE-FATAL-CONDITION #<SIMPLE-ERROR "~S is unknown handler." {100495FEC3}>)
4: (SB-KERNEL::%SIGNAL #<SIMPLE-ERROR "~S is unknown handler." {100495FEC3}>)
5: (ERROR "~S is unknown handler." :HUNCHENTOOT)
6: (CLACK.UTIL:FIND-HANDLER :HUNCHENTOOT)
7: (CLACK:CLACKUP #<CLOSURE (LAMBDA (LACK.MIDDLEWARE.SESSION::ENV) :IN "/home/lisp/quicklisp/dists/quicklisp/software/lack-20181018-git/src/middleware/session.lisp") {100467727B}> :ADDRESS "localhost" :SERVER :HUNCHENTOOT :PORT 4000 :DEBUG T)
8: (WEBLOCKS/SERVER::START-SERVER #<SERVER port=4000 stopped> :DEBUG T)
9: ((LAMBDA (#:G0 &REST #:G1) :IN WEBLOCKS/SERVER:START) NIL)
10: (WEBLOCKS/SERVER:START :DEBUG T :PORT 4000 :INTERFACE "localhost" :SERVER-TYPE :HUNCHENTOOT)
11: (TORRENTS:MAIN)
12: ((LAMBDA NIL :IN UIOP/IMAGE:RESTORE-IMAGE))
13: (UIOP/IMAGE:CALL-WITH-FATAL-CONDITION-HANDLER #<CLOSURE (LAMBDA NIL :IN UIOP/IMAGE:RESTORE-IMAGE) {1004410D9B}>)
14: ((FLET SB-UNIX::BODY :IN SAVE-LISP-AND-DIE))
15: ((FLET "WITHOUT-INTERRUPTS-BODY-34" :IN SAVE-LISP-AND-DIE))
16: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))
unhandled condition in --disable-debugger mode, quitting
and locally:
<INFO> [17:11:49] weblocks/server server.lisp (start) -
Starting weblocks WEBLOCKS/SERVER::PORT: 4001
WEBLOCKS/SERVER::SERVER-TYPE: :HUNCHENTOOT DEBUG: T
<INFO> [17:11:49] weblocks/server server.lisp (start-server) -
Starting webserver on WEBLOCKS/SERVER::INTERFACE: "localhost"
WEBLOCKS/SERVER::PORT: 4001 DEBUG: T
_
and it keeps listening.
They run with the same framework version (latest), they seem to run the same Quicklisp dist version (see lack-20181018 on the first stacktrace), which is my local version. The CI is built on daewok/lisp-devel-docker.
I use this starting point:
(defun start ()
(weblocks/debug:on)
(weblocks/server:start :port *port*))
(defun stop ()
(weblocks/server:stop))
(defun main ()
(defvar *port* (find-port:find-port))
(start)
(handler-case (bt:join-thread (find-if (lambda (th)
(search "hunchentoot" (bt:thread-name th)))
(bt:all-threads)))
(#+sbcl sb-sys:interactive-interrupt
#+ccl ccl:interrupt-signal-condition
#+clisp system::simple-interrupt-condition
#+ecl ext:interactive-interrupt
#+allegro excl:interrupt-signal
() (progn
(format *error-output* "Aborting.~&")
;; (weblocks:stop)
(uiop:quit 1))
;; for others, unhandled errors (we might want to do the same).
(error (c) (format t "Woops, an unknown error occured:~&~a~&" c)))))
Thanks again.
Adding the dependency
:clack-handler-hunchentoot
in my .asd did it. Thanks jkiiski.
/etc/nginx/sites-enabled/default:
location /pine {
add_header Access-Control-Allow-Origin http://localhost:6285;
add_header Access-Control-Allow-Credentials true;
access_by_lua_file /home/akuznetsov/auth.lua;
proxy_pass http://localhost:9100;
}
auth.lua (version 1, works fine):
ngx.req.set_header('x-user-id', ngx.var.cookie_sessionid)
auth.lua (version 2, not working):
ngx.req.set_header('x-user-id', 'session:' .. ngx.var.cookie_sessionid)
in /var/log/nginx/error.log i get this error:
2016/04/06 16:13:10 [error] 14183#0: *1 lua entry thread aborted: runtime error: /home/akuznetsov/auth.lua:2: attempt to concatenate field 'cookie_sessionid' (a nil value)
stack traceback:
coroutine 0:
/home/akuznetsov/auth.lua:2: in function </home/akuznetsov/auth.lua:1>, client: 127.0.0.1, server: localhost, request: "OPTIONS /pine HTTP/1.1", host: "localhost", referrer: "http://localhost:6285/chart/kjckCBcG/?pine=http://localhost/pine"
What's wrong with concat?
ngx.var.cookie_sessionid is nil and just as message tell you, you can't concatenate (i.e. ..) that. Provide if check with logic to handle this case or use ngx.req.set_header('x-user-id', 'session:' .. ngx.var.cookie_sessionid or "") if you okay with using empty string as default.
If you build SampleApp with the latest (i.e. May 25th's) JWrapper, it won't start on windows. According to SampleApp-SampleApp..log, it appears to be trying to load a MacOSX class (JWOSXEventListener) at runtime:
87342876 (+ 0) STDOUT test
87342876 (+ 0) STDERR test
87342876 (+ 0) [LogFolderCleaner] Keeping Wrapper-2015-05-26-17-42-08-852.log
87342876 (+ 0) [LogFolderCleaner] Keeping SampleApp-SampleApp-2015-05-26-17-42-22-876.log
87342876 (+ 0) [LogFolderCleaner] Keeping GenericUpdater-2015-05-26-17-42-18-945.log
87342876 (+ 0) [JWrapper] App Bundle Name: SampleApp
87342876 (+ 0) [JWrapper] App Bundle Version: 00036171806
87342876 (+ 0) [JWrapper] Virtual App: SampleApp
87342876 (+ 0) [JWrapper] Update URL: http://0.0.254.254/
87342876 (+ 0) [JWrapper] App Folder: C:\Users\christop\AppData\Roaming\JWrapper-SampleApp\JWrapper-SampleApp-00036171806-complete
87342876 (+ 0) [JWrapper] JRE: C:\Users\christop\AppData\Roaming\JWrapper-SampleApp\JWrapper-Windows32JRE-00036170991-complete
87342876 (+ 0) [JWrapper] JRE Version: 1.7.0_05
87342876 (+ 0) [JWrapper] Proxy: null
87342876 (+ 0) [EDT Exception Printer] Catching all EDT and uncaught exceptions and printing
87342892 (+ 16) [JWrapperNative] Detected 32-bit architecture
87342892 (+ 0) [JWrapperNative] Detected Windows OS
87342892 (+ 0) [JWrapper] Is Process DPI Aware: true
87342892 (+ 0) [JWrapper] sun.java2d.dpiaware: null
87342892 (+ 0) [JWDetectedProxy] No proxy configuration found to load.
87342892 (+ 0) JVM Option: -Xmx256m
87342892 (+ 0) [JWrapper] JVM Home: C:\Users\christop\AppData\Roaming\JWrapper-SampleApp\JWrapper-Windows32JRE-00036170991-complete
87342908 (+ 16) [PermissionsUtil] Setting permissions for C:\Users\christop\AppData\Roaming\JWrapper-SampleApp\JWApps\JRE-LastSuccessfulOptions-JWrapper-Windows32JRE-00036170991-complete to F
87342908 (+ 0) [PermissionsUtil] Finished setting permissions.
87342908 (+ 0) [JWrapper] Wrote JVM Options OK
87342908 (+ 0) [AWT/Swing Exception] (STDERR) Exception on thread main (java.lang.NoClassDefFoundError: jwrapper/hidden/events/JWOSXEventListener)
87342923 (+ 15) java.lang.NoClassDefFoundError: jwrapper/hidden/events/JWOSXEventListener
87342923 (+ 0) at java.lang.Class.forName0(Native Method)
87342923 (+ 0) at java.lang.Class.forName(Unknown Source)
87342923 (+ 0) at jwrapper.JWrapper.main(JWrapper.java:290)
87342923 (+ 0) Caused by: java.lang.ClassNotFoundException: jwrapper.hidden.events.JWOSXEventListener
87342923 (+ 0) at java.net.URLClassLoader$1.run(Unknown Source)
87342923 (+ 0) at java.net.URLClassLoader$1.run(Unknown Source)
87342923 (+ 0) at java.security.AccessController.doPrivileged(Native Method)
87342923 (+ 0) at java.net.URLClassLoader.findClass(Unknown Source)
87342923 (+ 0) at java.lang.ClassLoader.loadClass(Unknown Source)
87342923 (+ 0) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
87342923 (+ 0) at java.lang.ClassLoader.loadClass(Unknown Source)
87342923 (+ 0) ... 3 more
This is where JWrapper loads the app class, the exception is due to the class failing to load, this is due to a bug in the sample app code which references a class which has been moved in the latest JW build.