I am trying to disable Hunchentoot's caching of pages, to streamline web development on local host.
As I understand, the function no-cache is meant to be used, but I'm unsure how to incorporate this. I currently have the following code that evaluates without error, however this is relates to dynamically served content, so I am unable to check whether the no-cache function is correct. Also I have no idea how to add handler functions to handlers.
(hunchentoot:define-easy-handler
(test-fn :uri "/test.html") (name)
(setf (hunchentoot:content-type*) "text/plain")
(hunchentoot:no-cache)
(format nil "Hey~#[ ~A~]!" name))
I'm unable to find a way to add no-cache to static dispatch and handlers, such as the following:
(push (hunchentoot:create-static-file-dispatcher-and-handler
"/url.html" "/path/to/www_/actual-page.html")
hunchentoot:*dispatch-table*)
The above example bypasses caching because I believe it is dynamically serving the page.
Below is my server configuration. Static pages are being cached currently. I would like to understand the Hunchentoot mechanism to disable cache for all resources but also disabling for specified resources (leaving it on for other resources).
(defvar *ssg-web-server* (hunchentoot:start
(make-instance 'hunchentoot:easy-acceptor
:address "127.0.0.1"
:port 4242
:document-root #p"/path/to/www_/"
:persistent-connections-p t
:read-timeout 3.0
:write-timeout 3.0
:access-log-destination nil
:message-log-destination nil)))
create-static-file-dispatcher-and-handler accepts a callback:
CALLBACK is run just before sending the file, and can be used
to set headers or check authorization;
arguments are the filename and the (guessed) content-type.
I suppose you can write:
(defun no-cache-static-callback (file content-type)
(declare (ignore file content-type))
(no-cache))
And you would pass this function as follows, namely with a NIL content-type and the callback function:
(create-static-file-dispatcher-and-handler
"/url.html"
"/path/to/www_/actual-page.html"
nil
#'no-cache-static-callback)
Related
I am struggling to get AT commands working that will get the 3G SIM5230 module to make a clean HTTP request. When I build and send what I believe is a good request, I do not get a response from the web server, only the error +CHTTPACT: 237. I can get the module to send an invalid post (simply add a space in front of the POST in the command), and I get a response back from a web server 400, telling me it is an invalid http request. Anyone out there successfully doing HTTP POST commands with this module?
Model: SIMCOM_SIM5320A
Revision: SIM5320A_V1.5
I also had trouble POST and GET 'ing with the SIM5320a module. While this code is for GET, similar code can be used for POST.
I found the tricky bit was what keystrokes to send after the AT+CHTTPSSEND=86 command, and how many characters to include in this (NOTE that the 86 is crucial - it is the number of characters plus 6 = for carriage return and line feeds). It took a while to work out that you need two carriage returns and line feeds at the end.
I used TERATERM for testing. I have now made some simple code for my arduino uno connected to a SIM5320a (this has been running for over a week now, and gets data every hour). If anyone wants this code, send me a request.
TERATERM + SIM5320a
//This program is for a SIM5320A connection using TeraTerm (I am running this through an Arduino Uno that has been uploaded with a AT port-through script). The items in brackets are the keystrokes I used to make the request run. The information after -> is the expected result.
//SETUP
AT+CGDCONT=1,"IP","INSERT_YOUR_APN","0.0.0.0" (return) -> OK
AT+CGSOCKCONT=1,"IP","INSERT_YOUR_APN" (return) -> OK
AT+CSOCKSETPN=1 (return) -> OK
//Start HTTPS session
AT+CHTTPSSTART (return) -> OK
//Open HTTPS session at server
AT+CHTTPSOPSE="www.XXXXXXXXXXXXXXXX.com.au",80,1 (return) -> OK
//Send request - NOTE that the 86 is crucial - it is the number of characters in what you want to send plus 6 (6 carriage return and line feeds)
AT+CHTTPSSEND=86 (return) -> >
GET /SensorE.php?temp=11111&EC=3333 HTTP/1.1
(Ctrl M)
(Ctrl J)
Host: www.XXXXXXXXXXXXXXXX.com.au:80
(Ctrl M)
(Ctrl J)
(Ctrl M)
(Ctrl J) -> OK +CHTTPS: RECV EVENT
//Receive request
AT+CHTTPSRECV=4000 (return) -> lots of stuff we hope, but should contain the returned data from your website
//Close session at server (but will automatically after it gets a nice recv)
AT+CHTTPSCLSE (return) -> OK or error - it is ok if error pops up here
//Stop the http service
AT+CHTTPSSTOP (return) -> OK
//Power down the module
AT+CPOF
Please verify your actual model because you said you have the SIMCOM5320A Model and asked about the SIM5230A.
If you have the SIMCOM5320A I think you are getting that error because that AT Command is already deprecated.
You can check the SIMCOM5320A AT Commands here: AT Commands
I need to get client's IP address in regular HttpHandler like this:
http = HttpHandler() do req::Request, res::Response
Response( ismatch(r"^/hello/",req.resource) ? string("Hello ", split(req.resource,'/')[3], "!") : 404 )
end
Neither req nor http.sock contain this information.
The Approach
This can be done, if you know the internals of Julia a little. It turns out that Julia uses the library libuv for low level system processing and that library has a function called uv_tcp_getpeername. This function is not exported by Julia.Base, but you can gain access to it via ccall. Also, the module HttpServer allows for way define a callback for various events, including the connect event.
Example Module
module HTTPUtil
export get_server
using HttpServer
function handle_connect(client)
try
buffer = Array(Uint8,32)
bufflen::Int64 = 32
ccall(:uv_tcp_getpeername,Int64,(Ptr{Void},Ptr{Uint8},Ptr{Int64}),client.sock.handle,buffer,&bufflen)
peername::IPv4 = IPv4(buffer[5:8]...)
task_local_storage(:ip,peername)
catch e
println("Error ... $e")
end
end
function get_server()
http = HttpHandler() do req::Request, res::Response
ip = task_local_storage(:ip)
println("Connection received from from $ip")
Response(ismatch(r"^/hello/",req.resource)?string("Hello ",split(req.resource,'/')[3], " $(ip)!") : 404 )
end
http.events["connect"]=(client)->handle_connect(client)
server = Server(http)
end
end
Explained
Each time a connection request is made, a peer socket is created by the server, and the connect handler is called which is defined to be handle_connect. It takes one parameter, client, of type Client. A Client type has a field called sock of type TcpSocket, and a TcpSocket has a field handle which is used by libuv. The object, then is each time a connection request is made, the connect handler is called, which calls uv_tcp_getpeername with the data contained in the TcpSocket handle. A byte array is declared to act as a buffer, which then is cast back to Base.IPv4. The module HTTPServer creates exactly 1 task for each client using #async, so the ip address can be stored local to the client using task_local_storage; thus there is no race condition.
Using it
julia> using HTTPUtil
julia> server = get_server()
Server(HttpHandler((anonymous function),TcpServer(init),Dict{ASCIIString,Function} with 3 entries:
"error" => (anonymous function)
"listen" => (anonymous function)
"connect" => (anonymous function)),nothing)
julia> #async run(server,8000)
Listening on 8000...
Task (queued) #0x000000000767e7a0
julia> Connection received from from 192.168.0.23
Connection received from from 192.168.0.22
... etc
Notes
For illustration, the output is modified so that the server will respond to each browser "Hello ipaddr"
This should be included in Base and/or HttpServer, but currently is not, so you'll need to use this workaround until it is.
The typical looping structure is used in get_server to illustrate there is no requirement for it to change, except to add in the ip address.
Assumes IPv4, but can be improved to allow both IPv4 and IPv6 straightforwardly as libuv supports both.
Thanks to waTeim's answer, but it's from 2014 and things changed in Julia. This works nicely in Julia 6.0 and probably all above:
function ip(socket::TCPSocket)
buffer = Array{UInt8}(32)
bufflen::Int64 = 32
ccall(:uv_tcp_getpeername,Int64,(Ptr{Void},Ptr{UInt8},Ptr{Int64}), socket.handle, buffer, &bufflen)
peername::IPv4 = IPv4(buffer[5:8]...)
end
Building on the excellent answer from waTeim I simplified things a little to work with IPv6, and also for SSL connections:
using MbedTLS
function handle_connect(client)
ip, port = getsockname(isa(client.sock, MbedTLS.SSLContext) ? client.sock.bio : client.sock)
task_local_storage(:ip, ip)
end
(Would have added this as a comment to Josh Bode's answer, but I don't have the necessary reputation.)
Note that it is necessary to use getpeername() instead of getsockname() as of Julia v0.7.
https://github.com/JuliaLang/julia/pull/21825
Depending on your situation, you may be able to pass a linux command.
userIP = strip(readstring(`hostname -i`), ['\n'])
I would like to write an HTTP server that answer to request using a non-standard HTTP method (verb). For instance, the client would make a request like FOO / HTTP/.1.1. And on the server side, this request would be handled by something like:
var express = require('express');
var app = express.createServer();
app.configure(function(){
app.use(express.logger({ format: ':method :url' }));
app.use(express.methodOverride());
});
app.foo('/', function(req, res){
res.send('Hello World');
});
app.listen(3000);
I appended my non-standard method to the array exported in ExpressJS's lib/router/methods.js. This allow me to write my server code as expected. When using express.methodOverride() and a POST request with _method=foo, it works. But an actual FOO request doesn't work. As soon as the client send the first line of the request the connection is closed by the server:
$telnet localhost 3000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
FOO / HTTP/1.1
Connection closed by foreign host.
I would like to be able to implement this with ExpressJS and without avoid hacking into its core file.
Any idea if this is possible and how?
Short answer: No, it's not possible. Not without implementing your own HTTP module.
To test, start a barebones HTTP server ...
$ node
> require('http').createServer(function(req, res) {
... console.log(req.method);
... res.end();
... }).listen(8080);
Then (as you've already done) telnet to it and issue a GET and FOO request ...
$ telnet localhost 8080
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1
HTTP/1.1 200 OK
Connection: keep-alive
Transfer-Encoding: chunked
0
FOO / HTTP/1.1
Connection closed by foreign host.
$
In node console you'll see
GET
... but no FOO. So, node's native HTTP module, which Express uses, does not make these requests available.
Node has a hard-coded whitelist of acceptable HTTP verbs in C.
In order to accept custom verbs, you must modify the HTTP parser and recompile node.
You mentioned that you're trying to implement PURGE, which was added to the whitelist in v0.7.5.
As others have said, Node.js' HTTP server library is configured to accept only specific verbs. Ben Noordius' suggestion of using Parsley doesn't work either, since that library accepts an even smaller whitelist of verbs. (It also hasn't been maintained in quite some time.)
At this stage, if we want to support oddball requests, we have to take more drastic measures. Here's a nice ugly hack for you that involves duck punching some internal behavior. This works on v0.10.x of Node.js, but test carefully on newer versions as they become available.
In my case, I needed to support not only a non-standard verb, but a non-standard protocol version identifier as well, and a missing Content-Length header for Icecast source streams:
SOURCE /live ICE/1.0
The following should get you started:
server.on('connection', function (socket) {
var originalOnDataFunction = socket.ondata;
var newLineOffset;
var receiveBuffer = new Buffer(0);
socket.ondata = function (d, start, end) {
receiveBuffer = Buffer.concat([receiveBuffer, d.slice(start, end)]);
if ((newLineOffset = receiveBuffer.toString('ascii').indexOf('\n')) > -1) {
var firstLineParts = receiveBuffer.slice(0, newLineOffset).toString().split(' ');
firstLineParts[0] = firstLineParts[0].replace(/^SOURCE$/ig, 'PUT');
firstLineParts[2] = firstLineParts[2].replace(/^ICE\//ig, 'HTTP/');
receiveBuffer = Buffer.concat([
new Buffer(
firstLineParts.join(' ') + '\r\n' +
'Content-Length: 9007199254740992\r\n'
),
receiveBuffer.slice(newLineOffset +1)
]);
socket.ondata = originalOnDataFunction;
socket.ondata.apply(this, [receiveBuffer, 0, receiveBuffer.length]);
}
};
}
It's ugly, but works. I'm not particularly happy about it, but when choosing between a rough built-from-the-ground-up HTTP parser or tweaking an existing one, I choose to tweak in this instance.
For anyone who needs it, there is http-parser-js, which replaces Node's built-in HTTP parser.
Their README contains an example of monkey-patching the parser, though I find that it wasn't enough, as both the http-parser-js and the http modules have a hardcoded list of methods.
So, you have to replace the parser and edit the list of methods:
const { HTTPParser } = require('http-parser-js');
HTTPParser.methods.push('FOOBAR');
const binding = process.binding('http_parser');
binding.HTTPParser = HTTPParser;
binding.methods = HTTPParser.methods;
require('http').METHODS = HTTPParser.methods;
Later Node versions may not support process.binding, in which case, you can use the --expose-internals flag for Node (see this issue):
const { internalBinding } = require('internal/test/binding');
const binding = internalBinding('http_parser');
From the looks of it, the http2 module's parser accepts any method, in case that's an option. See this issue about invalid HTTP methods. Unfortunately, express and the likes do not use http2.
And for anyone who was in my shoes, proxying requests to a legacy server in Create React App, use the above snippet in webpack-dev-server, at the top of Server.js, in order to monkey-patch the parser. Hopefully everything switches to http2 soon...
As my web application is currently a REST-only system (with no pages meant for user view) I'd like to send a 404 HTTP error when index page is requested. How to do this in Lift?
Does this work for you?
serve {
case XmlPut("order" :: orderId :: Nil, (_, _)) =>
Full(NotFoundResponse())
}
... or just Empty box. With an empty box later serve blocks may override.
I'm writing my first Sinatra-based web app as a frontend to another TCP-based service, using EventMachine and async_sinatra to process incoming HTTP requests asynchronously. When I'm testing my app, all requests to synchronous routes are logged to stdout in common log format, but asynchronous requests are not.
I've read through bits of the source code to async_sinatra, Sinatra, Thin, and Rack, and it looks like logging of synchronous requests is done through CommonLogger#call. However, I can't find anywhere in the asynchronous code in async_sinatra or Thin that seems to pass asynchronous requests through the logging middleware (I'm looking at Sinatra::Helpers#body in async_sinatra and at Thin::Connection.post_process which is written into env['.async_callback'] in Thin's connection.rb:68 and request.rb:132).
I'm experienced with C but relatively new to Ruby, so if I've used some terminology or notation incorrectly, please correct me. Thanks in advance.
Edit: this also affects error handling. If an exception is raised in an asynchronous request, the request is never finished and the error is never logged.
I eventually found that using rack-async with async_sinatra was causing problems with 404 pages, exception handling, and logging:
!! Unexpected error while processing request: undefined method `bytesize' for nil:NilClass
Instead I used the following wrapper around aroute for logging:
module Sinatra::Async
alias :oldaroute :aroute
def aroute verb, path, opts = {}, &block
# Based on aroute from async_sinatra
run_method = :"RunA#{verb} #{path} #{opts.hash}"
define_method run_method, &block
log_method = :"LogA#{verb} #{path} #{opts.hash}"
define_method(log_method) { |*a|
puts "#{request.ip} - #{status} #{verb} #{path}"
}
oldaroute verb, path, opts do |*a|
oldcb = request.env['async.callback']
request.env['async.callback'] = proc { |*args|
async_runner(log_method, *a)
oldcb[*args]
}
async_runner(run_method, *a)
end
end
end
This is for the same versions of async_sinatra, Thin, and Rack that I was using when I asked this question last year; newer versions may allow the use of common Rack middleware for logging.
I am running on sinatra-synchrony and therefore I have a slightly different core than you.
But basically I solved the same problem.
Here is an abstract of the solution:
I am not using Rack::CommonLogger, I use my own Logger
You need to buffer log output in an async aware storage
The buffered log output must be flushed at the end of the request
In my sinatra-synchrony application I am running the following middleware for logging:
# in app.rb I register Logger::Middleware as the first middleware
use Logger::Middleware
# in logger.rb
module Logger
attr_accessor :messages
def log(message)
stack << message
end
def stack
# This is the important async awareness
# It stores messages for each fiber separately
messages[Fiber.current.object_id] ||= []
end
def flush
STDERR.puts stack.join("\n") unless stack.empty?
messages.delete Fiber.current.object_id
end
extend self
class Middleware
def initialize(app)
#app = app
end
def call(env)
# before the request
Logger.log "#{env['REQUEST_METHOD']} #{env['REQUEST_URI']}"
result = #app.call(env)
# after the request
Logger.flush
result
end
end
end
Logger.messages = {} # initialize the message storage
Everywhere in the application I am able to use Logger.log("message") for logging.