How do I log asynchronous thin+sinatra+rack requests? - asynchronous

Question

I'm writing my first Sinatra-based web app as a frontend to another TCP-based service, using EventMachine and async_sinatra to process incoming HTTP requests asynchronously. When I'm testing my app, all requests to synchronous routes are logged to stdout in common log format, but asynchronous requests are not.
I've read through bits of the source code to async_sinatra, Sinatra, Thin, and Rack, and it looks like logging of synchronous requests is done through CommonLogger#call. However, I can't find anywhere in the asynchronous code in async_sinatra or Thin that seems to pass asynchronous requests through the logging middleware (I'm looking at Sinatra::Helpers#body in async_sinatra and at Thin::Connection.post_process which is written into env['.async_callback'] in Thin's connection.rb:68 and request.rb:132).
I'm experienced with C but relatively new to Ruby, so if I've used some terminology or notation incorrectly, please correct me. Thanks in advance.
Edit: this also affects error handling. If an exception is raised in an asynchronous request, the request is never finished and the error is never logged.

Answer 1

I eventually found that using rack-async with async_sinatra was causing problems with 404 pages, exception handling, and logging:
!! Unexpected error while processing request: undefined method `bytesize' for nil:NilClass
Instead I used the following wrapper around aroute for logging:
module Sinatra::Async
alias :oldaroute :aroute
def aroute verb, path, opts = {}, &block
# Based on aroute from async_sinatra
run_method = :"RunA#{verb} #{path} #{opts.hash}"
define_method run_method, &block
log_method = :"LogA#{verb} #{path} #{opts.hash}"
define_method(log_method) { |*a|
puts "#{request.ip} - #{status} #{verb} #{path}"
}
oldaroute verb, path, opts do |*a|
oldcb = request.env['async.callback']
request.env['async.callback'] = proc { |*args|
async_runner(log_method, *a)
oldcb[*args]
}
async_runner(run_method, *a)
end
end
end
This is for the same versions of async_sinatra, Thin, and Rack that I was using when I asked this question last year; newer versions may allow the use of common Rack middleware for logging.

Answer 2

I am running on sinatra-synchrony and therefore I have a slightly different core than you.
But basically I solved the same problem.
Here is an abstract of the solution:
I am not using Rack::CommonLogger, I use my own Logger
You need to buffer log output in an async aware storage
The buffered log output must be flushed at the end of the request
In my sinatra-synchrony application I am running the following middleware for logging:
# in app.rb I register Logger::Middleware as the first middleware
use Logger::Middleware
# in logger.rb
module Logger
attr_accessor :messages
def log(message)
stack << message
end
def stack
# This is the important async awareness
# It stores messages for each fiber separately
messages[Fiber.current.object_id] ||= []
end
def flush
STDERR.puts stack.join("\n") unless stack.empty?
messages.delete Fiber.current.object_id
end
extend self
class Middleware
def initialize(app)
#app = app
end
def call(env)
# before the request
Logger.log "#{env['REQUEST_METHOD']} #{env['REQUEST_URI']}"
result = #app.call(env)
# after the request
Logger.flush
result
end
end
end
Logger.messages = {} # initialize the message storage
Everywhere in the application I am able to use Logger.log("message") for logging.