nginx module: capture whole response body - nginx

Question

Although Nginx is a really interesting piece of software, the lack of documentation is making me crazy.
Goal: capture the whole response body, which would be logged on the server.
Problem: I have always a single buffer which size is ZERO.
Approach
I would expect to be able to accomplish this requirement with a body filter, which would "wait" for last_buf before iterate the full buffers chain.
/**
* #param ngx_http_request_t *r HTTP request
* #param ngx_chain_t *in Buffer chain
*/
static ngx_int_t
create_response_snapshot(ngx_http_request_t *r, ngx_chain_t *in)
{
ngx_chain_t *chain = NULL;
int chain_contains_last_buffer = 0;
size_t buffer_size = 0;
// check if body is complete
chain = in;
for ( ; ; )
{
if (chain->buf->last_buf)
{
chain_contains_last_buffer = 1;
}
if (NULL == chain->next)
break;
chain = chain->next;
}
if (0 == chain_contains_last_buffer)
{
// response is not complete
return ngx_http_next_body_filter(r, in);
}
// Response Content-Length
ngx_log_error(NGX_LOG_ALERT,r->connection->log,0,"Content-Length: %d",
r->headers_out.content_length_n);
// lets iterate buffers chain
for (chain = in; NULL != chain; chain = chain->next)
{
buffer_size = ngx_buf_size(chain->buf);
ngx_log_error(NGX_LOG_ALERT,r->connection->log,0,"buffer_size#%d",buffer_size);
}
return ngx_http_next_body_filter(r, in);
}

Answer 1

My comment got too big to be a comment, but I don't feel like it's a proper answer - oh well.
To re-iterate, the problem with the code you've posted is that your module's body filter function won't be called on the whole chain at once. It gets called on the first piece, then the second piece, until the nth piece. Finally it gets called on a completely empty chain, for whatever reason the buf with last_buf = 1 is always by itself and empty.
So I think what you want to do is "dam" the flow of buffers by accumulating them in your module without releasing any to the next filter until you have all of them at once.
Check out the substitution filter module: http://lxr.nginx.org/source//src/http/modules/ngx_http_sub_filter_module.c
It uses a "busy" chain which is what I was referring to. From what I've been able to tell it uses it to keep track of which buffers have actually been sent (when this happens the size gets set to zero) and adds those to the module context's free list for re-use. See ngx_http_sub_output on line 438 for this behavior.
My suggestion was to do something like what that module does, except without calling the next filter until you have the entire page. You can't call next_filter if you want to process the entire page as a whole, since doing that will result in data getting sent to the client. Again this runs counter to Nginx's design, so I think you should find an alternative that doesn't require the whole response body at once if you can.