It seems like once gzip is turned on, it will only gzip even if the request doesnt set Accept-Encoding: gzip in the header.
Is there anyway so that if the request doesnt set Accept-Encoding, it will not return gzip?
Thanks!
Try viewing the headers with curl -I http://example.com/ (where example.com is replaced with your site). Verify that Content-Encoding: gzip isn't in the resulting headers.
Often other test clients add headers (including Accept-Encoding) automatically, which can make testing problematic.
Side note: If you are interested in verifying that the server does return gzip encoding when requested, you can use the alternate test query 'curl -I --compressed http://example.com/`.
Related
I'm trying to stream mp4 file in loop to my nginx server. And i need to remove old segments:
ffmpeg -re -stream-loop -1 -i /data/samples/BigBuckBunny.mp4 -c copy -f hls -hls_time 5 -hls_flags delete_segments -hls_list_size 5 http://127.0.0.1:8080/upload/stream.m3u8
Everythink is ok, but when ffmpeg tries to remove old segment I've got this error in nginx:
[error] 22#22: *73174 DELETE with body is unsupported, client:
127.0.0.1, server: _, request: "DELETE /upload/stream16.ts HTTP/1.1", host: "127.0.0.1:8080"
My nginx config:
location /upload {
root /data/live;
dav_access user:rw group:rw all:rw;
dav_methods PUT DELETE MKCOL COPY MOVE;
create_full_put_path on;
charset utf-8;
autoindex on; }
ffmpeg 4.4.1
nginx 1.21.4
What I'm doing wrong?
It seems that the ffmpeg http muxer that underlies the hls muxer defaults to chunked transfer encoding. When the DELETE request is made there is no body, but ffmpeg still makes the request with a single zero-length chunk.
The error message in nginx could be slightly more helpful. Indeed, it does not support webdav DELETE requests with a body, but it also does not support DELETE requests which are marked as chunked transfer encoded, regardless of whether there is a body (see: https://github.com/nginx/nginx/blob/master/src/http/modules/ngx_http_dav_module.c#L315), hence the error.
It looks like it should be possible to disabled this behaviour in ffmpeg using the chunked_post option, but it doesn't appear to be working. Not sure if this is a bug or not, or it seems like a bit of a hack anyway.
I want implement this answer in Kemal.
My current setup has a pdf file in app/public/map.pdf, and the following code in my main crystal file:
require "kemal"
#...
get "/map.pdf" do |env|
env.response.headers["Content-Type"] = "application/pdf"
env.response.headers["Content-Disposition"] = %(inline;filename="myfile.pdf")
end
Kemal.run
When I test my code by opening localhost:3000/map.pdf in a browser (firefox), it prompts to download the file (while I want it to attempt to display it). And curl -I results in the following:
HTTP/1.1 200 OK
Connection: keep-alive
X-Powered-By: Kemal
Content-Type: application/octet-stream
ETag: W/"1494983019"
Content-Length: 1170498
Where I would hope to see Content-Type: application/pdf.
Kemal author here,
If the headers are ok you should be good to go with send_file. However be sure that the route name and file are not the same. In this case the route is /pdf
get "/pdf" do |env|
env.response.headers["Content-Type"] = "application/pdf"
env.response.headers["Content-Disposition"] = %(inline;filename="map.pdf")
send_file env, "./public/map.pdf"
end
Where I would hope to see Content-Type: application/pdf.
The point is that Content-Disposition doesn't exist as a header.
Maybe you don't send any headers at all for some reason.
For example in PHP, if your script outputs a single byte before it sends the headers then no headers will be sent; only the headers before the first output.
If you set a http header after some data is sent then the header will be ignored.
Following config is working for me:
server {
listen 80;
root /app/web;
index index.json;
location / {
return 409;
}
}
If I hit the website the 409 page will be presented. However following is not working:
server {
listen 80;
root /app/web;
index index.json;
location / {
return 409 "foobar";
}
}
The page is unreachable. But according to the docs http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return
return 409 "foobar";
should work. Any ideas whats wrong? There are no logs in nginx/error.log.
The thing is, Nginx does exactly what you ask it to do. You can verify this by calling curl -v http://localhost (or whatever hostname you use). The result will look somewhat like this:
* Rebuilt URL to: http://localhost/
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost
> Accept: */*
>
< HTTP/1.1 409 Conflict
* Server nginx/1.4.6 (Ubuntu) is not blacklisted
< Server: nginx/1.4.6 (Ubuntu)
< Date: Fri, 08 May 2015 19:43:12 GMT
< Content-Type: application/octet-stream
< Content-Length: 6
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
foobar
As you can see, Nginx returns both 409 and foobar, as you ordered.
So the real question here is why your browser shows the pretty formatted error page when there is no custom text after the return code, and the gray "unreachable" one, when such text is present.
And the answer is: because of the Content-Type header value.
The HTTP standard states that some response codes should or must come with the response body. To comply with the standard, Nginx does this: whenever you return a special response code without the required body, the web server sends its own hardcoded HTML response to the client. And a part of this response is the header Content-Type: text/html. This is why you see that pretty white error page, when you do return 409 without the text part — because of this header your browser knows that the returned data is HTML and it renders it as HTML.
On the other hand, when you do specify the text part, there is no need for Nginx to send its own version of the body. So it just sends back to the client your text, the response code and the value of Content-Type that matches the requested file (see /etc/nginx/mime.types).
When there is no file, like when you request a folder or a site root, the default MIME type is used instead. And this MIME type is application/octet-stream, which defines some binary data. Since most browsers have no idea how to render random binary data, they do the best they can, that is, they show their own hardcoded error pages.
And this is why you get what you get.
Now if you want to make your browser to show your foobar, you need to send a suitable Content-Type. Something like text/plain or text/html. Usually, this can be done with add_header, but not in your case, for this directive works only with a limited list of response codes (200, 201, 204, 206, 301, 302, 303, 304, or 307).
The only other option I see is to rewrite your original request to something familiar to Nginx, so that it could use a value from /etc/nginx/mime.types for Content-Type:
server {
listen 80;
root /app/web;
index index.json;
location / {
rewrite ^.*$ /index.html;
return 409 "foobar";
}
}
This might seem somewhat counter-intuitive but this will work.
EDIT:
It appears that the Content-Type can be set with the default_type directive. So you can (and should) use default_type text/plain; instead of the rewrite line.
Updating #ivan-tsirulev 's answer:
By now you can set headers even for page with status codes for errors using always.
location #custom_error_page {
return 409 "foobar";
add_header Content-Type text/plain always;
}
But if you set default_type, the response headers will have two Content-Type headers: default, then added. Nevertheless, it works fine.
I am using nginx as a reverse proxy and trying to read a custom header from the response of an upstream server (Apache) without success. The Apache response is the following:
HTTP/1.0 200 OK
Date: Fri, 14 Sep 2012 20:18:29 GMT
Server: Apache/2.2.17 (Ubuntu)
X-Powered-By: PHP/5.3.5-1ubuntu7.10
Connection: close
Content-Type: application/json; charset=UTF-8
My-custom-header: 1
I want to read the value from My-custom-header and use it in a if clause:
location / {
// ...
// get My-custom-header value here
// ...
}
Is this possible?
It's not only possible, it's easy:
in nginx the response header values are available through a variable (one per header).
See http://wiki.nginx.org/HttpCoreModule#.24sent_http_HEADER for the details on those variables.
In your examle the variable would be $sent_http_My_custom_header.
I was facing the same issue. I tried both $http_my_custom_header and $sent_http_my_custom_header but it did not work for me.
Although solved this issue by using $upstream_http_my_custom_header.
When using NGINX as a proxy, there are four sets of headers:
client -> nginx: the client request headers
nginx -> upstream: the upstream request headers
upstream -> nginx: the upstream response headers
nginx -> client: the client response headers
You appear to be asking about the upstream response headers. Those are found in the $upstream_http_name variables.
However, take into account that any response headers are only set after the headers from the upstream server response have been received. Any if directives are run before sending the upstream request, and will not have access to any response headers! In other words, if directives are run after the client request has been received, before making the upstream request.
If you need to change how a response is handled, you can use a map directive however to set variables based on response headers, then use those variables in add_header (set client response headers), log_format or any othere directives that are active during the response phases (internally named the NGX_HTTP_CONTENT_PHASE and NGX_HTTP_LOG_PHASE phases). For more complex control you'll have to use a scripting add-on such as the Lua module (e.g. using a header_filter_by_lua_block directive).
To read or set individual headers, use:
from
to
type
read (variable)
write (directive)
client
nginx
request
$http_name
–
ngnix
upstream
request
–
proxy_set_header
upstream
nginx
response
$upstream_http_name
–
nginx
client
response
$sent_http_name
add_header
NGINX copies certain headers from client request to upstream request, and from upstream response to client response using various proxy_ directives, giving you options to omit or explicitly include headers for either direction. So if an upstream response header is only found in $upstream_http_name variables, then those headers were specifically not copied to the client response, and the set of available $sent_http_name variables will include any extra headers set by NGINX that are not present in the upstream response.
Use $http_MY_CUSTOM_HEADER
You can write some-thing like
set my_header $http_MY_CUSTOM_HEADER;
if($my_header != 'some-value') {
#do some thing;
}
Given this snippet:
(defroutes main-routes
(POST "/input/:controller" request
(let [buff (ByteArrayOutputStream.)]
(copy (request :body) buff)
;; --- snip
The value of buff will be a non-empty byte array iff there's the Content-Type header in the request. The value can be nonsencial, the header just has to be there.
However, I need to dump the body (hm... that came out wrong) if the request came without a content type, so that the client can track down the offending upload. (The uploading software is not under my control and its maintainers won't provide anything extra in the headers.)
Thank you for any ideas on how to solve or work around this!
EDIT:
Here are the headers I get from the client:
{
"content-length" "159",
"accept" "*/*",
"host" (snip),
"user-agent" (snip)
}
Plus, I discovered that Ring, using an instance of Java's ServletRequest, fills in the content type with the standard default, x-www-form-urlencoded. I'm now guessing that HTTPParser, which supplies the body through HTTPParser#Input, can't parse it correctly.
I face the same issue. It's definitely one of the middleware not being able to parse the body correctly and transforming :body. The main issue is that the Content-Type suggest the body should be parsable.
Using ngrep, I found out how curl confuses the middleware. The following, while intuitive (or rather sexy) on the command line sends a wrong Content-Type which confuses the middleware:
curl -nd "Unknown error" http://localhost:3000/event/error
T 127.0.0.1:44440 -> 127.0.0.1:3000 [AP]
POST /event/error HTTP/1.1.
Authorization: Basic SzM5Mjg6ODc2NXJkZmdoam5idmNkOQ==.
User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3.
Host: localhost:3000.
Accept: */*.
Content-Length: 13.
Content-Type: application/x-www-form-urlencoded.
.
Unknown error
The following however forces the Content-Type to being opaque and the middleware will not interfere with the :body.
curl -nd "Unknown error" -H "Content-Type: application/data" http://localhost:3000/event/error
T 127.0.0.1:44441 -> 127.0.0.1:3000 [AP]
POST /event/error HTTP/1.1.
Authorization: Basic SzM5Mjg6ODc2NXJkZmdoam5idmNkOQ==.
User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3.
Host: localhost:3000.
Accept: */*.
Content-Type: application/data.
Content-Length: 13.
.
Unknown error
I'm considering replacing the middleware with a more liberal one because even though the request is wrong, I'd still like to be able to decide what to do with the body myself. It's a really weird choice to zero the request body when the request doesn't make sense. I actually think a more correct behavior would be to pass it to an error handler which by default would return a 400 Bad Request or 406 Not Acceptable.
Any thoughts on that? In my case I might propose a patch to Compojure.
According to:
http://mmcgrana.github.com/ring/ring.middleware.content-type-api.html
the default content type is application/octet-stream. Unless you actively support that content type, can't you just check if the content type matches that one, and then dump whatever you need based on that?