nginx - read custom header from upstream server - nginx

I am using nginx as a reverse proxy and trying to read a custom header from the response of an upstream server (Apache) without success. The Apache response is the following:
HTTP/1.0 200 OK
Date: Fri, 14 Sep 2012 20:18:29 GMT
Server: Apache/2.2.17 (Ubuntu)
X-Powered-By: PHP/5.3.5-1ubuntu7.10
Connection: close
Content-Type: application/json; charset=UTF-8
My-custom-header: 1
I want to read the value from My-custom-header and use it in a if clause:
location / {
// ...
// get My-custom-header value here
// ...
}
Is this possible?

It's not only possible, it's easy:
in nginx the response header values are available through a variable (one per header).
See http://wiki.nginx.org/HttpCoreModule#.24sent_http_HEADER for the details on those variables.
In your examle the variable would be $sent_http_My_custom_header.

I was facing the same issue. I tried both $http_my_custom_header and $sent_http_my_custom_header but it did not work for me.
Although solved this issue by using $upstream_http_my_custom_header.

When using NGINX as a proxy, there are four sets of headers:
client -> nginx: the client request headers
nginx -> upstream: the upstream request headers
upstream -> nginx: the upstream response headers
nginx -> client: the client response headers
You appear to be asking about the upstream response headers. Those are found in the $upstream_http_name variables.
However, take into account that any response headers are only set after the headers from the upstream server response have been received. Any if directives are run before sending the upstream request, and will not have access to any response headers! In other words, if directives are run after the client request has been received, before making the upstream request.
If you need to change how a response is handled, you can use a map directive however to set variables based on response headers, then use those variables in add_header (set client response headers), log_format or any othere directives that are active during the response phases (internally named the NGX_HTTP_CONTENT_PHASE and NGX_HTTP_LOG_PHASE phases). For more complex control you'll have to use a scripting add-on such as the Lua module (e.g. using a header_filter_by_lua_block directive).
To read or set individual headers, use:
from
to
type
read (variable)
write (directive)
client
nginx
request
$http_name
–
ngnix
upstream
request
–
proxy_set_header
upstream
nginx
response
$upstream_http_name
–
nginx
client
response
$sent_http_name
add_header
NGINX copies certain headers from client request to upstream request, and from upstream response to client response using various proxy_ directives, giving you options to omit or explicitly include headers for either direction. So if an upstream response header is only found in $upstream_http_name variables, then those headers were specifically not copied to the client response, and the set of available $sent_http_name variables will include any extra headers set by NGINX that are not present in the upstream response.

Use $http_MY_CUSTOM_HEADER
You can write some-thing like
set my_header $http_MY_CUSTOM_HEADER;
if($my_header != 'some-value') {
#do some thing;
}

Related

Specific location path is not refreshing

We have a OpenResty, Nginx and Redis stack. I have a location block in my nginx.conf which doesn't reponse to any changes I do.
I have a Lua module function that is invoked by location in nginx.conf. The function is like:
function _M.myFunction()
ngx.header["Content-Type"] = "text/html"
ngx.say("debug")
end
I have a location like this - this location was set to another function that returned text/plain Content-Type with an empty string (ngx.say(" ") and I now pointed it to my debug function, it seems that it is "stuck" on previous response:
location /.faulty/foo/{
content_by_lua_block {
myLuaModule:myFunction()
}
}
responding with:
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive
with no body in response.
Testing by adding a different locations seems to working great for the same function and getting me the response I want:
location /.test/foo/{
content_by_lua_block {
myLuaModule:myFunction()
}
}
responding with:
HTTP/1.1 200 OK
Content-Type: text/html
Connection: keep-alive
debug
I checked Redis for any key that might hold this location, and couldn't find anything relevant.
so, for future generations -
there was a LB server in front of that server with the exact same function and location.
once changed there the change was refreshed.

When can you have relative URLs on the request line of GET http requests?

I tried the following request:
GET index.htm HTTP/1.1
connection: close
host: example.com
content-length: 0
But earned a 400 Bad Request. Why? The server should be able to piece together the absolute URL: http://example.com/index.htm . Why does it complain? Do I need a referer header to use relative URLs on the request line?
Short answer
You can't use relative path because HTTP specification (RFC7230) requires the use of absolute path.
Explanation
If you just refer to HTTP specification, it easy to find out why your request got a 400 Bad Request response: it violates the specification.
RFC7230 defines that in your scenario, the request target must use what is called the origin-form that requires absolute path:
origin-form = absolute-path [ "?" query ]
For instance, the HTTP request for http://example.org/where?q=now would be:
GET /where?q=now HTTP/1.1
Host: example.org
If the path is empty, such as http://example.org the HTTP request would be:
GET / HTTP/1.1
Host: example.org
This is because the absolute path is required as explained in Section 5.3.1 as follows (emphasis mine):
When making a request directly to an origin server, other than a
CONNECT or server-wide OPTIONS request (as detailed below), a client
MUST send only the absolute path and query components of the target
URI as the request-target.
I think on this line GET index.htm HTTP/1.1 is missing an 'l' on index.html.
Hope that helps!
I found the answer: If the URL on the request line isn't absolute it must be an absolute path. This means that you can omit the protocol and host name, but never any part of the path. The following worked:
GET /index.htm HTTP/1.1
connection: close
host: example.com
content-length: 0

Getting 404 error if requesting a page through proxy, but 200 if connecting directly

I am developing an HTTP proxy in Java. I resend all the data from client to server without touching it, but for some URLs (for example this) server returns the 404 error if I am connecting through my proxy.
The requested URL uses Varnish caching, so it might be the root of problem. I cannot reconfigure it - it is not my.
If I request that URL directly with browser, the server returns 200 and the image is shown correctly.
I am stuck because I even do not know what to read and how to compose a search request.
Thanks a lot.
Fix the Host: header of the re-issued request. The request going out from the proxy either has no Host header or it is broken (or only X-Host exists). Also take note that the proxy application will execute its own DNS lookup and that might yield a different IP address than your local computer (where you issued the original request).
This works:
> curl -s -D - -o /dev/null http://212.25.95.152/w/w-200/1902047-41.jpg -H "Host: msc.wcdn.co.il"
HTTP/1.1 200 OK
Content-Type: image/jpeg
Cache-Control: max-age = 315360000
magicmarker: 1
Content-Length: 27922
Accept-Ranges: bytes
Date: Sun, 05 Jul 2015 00:52:08 GMT
X-Varnish: 2508753650 2474246958
Age: 67952
Via: 1.1 varnish
Connection: keep-alive
X-Cache: HIT

Nginx return statement not accepting "text"

Following config is working for me:
server {
listen 80;
root /app/web;
index index.json;
location / {
return 409;
}
}
If I hit the website the 409 page will be presented. However following is not working:
server {
listen 80;
root /app/web;
index index.json;
location / {
return 409 "foobar";
}
}
The page is unreachable. But according to the docs http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return
return 409 "foobar";
should work. Any ideas whats wrong? There are no logs in nginx/error.log.
The thing is, Nginx does exactly what you ask it to do. You can verify this by calling curl -v http://localhost (or whatever hostname you use). The result will look somewhat like this:
* Rebuilt URL to: http://localhost/
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost
> Accept: */*
>
< HTTP/1.1 409 Conflict
* Server nginx/1.4.6 (Ubuntu) is not blacklisted
< Server: nginx/1.4.6 (Ubuntu)
< Date: Fri, 08 May 2015 19:43:12 GMT
< Content-Type: application/octet-stream
< Content-Length: 6
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
foobar
As you can see, Nginx returns both 409 and foobar, as you ordered.
So the real question here is why your browser shows the pretty formatted error page when there is no custom text after the return code, and the gray "unreachable" one, when such text is present.
And the answer is: because of the Content-Type header value.
The HTTP standard states that some response codes should or must come with the response body. To comply with the standard, Nginx does this: whenever you return a special response code without the required body, the web server sends its own hardcoded HTML response to the client. And a part of this response is the header Content-Type: text/html. This is why you see that pretty white error page, when you do return 409 without the text part — because of this header your browser knows that the returned data is HTML and it renders it as HTML.
On the other hand, when you do specify the text part, there is no need for Nginx to send its own version of the body. So it just sends back to the client your text, the response code and the value of Content-Type that matches the requested file (see /etc/nginx/mime.types).
When there is no file, like when you request a folder or a site root, the default MIME type is used instead. And this MIME type is application/octet-stream, which defines some binary data. Since most browsers have no idea how to render random binary data, they do the best they can, that is, they show their own hardcoded error pages.
And this is why you get what you get.
Now if you want to make your browser to show your foobar, you need to send a suitable Content-Type. Something like text/plain or text/html. Usually, this can be done with add_header, but not in your case, for this directive works only with a limited list of response codes (200, 201, 204, 206, 301, 302, 303, 304, or 307).
The only other option I see is to rewrite your original request to something familiar to Nginx, so that it could use a value from /etc/nginx/mime.types for Content-Type:
server {
listen 80;
root /app/web;
index index.json;
location / {
rewrite ^.*$ /index.html;
return 409 "foobar";
}
}
This might seem somewhat counter-intuitive but this will work.
EDIT:
It appears that the Content-Type can be set with the default_type directive. So you can (and should) use default_type text/plain; instead of the rewrite line.
Updating #ivan-tsirulev 's answer:
By now you can set headers even for page with status codes for errors using always.
location #custom_error_page {
return 409 "foobar";
add_header Content-Type text/plain always;
}
But if you set default_type, the response headers will have two Content-Type headers: default, then added. Nevertheless, it works fine.

Location header wrong after HTTP POST?

please forgive me if I do something terribly wrong, I shall correct it at once).
I'm creating a RESTservice and I need the location of the created resources, both for testing purposes and for further development.
As far is I have understood from the rfc 2616, the location header that I get back after a HTTP POST should give me at link to the resource I have created.
14.30 Location
The Location response-header field is used to redirect the recipient
to a location other than the Request-URI for completion of the
request or identification of a new resource. For 201 (Created)
responses, the Location is that of the new resource which was created
by the request. For 3xx responses, the location SHOULD indicate the
server's preferred URI for automatic redirection to the resource. The
field value consists of a single absolute URI.
Location = "Location" ":" absoluteURI
An example is:
Location: http://www.w3.org/pub/WWW/People.html
link to rfc 2616
However, when I POST to create a new resource (for example with curl, but I tried different clients) as such:
curl -X POST -H "Content-Type: application/xml" -d "<?xml version=\"1.0\" encoding=\"UTF-8\"?><participant><first-name>Severinus</first-name><email>severinus#gmail.com</email></participant>" -v http://localhost:8080/VCTAserverFirstTest/services/participants
This is the header I get back:
upload completely sent off: 133 out of 133 bytes
HTTP/1.1 201 Created
Server Apache-Coyote/1.1 is not blacklisted
Server: Apache-Coyote/1.1
Location: http://localhost:8080/participants/1
Content-Length: 0
Date: Sat, 19 Apr 2014 16:07:31 GMT
Connection #0 to host localhost left intact
Here the Location is http://localhost:8080/participants/1 but the resources is in fact accessed on http://localhost:8080/VCTAserverFirstTest/services/participants/1
This is not what I expected. Do you need the server side code or is there an obvious answer to this?
Thanks
Lars
EDIT: Not a good answer for anybody else, but it was solved by using the resteasy framework.

Resources