Nginx is serving only static files, yet, some of file names contains '?'. Yes, the question mark.
All URLs that contains '?' yield 404 even though file actually exists. e.g.
> GET /foo?lang=ar.html HTTP/1.1
...
...
< HTTP/1.1 404 Not Found
While a file named foo?lang=ar.html does exists in the expected location.
> GET /foo%3flang=ar.html HTTP/1.1
...
...
< HTTP/1.1 200 OK
How do I write a rewrite directive so all '?' will be redirected to %3f?
You should url-encode your query string to escape special characters such as ? and =
Specifically, the name of your file you have to request for, once encoded, is this:
foo%3Flang%3Dar.html
In Javascript you can url-encode the filename with encodeURIComponent() function, in PHP you have urlencode().
You MUST encode the ? as %3F before the http call to nginx.
The reason is that the url rfc reserves the ? character as a special character (specifcally see section 3.3). Consequently nginx will, correctly, interpret an unescaped ? character as the end of the path part of the url
Related
I want to redirect from nginx to a URL that is send as an argument in the encoded form.
location /v1/redirect {
access_log /var/log/nginx/redirect/access.log main;
error_log /var/log/nginx/redirect/error.log;
return 302 $arg_dest;
}
When I am passing unencoded URL for e.g. https://ex2.example2.com/v1/redirect?dest=https://ex1.example2.com It is working fine.
But on encoding dest argument as https://ex2.example2.com/v1/redirect?dest=https%3A%2F%2Fex1.example2.com. I am getting status code 304 and my URL is changing to https://ex2.example2.com/v1/https%3A%2F%2Fex1.example2.com.
Can someone help me identify what I am doing wrong, Is there a way to possibly URL decode arg_dest before passing to return? Thanks
njs can be used to solve the above problem. Take a look at https://github.com/xeioex/njs-examples/blob/master/conf/http/decode_uri.conf and https://github.com/xeioex/njs-examples/blob/master/njs/http/decode_uri.js to solve the above problem.
I am new to nginx environment. May i know how to deal with question mark , arguments and values from the URL in nginx.
Below are the requests that nginx config file receives.
http://example.com/api/student?id=123,
http://example.com/api/student?name=yuva1,
http://example.com/api/student?place=KL, http://example.com/api/student?marks=56 .
The values after "=" (equal to) symbol can be anything. id may get 456 and name may receive jam5 etc.,
Let me know how to deal with this in nginx config file. Appreciate your help.
Thanks.
Nginx embedded variables will provide the value for the parameter.
For example,
http://example.com/api/student?id=123
$arg_id = 123
$arg_name
argument name in the request line
Source: http://nginx.org/en/docs/http/ngx_http_core_module.html#variables
I want to create a location regex match in nginx that allows routes starting with /subscriptions[optional shash /][...anything...] as unless additional slashes are found later in the path.
/subscriptions/000000 OK
/subscriptions OK
/subscriptions/ OK
/subscriptions/dasddsad/ NOT OK
/subscriptions/ NOT OK
the reason is that I have other regex matches that also matches those cases where additional slashes are present, so when it hits any of the NOT OK above, it shuold just continue to other location checks.
I've found the good article about using of nginx custom log format for logstash. In one on topic comment there is:
Be careful: Between two (referrer, user agent) and four (request method, remote_user) can be set by the end-user and thus can cause the JSON to be invalid (i.e. set the user-agent to "}).
As long as nginx doesn't have explicit JSON support, I would recommend against manually >building JSON and stick to combined log which is well-supported by logstack.
How should I use custom log format to prevent this issue?
Create grok with content
NGUSERNAME [a-zA-Z\.\#\-\+_%]+
NGUSER %{NGUSERNAME}
NGINXACCESS %{IPORHOST:clientip} %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} (?:%{NUMBER:bytes}|-) (?:"(?:%{URI:referrer}|-)"|%{QS:referrer}) %{QS:agent}
And filter
filter {
grok {
patterns_dir => ["Grok_DIR"]
match => { "message" => "%{NGINXACCESS}" }
}
}
It is not bulletproof in any version of nginx to hand craft the JSON - \xXX is not valid JSON encoding, it should be \u00XX. Any proper JSON parser will reject the JSON string if nginx escapes with \xXX.
If you do this with LogStash you will mostly be OK but expect json parse failures from time to time.
I think it's fine in recent versions of nginx:
Changes with nginx 1.1.6 17 Oct 2011
*) Change: now the 0x7F-0x1F characters are escaped as \xXX in an
access_log.
Changes with nginx 0.7.0 19 May 2008
*) Change: now the 0x00-0x1F, '"' and '\' characters are escaped as \xXX
in an access_log.
Thanks to Maxim Dounin.
Whenever a URL that has %2F which is the hex code for / is posted to my JBOSS Server, I get an error:
HTTP 400 Bad Request error message.
Here is the URL:
http://localhost:8080/application/**abc%2Fhi**?msg=hello"
If I remove the %2F from the URL the link works fine.
This %2F has to be part of the URL and cannot be a request parameter.
Finally figured out the cause of this (both for JBoss and Apache). Both applications intentionally reject URIs with an encoded slash (%2F for / and %5C for \) to prevent possible security vulnerabilities.
Links:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-0450
http://securitytracker.com/id/1018110 (Look at section 4. Solution)
And here are the instructions they provide for enabling this behavior in JBoss:
Note: In response to CVE-2007-0450, JBoss AS considers encoded slashes and backslashes in URLs invalid and its usage will result in HTTP 400 error. It is possible to allow encoded slashes and backslashes by following the steps outlined below, however doing so will expose you to CVE-2007-0450 related attacks:
a) If you use the /var/lib/jbossas/bin/run.sh setup, please edit /etc/jbossas/run.conf and append
- -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
- -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true to the string assigned to JAVA_OPTS
b) If you use the init script setup to run multiple JBoss AS services and you wish to allow encoding by default on all services, please edit /etc/jbossas/jbossas.conf and add the line JAVA_OPTS="${JAVA_OPTS}
- -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
- -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true"
c) If you use the init script setup to run multiple JBoss AS services and want to allow encoding of slashes and backslashes for a particular service, please edit /etc/sysconfig/${NAME} (where NAME is the name of your service) and add the line JAVA_OPTS="${JAVA_OPTS}
- -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
- -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true"
For Apache, it's as simple as setting "AllowEncodedSlashes NoDecode" somewhere in your apache conf or vhost conf (doesn't work in an .htaccess, however).
Apache link: http://httpd.apache.org/docs/current/mod/core.html#allowencodedslashes