I'm trying to secure a HLS stream using the secure_link module in nginx. I'm doing this in two parts:
First the URL is a "SEO" friendly version, this gets rewritten to the secure_link URL which has the ?token=xxx&?expires=000.
This gets processed in the next block, and here I ONLY want to test a part of the URL (stream name in this case) and not the whole URI. I'm using a regex to get the part of the URL I need to test in a variable. But for some reason every time I execute a request I get the wrong variable here (as outputted in the debug return line) if I use the "SEO" friendly URL that is rewritten, I get the $1 from the original url, and not the $1 of the "new* (re-written) URL.
If I submit the manually rewritten request I get the correct output for that variable. So this is only happening when I'm using the first rewritten URL.
Can someone point me as to what I going on here and how I can fix this? I'm out of ideas.
Demo URL:
https://myserver.com/live/hls/PSBOBrr5bVsd3wPLMMQE7Q/1658978846/AdminTest_1080/index.m3u8
Nginx config:
location /live {
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Expose-Headers' 'Content-Length' always;
rewrite /hls/([a-zA-Z0-9_\-]*)/([0-9]*)/([a-zA-Z0-9_\-]*)/(.*)\.(ts|key|m3u8)$ /hls/$3/$4.$5?token=$1&expires=$2;
root /mnt/not-exist;
}
location ~ "/hls/([a-zA-Z0-9_\-]*)/" {
# location ~ /hls/(?<alias>[a-zA-Z0-9_]+)/ { #TRIED BOTH REGEX HERE, BOTH SAME RESULT
#internal;
secure_link $arg_token,$arg_expires;
secure_link_md5 "$host $1 $secure_link_expires $remote_addr secret";
# if ($secure_link = "") { return 403; }
# if ($secure_link = "0") { return 410; }
add_header Content-Type text/plain;
return 200 "secure_link: $host $1 $secure_link_expires $remote_addr "; # FOR DEBUGGING
}
When submitting the above example to the first location block I get the following output:
secure_link: myserver.com PSBOBrr5bVsd3wPLMMQE7Q 127.0.0.1
Which is clearly incorrectly returning another part of the URL ($1 from the first location block, not the current location block)
When using a manually crafted URL:
https://myserver.com/hls/AdminTest_1080/index.m3u8?token=token&expires=0000
I get the following (correct output):
secure_link: myserver.com AdminTest_1080 127.0.0.1
At this point I'm even considering that this might be a bug? Hoping for valuable input from the community!
Related
I have a requirement to do a proxy call to url delivered via a query parameter as per example:
My nginx proxy is deployed at: https://myproxy.net
if the redirect parameter is not url encoded I can do the call with this block:
location /basepath {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
proxy_pass $arg_redirect;
proxy_intercept_errors on;
error_page 301 302 307 = #handle_redirects;
}
the error intercepts and #handle_redirects then take care of othe 30X codes that might pop up at new destination.
This works for a request:
GET: https://myproxy.net/basepath?redirect=https://destination.com/somepath/uuid
What do I need to do to make it work for:
GET: https://myproxy.net/basepath?redirect=https%3A%2F%2Fdestination.com%2Fsomepath%2Fuuid
Additionally as part of spec it has to be pure nginx, not additional modules, lua etc.
Thanks!
Actually, proxy_pass does normalisation by default, but it only affects $uri part. Thus you only need to decode the beginning of the passed string to get it working:
location / {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
if ( $arg_redirect ~ (.+)%3A%2F%2F(.+) ){ # fix :// between scheme and destination
set $arg_redirect $1://$2;
}
if ( $arg_redirect ~ (.+?)%3A(.*) ){ # fix : between destination and port
set $arg_redirect $1:$2;
}
if ( $arg_redirect ~ (.+?)%2F(.*) ){ # fix / after port, the rest will be decoded by proxy_pass
set $arg_redirect $1/$2;
}
proxy_pass $arg_redirect;
}
With the above I managed to access http://localhost/?redirect=http%3A%2F%2F127.0.0.1%3A81%2Fsfoo%20something%2Fs
The solution seems dirty and the only alternative using default modules is map (even less cleaner in my opinion). I'd rather split redirect argument into pieces: scheme (http or https), destination, port, and uri. With that you would be able to construct full address without rewriting:
proxy_pass $arg_scheme://$arg_dest:$arg_port/$arg_uri
Ok, there is very weird and curious solution
server {
listen 80;
resolver x.x.x.x;
location /basepath {
if ($arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
proxy_pass http://127.0.0.1:80/basepath/$arg_redirect;
}
location ~ ^/basepath/(?<proto>\w+):/(?<redir>.+)$ {
proxy_pass $proto://$redir;
}
}
Nginx does not encode path with variables in proxy_pass and send it as is. So, I make $arg_* part of proxy_pass uri, send request to self and nginx will receive new request which will be decoded.
But because Nginx will clean path and replace // to / I split protocol part in regexp.
And ... I would never recommend using this solution, but it works :)
try like this and let me know if it works
location /basepath {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
set_unescape_uri $decodedredirect $arg_redirect;
proxy_pass $decodedredirect;
proxy_intercept_errors on;
error_page 301 302 307 = #handle_redirects;
}
I need to redirect /referral/ID and /referral/ID/ urls (with or without trailing slash) to the root url, but store the ID value in a cookie. I'm trying to do it with the following location section, but this doesn't work.
location /referral {
if ($request_uri ~ "/referral/(.*)(?\/)(.*)") {
add_header Set-Cookie "referral-id=$1;Max-Age=100000";
}
rewrite ^ / permanent;
}
How can I achieve it with nginx?
One of the solutions:
http {
...
map $id $ref_id {
~. "referral=$id;Path=/;Max-Age=100000"; # any non-empty value
# otherwise result of map translation would be an empty string
}
server {
...
location ~ ^/referral(?:/(?<id>[^/]+))? {
add_header Set-Cookie $ref_id;
return 301 /;
}
}
}
If $id variable is empty or null, result of $id to $ref_id translation will be an empty string, and nginx would not set Set-Cookie (or any other HTTP header when an empty value passed as parameter value of add_header directive) at all.
I need nginx to reject requests if header StaticCookie is not present. I don't care about its value, I just need for it to exist.
What I came up with is this, but this doesn't work. Nginx allows requests with no headers at all.
if ($http_StaticCookie = false) {
return 403;
}
root /usr/share/asuno/www;
location ~* /css/ {
expires max;
}
location ~* /js/ {
expires max;
}
I saw this post -
Nginx: Reject request if header is not present or wrong - but it deals with defined header values. What I need is to check mere existence of the header.
I tried putting location directives inside the if clause but then nginx throws errors trying to read the config.
How can this be done?
the comment by Alexey Ten is correct, thanks.
if ($http_StaticCookie = "") { return 403; }
I have a nginx proxy to a API server. The API sometimes sets the cache control header. If the API hasnt set the cache control I want nginx to override it.
How do I do that?
I think I want to do something like this, but it doesnt work.
location /api {
if ($sent_http_cache_control !~* "max-age=90") {
add_header Cache-Control no-store;
add_header Cache-Control no-cache;
add_header Cache-Control private;
}
proxy_pass $apiPath;
}
You cannot use if here, because if, being a part of the rewrite module, is evaluated at a very early stage of the request processing, way before proxy_pass is called and the header is returned from the upstream server.
One way to solve your problem is to use map directive. Variables defined with map are evaluated only when they are used, which is exactly what you need here. Sketchily, your configuration in this case would look like this:
# When the $custom_cache_control variable is being addressed
# look up the value of the Cache-Control header held in
# the $upstream_http_cache_control variable
map $upstream_http_cache_control $custom_cache_control {
# Set the $custom_cache_control variable with the original
# response header from the upstream server if it consists
# of at least one character (. is a regular expression)
"~." $upstream_http_cache_control;
# Otherwise set it with this value
default "no-store, no-cache, private";
}
server {
...
location /api {
proxy_pass $apiPath;
# Prevent sending the original response header to the client
# in order to avoid unnecessary duplication
proxy_hide_header Cache-Control;
# Evaluate and send the right header
add_header Cache-Control $custom_cache_control;
}
...
}
Awswer from Ivan Tsirulev is correct but you don't have to use regex.
Nginx uses the first parameter of map as default value automatically so you don't have to add that either.
# Get value from Http-Cache-Control header but override it when it's empty
map $upstream_http_cache_control $custom_cache_control {
'' "no-store, no-cache, private";
}
server {
...
location /api {
# Use the value from map
add_header Cache-Control $custom_cache_control;
}
...
}
Original URL: /api/url%2Fencoded%2F/?with=queryParams
nginx:
location /api {
client_max_body_size 2G;
proxy_pass https://oursite;
}
With this configuration, I was able to preserve the URL encoding when passing through the proxy. If I add a "/" after "oursite", it will decode the URL.
Problem:
Now the URL after being proxied still contains "/api/". I need to remove "/api/" only while still preserving the URL encoded parts.
Not a long time ago there was identical question without an answer. In my opinion, you should rething api to not have such weird URLs. Another way is to have api on subdomain. – Alexey Ten Mar 11 '15 at 22:58
stackoverflow.com/q/28684300/1016033 – Alexey Ten Mar 11 '15 at 23:01
Year-old challenge accepted!
location /api/ {
rewrite ^ $request_uri;
rewrite ^/api/(.*) $1 break;
return 400;
proxy_pass http://127.0.0.1:82/$uri;
}
That's it, folks!
More details at Nginx pass_proxy subdirectory without url decoding, but it does work even with the query string, too:
% curl "localhost:81/api/url%2Fencoded%2F/?with=queryParams"
/url%2Fencoded%2F/?with=queryParams
%
Disclaimer: I am sure this looks like an hack - and maybe it is. It is using the auth-subrequest feature for something else than auth, but it works!
If you want to keep any url-encoded part after /api/ from the original $request_uri I use NJS to set a variable and use it afterwards in the proxy_pass
js_import /etc/nginx/conf.d/http.js; # Import your njs file here
js_set $encodedUrlPart 'empty'; # Define a variable
location ~* api\/(.*)$ {
auth_request /urlencode; #This will get executed before proxy_pass
proxy_pass http://127.0.0.1:82/$encodedUrlPart;
}
and the http.js can look like this
function urlencode(r){
let regex = "(?<=\/api\/)(.*$)";
let url = r.variables.request_uri # this holds the original, non touched url
let lastPart = url.match(regex);
r.variables.encodedUrlPart = lastPart;
r.log("The encoded url part: " + r.variables.encodedUrlPart);
r.return(200); // need to return 200 so the 'auth' doesn't fail
}
export default {urlencode};
Is this considered unsafe? We could do some checking in the njs part though!