I am trying to take some theoretical study about HTTP into practice. So I tried to make a HEAD (also tried GET but prefer HEAD since I am interested in the actual object) and it went as follows:
~$ telnet youtube.com 80
Trying 216.58.211.110...
Connected to youtube.com.
Escape character is '^]'.
HEAD /watch?v=GJvGf_ifiKw HTTP/1.1
Host: youtube.com
HTTP/1.1 301 Moved Permanently
Content-Length: 0
Location: https://youtube.com/watch?v=GJvGf_ifiKw
Date: Thu, 12 Dec 2019 15:48:41 GMT
Content-Type: text/html
Server: YouTube Frontend Proxy
X-XSS-Protection: 0
As you can see, I am requesting the object locating at /watch?v=GJvGf_ifiKw on the host located at youtube.com and this must sum to youtube.com/watch?v=GJvGf_ifiKw which is the URL of the location header field. What's going on here? Why does it say it has moved to the identical location?
If you looked closely to the output you will find that you've been redirected to HTTPS as your initial request was telnet on port 80 which is the default HTTP port
and since they are enforcing redirection to HTTPS
so the it's redirected to identical location BUT over Secured HTTP with is HTTPS
Related
I'm running a reverse proxy using proxy_pass directive from ngx_http_proxy_module. I want to forbid access to certain backend IP address ranges (like 172.0.0.0/24). I've tried
if ($upstream_addr ~* "^172.*") {
return 403;
}
add_header X-mine "$upstream_addr";
both in server and location context but it doesn't work, i.e. Nginx still returns 200:
$ curl localhost -I
HTTP/1.1 200 OK
Server: nginx/1.17.0
Date: Thu, 13 Feb 2020 12:58:36 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Tue, 24 Sep 2019 14:49:10 GMT
ETag: "5d8a2ce6-264"
Accept-Ranges: bytes
X-mine: 172.20.0.2:80
What am I missing? (Note that I added the content of $upstream_addr variable to X-mine header for debugging.)
My understanding is that the if directive is run before the upstream request is sent, while the $upstream_addr header is only set after the upstream request has completed. I have tried and failed to find definitive documentation that explains the precise process, but the nginx documentation seems to be missing a number of things that one might wish for.
See this answer, and also If is evil for a little more guidance. I'm not actually sure quite what you're trying to achieve so I can't offer any hope about whether or not it's possible.
I have a web application running on centos ec2 instance behind Nginx reverse proxy with SSL certification (Let's Encrypt).
I have a javascript file located at the URL for example https://example.com/static/src/js/allEnd.js
I used CloudFront to delivered the static file with the origin server as the HTTP ec2 instance (not using the s3 bucket.)
My origin server is mapped with the domain name https://example.com I have the following configuration I have made so far:
1. www.example.com is redirected to example.com in Nginx
2. The CloudFront URL is an alias with my custom domain ie cdn.example.com
3. SSL for example.com is done on Nginx whereas SSL for cdn,example.com is done on AWS.
What I have understood so far is the first time the CloudFront will serve the static content by getting the file from my ec2 server and then the next time it will serve from CloudFront but every time the CloudFront redirect to the origin server to get the static file which CloudFront is not serving in my case.
Here is the header for both the origin server and CloudFront server.
1. Origin server (https://example.com)
get https://example.com/static/src/js/allEnd.js
HTTP/2 200
server: nginx/1.12.2
date: Sun, 12 May 2019 12:27:50 GMT
content-type: application/javascript
content-length: 168435
etag: "wzsdm-1557567525-168435-283837276"
cache-control: max-age=604800, public
expires: Sun, 19 May 2019 12:27:50 GMT
strict-transport-security: max-age=15768000; includeSubdomains; preload
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
strict-transport-security: max-age=15768000
2. CloudFront with origin as https://example.com (https://cdn.example.com)
get https://cdn.example.com/static/src/js/allEnd.js
HTTP/2 301
content-type: text/html
content-length: 185
location: https://example.com/static/src/js/allEnd.js
server: nginx/1.12.2
date: Sun, 12 May 2019 09:17:40 GMT
strict-transport-security: max-age=15768000; includeSubdomains; preload
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
strict-transport-security: max-age=15768000
age: 17
x-cache: Hit from cloudfront
via: 1.1 76d9b50884e58e2463b175e34e790838.cloudfront.net (CloudFront)
x-amz-cf-id: HBbfJXbFgKQz4eYlQSpLPQAk8pxZRKuGsb6YjFu8-AJL0JwPfs8FZw==
As you can see the response header that the cdn.example.com (CloudFront) redirect to the origin (example.com).
Also, I am confused with the content-type:text/html which should be content-type: application/javascript
What are the possibilities that I may have misconfigured?
If anything more you want to know, Please feel free to ask. Thanks.
P.S: I am new to the Nginx and AWS configuration and most importantly cache control.
You will need to examine the origin server configuration and log files to understand why this happened.
Check the origin server logs and you will find that the origin server originally generated that 301 redirect response -- not CloudFront.
Notice that the response headers from CloudFront include server: nginx/1.12.2. CloudFront did not add this.
The Content-Type is set as it is, because your origin server also returned HTML saying something like "object has moved" or "you are being redirected." Modern browsers typically don't display that message, they just follow the redirect.
In any event, if the origin server returns a redirect, CloudFront does not follow the redirect returned by the origin server -- it simply returns the redirect to the browser.
One possible explanation for the behavior you see is that you set the Origin Protocol Policy in CloudFront to "HTTP Only" instead of "HTTPS Only" or "Match Viewer," so the origin is trying to redirect the connection to use HTTPS since it sees the incoming connection (from CloudFront) as HTTP instead of HTTPS.
Another possibility is that you configured CloudFront's Cache Behavior settings to whitelist the Host header for forwarding to the origin.
I tried to look around before posting but I wasn't able to found a solution for my issue.
I used to host a blog in a domain like http://example.com/blog and I moved it to the root /
What I'd like to do is to write a rule in NGINX telling to redirect my old articles from :
http://example.com/blog/my-dope-article
to
http://example.com/my-dope-article
Is there anyway to do that?
Thanks !
I tried this :
rewrite ^/blog/(.*) http://$server_name/$1 permanent;
But it doesn't work because http://example.com/blog/my-dope-article still got rewritten to http://example.com/blog/my-dope-article where I want to delete "/blog" part from the new URL.. :(
Have you looked at the redirects with curl or other tools to examine what exactly happens?
If you're running wordpress on / instead of /blog/ now but haven't yet changed the home- and siteurl-options, WP will create a redirect to the URL it thinks it is running at.
When you look at the headers (e.g. with curl -I http://example.com/ or by using the developer tools in your browser), you'll usually spot differences between redirects caused by nginx and redirects caused by whatever is behind nginx.
For example, a redirect by nginx itself might look like this:
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Tue, 03 Oct 2017 18:37:06 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://www.example.org/
While a redirect from apache/php/WP behind nginx might look like this
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Tue, 03 Oct 2017 18:39:44 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: keep-alive
Location: https://www.example.org/
Vary: Accept-Encoding
Content-Length: 339
Notice the moved Content-Length header, added Vary header and different Content-Type.
And, of course, you can look at the body they send with redirects as well (e.g. curl -v https://www.example.org/).
Example for nginx:
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
apache/WP
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved here.</p>
<hr>
<address>Apache/2.2.16 (Debian) Server at example.org Port 80</address>
</body></html>
Yes I tried to CURL to see what happened. In the end, I found what I've done wrong: it was the syntax..
So I just put : rewrite ^/blog(.*) /$1 last; at the server directive in my nginx conf file and voilà
I'm trying to specify a custom 404 page and preserve the URL, however using the below gives me the error nginx: [emerg] "proxy_pass" cannot have URI part in location given by regular expression, or inside named location, or inside "if" statement, or inside "limit_except" block in...
location / {
error_page 404 #404;
}
location #404 {
proxy_pass https://example.com/404.html;
}
Does anyone know a solution to this?
I have tested your config on nginx 1.6.2. The error message complains about the URL in proxy_pass having path specified. Specifying only https://server would be OK, but doesn't solve your issue.
Named locations (like #fallback) have been introduced also to avoid having to use non-existing locations (which could later become existing and cause issues). But we can also use a location which should never exist on the server itself, for example:
location / { error_page 404 /xx-404-xx; }
location /xx-404-xx { proxy_pass https://example.com/404.html; }
Redirecting to relative URL causes only internal nginx redirect, which does not change the URL as browser sees it.
EDIT: Here is the test result:
kenny:/etc/nginx/sites-enabled:# wget -S server.com/abc/abc
HTTP request sent, awaiting response...
HTTP/1.1 404 Not Found
Server: nginx/1.6.2
Date: Thu, 16 Jul 2015 07:01:38 GMT
Content-Type: text/html
Content-Length: 5
Connection: keep-alive
Last-Modified: Thu, 16 Jul 2015 07:00:59 GMT
ETag: "5037762-5-51af8a241434b"
Accept-Ranges: bytes
2015-07-16 09:01:38 ERROR 404: Not Found.
From apache access log:
1.2.3.4 - - [16/Jul/2015:09:01:38 +0200] "GET /test.html HTTP/1.0" 200 5 "-" "Wget/1.13.4 (linux-gnu)"
In nginx I have proxy_pass to a html file on Apache webserver with just "test\n" in it. As you can see, nginx fetched that, added headers from Apache (Last-Mod, ETag) and also Content-Length: 5, so it received the html file from Apache, but wget doesn't save the content of 404 errors. Also many browsers don't display 404 errors by default if they are smaller than 1 kB (they show their own error page instead). So either make your 404 page bigger or you can configure nginx to serve it as normal html with "200" result code (error_page 404 =200 /xx).
I have no idea why you are receiving external redirects with the same config. Try it with wget to see which headers exactly nginx sent. Also mixing http and https for proxy should be no issue. Try to remove everything from your config and test only this error page, maybe some other directive is causing this (like other location is used instead).
I'm doing PHP get_headers() on an mp3 file on my server and receive HTTP 404 when using the non-www address and HTTP 200 when using www.
I can access the file from either address in the browser, so why the 404? Can I fix this somehow with .htaccess?
1) WordPress is configured to use the non-www address (example.com)
2) The files are in the wp-content/uploads area of the WordPress install
3) The www subdomain has a DNS CNAME pointing to the non-www domain (www.example.com -> example.com)
Headers for: http://lhcsj.org/wp-content/uploads/2012/05/2012-5-6-sj.mp3
HTTP/1.1 404 Not Found
Date: Tue, 08 May 2012 21:11:43 GMT
Server: Apache/2.2.3 (CentOS)
Content-Length: 314
Connection: close
Content-Type: text/html; charset=iso-8859-1
Headers for: http://www.lhcsj.org/wp-content/uploads/2012/05/2012-5-6-sj.mp3
HTTP/1.1 200 OK
Date: Tue, 08 May 2012 21:08:05 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Mon, 07 May 2012 17:19:47 GMT
ETag: "9c52430-e3626f-7a1332c0"
Accept-Ranges: bytes
Content-Length: 14901871
Connection: close
Content-Type: audio/mpeg
The fact that www.example.com and example.com point to the same IP address via a DNS CNAME entry doesn't mean that the server is configured to serve both. This server could be configured to handle a multitude of HTTP hosts and the default might not be www.example.com but something else. It would in fact be unsurprising behaviour that it returns a 404 status for a host for which it's not configured (not even a default host).
Check that there is a VirtualHost entry in your Apache Httpd configuration for each of www.example.com and example.com. The fact that Wordpress is configured for a particular host only comes into play after having passed that step.