For some reason as of yesterday, my cloudfront doesn't work. I am constantly getting 502 errors.
I am using Nginx + Lets Encrypt on Ubuntu 16. Everything used to work just fine and I haven't made any changes to the server.
I spent hours looking for an answer and I am really stuck here as I don't know what's going wrong.
I checked the SSL certificate and it shows as valid with no errors. My SSL ciphers that I use in Nginx are:
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
Doing curl -i https://mycloudfrontdomain.cloudfront.net/wp-includes/js/jquery/jquery.js?ver=1.12.4 produces a valid response in my opinion.
HTTP/1.1 200 OK
Content-Type: application/javascript
Content-Length: 97184
Connection: keep-alive
Server: nginx
Date: Tue, 31 Oct 2017 08:54:04 GMT
Last-Modified: Sun, 04 Sep 2016 21:11:41 GMT
ETag: "xxx-17ba0"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Cache-Control: max-age=315360000
Access-Control-Allow-Origin: *
Accept-Ranges: bytes
X-Cache: Miss from cloudfront
Via: 1.1 xxx.cloudfront.net (CloudFront)
X-Amz-Cf-Id: xxx-id
The site loads fine with no errors, if I don't use Cloudfront.
ok changing the chiphers to (without the quotes):
ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:RSA+3DES:AES128-SHA:!ADH:!AECDH:!MD5;
seems to do the trick. Although I am not quite sure why it used to work fine so far before.
Related
My server was working fine until I restarted the server and now my program with cURL API stops working. After troubleshooting for a long time, I figured out what the problem is.
When I use this command:
curl -i https://server.my-site.com/checkConnection
Nginx returns error:
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 04 Jul 2019 17:14:40 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Location: /checkConnection/
but if I use this command:
curl -i -L https://server.my-site.com/checkConnection
Then the server return:
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 04 Jul 2019 17:14:40 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Location: /checkConnection/
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 04 Jul 2019 17:14:40 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 2
Connection: keep-alive
X-Frame-Options: SAMEORIGIN
ok
And if I use a browser, then everything works. I have no clue what the error comes from. and how to fix it.
Any help is appreciated!
This is what happens when the path maps to a directory. In theory, a URL like http://example.org/directory could map to a directory like /wherever/public_html/directory, and being a directory, show an index.html or similar file from there; however, that would cause surprising issues when you go to refer to other things like images in the same directory. <img src="picture.jpg"> would load http://example.org/picture.jpg rather than http://example.org/directory/picture.jpg since it's relative to the URL the browser is actually viewing. Because of this, HTTP servers generally issue a redirect to add a slash at the end, which then both loads the right page and at a URL where relative paths do what humans expect.
Adding -L to your curl commandline causes it to follow the redirect, as browsers do, and you get the result you were expecting. Without -L, curl is a more naive http client and lets you do what you will with the information.
Maybe you have a rule for www.server.my-site.com and that is why this is returning the 301 because it is redirecting from server.my-site.com to www site maybe you should share your configuration to check it
Ok. I finally fix it by adding an internal routing to uwsgi. Everything working fine now.
My application uses Varnish 3.0.2. I am facing a weird problem here. Some of the times the pages are served from Varnish with a HIT. But immediately after it returns MISS.
I was under the impression that once it gets served from the cache, it will continue to do so until the TTL expires. Am I wrong in understanding that?
Here are the two response headers for both the scenario:
HIT
HTTP/1.1 200 OK
Server: Apache/2.4.16 (Unix) mod_auth_kerb/5.4 PHP/5.3.29
X-Powered-By: PHP/5.3.29
X-Drupal-Cache: MISS
Content-Language: en
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
cache-control: max-age=86400, public
X-Cookie-Debug: Request cookie:
X-Request-URL: /org/31633421?unit=31633421
Content-Length: 11986
Accept-Ranges: bytes
Date: Wed, 24 Apr 2019 14:26:43 GMT
X-Varnish: 330015711 330015651
Via: 1.1 varnish
Connection: keep-alive
X-Varnish-Cache: HIT
X-Varnish-Cache-Hits: 1
X-Varnish-Age: 188
X-Varnish-Leg: 128.87.225.172
X-Varnish-Cache-Version: 3.0.2
MISS
HTTP/1.1 200 OK
Server: Apache/2.4.16 (Unix) mod_auth_kerb/5.4 PHP/5.3.29
X-Powered-By: PHP/5.3.29
X-Drupal-Cache: MISS
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=300
Content-Language: en
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
X-Cookie-Debug: Request cookie: _gat_UA-15166137-36=1
X-Request-URL: /org/31633421?unit=31633421
Content-Length: 11978
Accept-Ranges: bytes
Date: Wed, 24 Apr 2019 14:23:52 GMT
X-Varnish: 1900997574
Via: 1.1 varnish
Connection: keep-alive
X-Varnish-Cache: MISS
X-Varnish-Age: 0
X-Varnish-Leg: 128.87.225.158
X-Varnish-Cache-Version: 3.0.2
I have tried to increase the TTL value, remove all the cookies (including Google Analytics) but still it's behaving abruptly.
Any idea why?
Update
Seems like this is happening for including the following Google Tag manager JS code in my view template.
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-XXX');</script>
Turns out it WAS actually a problem in the VCL configuration with the regex I was using. I had not considered the non-alpha characters of the Google Ananlytics cookie. Modified the regex to _[_\-\.\=a-zA-Z0-9] and everything is fun again!
Hope this helps somebody.
My guess is that it comes from two different varnish servers based on the two response headers:
X-Varnish-Leg: 128.87.225.172
and
X-Varnish-Leg: 128.87.225.158
+1 for Ronald, also please consider upgrading to the latest Varnish 6 as years have passed since Varnish 3, many bugs have been fixed and improvements built. Furthermore V3 is end of life.
We launched a web app about a week ago, experienced a heavy load spike and were down for almost 2 hours. I won't mention the company by name, but we were leaning on them for recommendations to prevent this exact thing.
They said that since we were using Varnish, we could handle the traffic influx quite easily. However, we didn't verify caching was working as intended. It was not.
TLDR: Our web app is sending Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 headers with requests and there's no indication of why that is.
Where can I look to prevent these headers from being sent?
PHP: 5.6
Nginx: 1.4.6
Varnish: 1.1
Wordpress: 4.6.12
Timber: 1.2.4
The linux admins we're working with have said they scoured the configs and haven't found anything specifying those headers except for AJAX requests.
#dont cache ajax requests
if(req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache" || req.url ~ "(control.php|wp-comments-post.php|wp-login.php|bb-login.php|bb-reset-password.php|register.php)")
Here's a curl from Pre-launch when we correctly configured Varnish to cache after forcing HTTPS(force-https plugin) on the site:
$ curl -Ik -H'X-Forwarded-Proto: *************
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Content-Type: text/html; charset=UTF-8
Vary: Accept-Encoding
X-Server: *****
Date: Sat, 03 Nov 2018 22:36:43 GMT
X-Varnish: 53061104
Age: 0
Via: 1.1 varnish
Connection: keep-alive
And from post launch:
curl -ILk ***********
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
X-Varnish: 691817320
Vary: Accept-Encoding
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Content-Type: text/html; charset=UTF-8
X-Server: ****
Date: Mon, 19 Nov 2018 19:17:02 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Transfer-Encoding: chunked
Accept-Ranges: bytes
Via: 1.1 varnish
Connection: Keep-Alive
Set-Cookie: X-Mapping-fjhppofk=33C486CB71216B67C5C5AB8F5E63769E; path=/
Age: 0
Force-https plugin: We activated this, updated the Varnish config to avoid a redirect loop and confirmed it was working a week prior to launch.
Plugins: These did not change, except for force-https.
Web App: It's an updated version of the previous app, complete redesign but nothing in the app from what I can tell is specifying no-store no-cache headers to be sent.
Where should I start? Thanks!
What is sending these headers is PHP engine.
It does so whenever you initiate a session, which clearly happens based on Set-Cookie presence.
Make sure that PHP sessions are initiated only when absolutely needed. By default, Varnish will not cache when response includes either Set-Cookie or "negative" Cache-Control, you have both.
So getting rid of extraneous session_start() and/or setcookie() calls is the key here.
You can find more info on when you can expect anti-caching headers sent here.
You need to fix the backend, but at the very least, you can strip the annoying header, and/or bypass it in varnish with this vcl snippet;
sub vcl_backend_response {
# kill the CC header so that application downstream don't see it
unset req.http.Cache-Control;
# alternatively, you can also override that header with
# set req.http.Cache-Control = "whatever string you desire";
# you can also force the TTL
beresp.ttl;
# also, if you return now, the builtin.vcl (https://github.com/varnishcache/varnish-cache/blob/master/bin/varnishd/builtin.vcl)
# doesn't get executed; this is generally the one deciding content is uncacheable
return (deliver);
}
I'm having a strange issue when I reload a page sequentially, sometimes it loads fine, but sometimes it loads with missing images/css, and sometimes it redirects to my site's 404 file. Below are 2 sequential curl commands which might be helpful.
For background, I've already cleared out the previous page slugs in the DB, and restored the .htaccss file. Any pointers are greatly appreciated!!
[my machine]:$ curl -s -D - [my url] -o /dev/null
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Date: Tue, 04 Mar 2014 21:44:50 GMT
Server: Apache
X-Powered-By: PHP/5.3.14
Content-Length: 6131
Connection: keep-alive
[my machine]:$ curl -s -D - [my url] -o /dev/null
HTTP/1.1 404 Not Found
Cache-Control: no-cache, must-revalidate, max-age=0
Content-Type: text/html; charset=UTF-8
Date: Tue, 04 Mar 2014 21:44:51 GMT
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Last-Modified: Tue, 04 Mar 2014 21:44:51 GMT
Pragma: no-cache
Server: Apache
X-Pingback: http://promotions.glamour.com/xmlrpc.php
X-Powered-By: PHP/5.3.14
transfer-encoding: chunked
Connection: keep-alive
Wordpress randomly returning 404
I had a similar problem and I discovered the fix. Leaving this for whoever may come to this page as I did in search of a solution.
Turned out that it was because some plug-ins were causing a spike in memory consumption and the shared hosting (Dreamhost) was killing requests that took too much memory, and returned a 404 error. In my log files I was seeing "Premature end of script headers".
I disabled all non-critical plug-ins and not only have the random 404s stopped, but the site is loading much faster overall.
So it turns out the solution had nothing to do with Wordpress.
Long story short - a second duplicate server had been quietly set up to handle large traffic spikes, and our build scripts knew nothing about that new server and weren't deploying to it.
Ahh the joys of troubleshooting at a large company...
I have a quick question but in advance I've read the RFC 2616 Chapter 14.22 about Host and HTTP Header but I still not understand where in httpd.conf or configuration file of a webserver should be changed? Please correct me if I'm wrong.
Look at following two HTTP GET I did to an Apache. The first one is GET for HTTP 1.0 , the other one is GET for HTTP 1.1. See the output:
HTTP/1.0 200 OK
Date: Thu, 24 Oct 2013 03:46:22 GMT
Server: Apache/1.3.41 (Unix) mod_gzip/1.3.26.1a PHP/5.2.9 mod_throttle/3.1.2 mod_psoft_traffic/0.2 mod_ssl/2.8.31 OpenSSL/0.9.8b
Vary: *
Last-Modified: Fri, 10 Aug 2012 20:22:30 GMT
ETag: "17c815b-3b-50256d86"
Accept-Ranges: bytes
Content-Length: 59
Connection: close
Content-Type: text/html
<html>
<body>
<center>webli7</center>
</body>
</html>
HTTP/1.1 400 Bad Request
Date: Thu, 24 Oct 2013 04:04:40 GMT
Server: Apache/1.3.41 (Unix) mod_gzip/1.3.26.1a PHP/5.2.9 mod_throttle/3.1.2 mod_psoft_traffic/0.2 mod_ssl/2.8.31 OpenSSL/0.9.8b
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
16e
The HTTP protocol version is decided dynamicaly, not through configuration files. The client send a request specifying the highest protocol version that its support. Then, the server must respond with either the version requested by the client, or any earlier version that it prefers.
Since Apache does support HTTP/1.1, it should therefore match exactly the version provided by the client.
There exist a flag that you may set in Apache's config to force Apache to use HTTP/1.0 in certain situations, even though the browser requested HTTP/1.1. This is used to fix bugs in HTTP/1.1 handling of some very old browser. Today, you should not need to play with this flag.
As for your error, I would suggest that you make sure that your GET does provide the Host: header. This header is required in HTTP/1.1, yet optional in HTTP/1.0, and having it missing would certainly result in a 400 error.