I've put my linux apache webserver running on GCP behind the google load balancer. Because i only want https traffic i've redirected port 80 to 443 as shown below:
<VirtualHost *:80>
ServerName spawnparty.com
ServerAlias www.spawnparty.com
DocumentRoot /var/www/html/wwwroot
Redirect permanent / https://www.spawnparty.com
</VirtualHost>
i've given the vm an external ip adress to test if the redirect works and it does.
I've then configured the load balancer. i've made it so that the frondend accepts both http and https. for the backend i made 2 services:
one that uses http and one that uses https so that if somoeone enters though http it is forwarded and then redirected to https by the code shown above.
for both backend services is made a basic health check:
for http: port: 80, timeout: 5s, check interval: 5s, unhealthy
threshold: 2 attempts
for https: port: 443, timeout: 5s, check interval: 5s, unhealthy
threshold: 2 attempts
the https one works fine and states 1 of 1 instance healthy but the http health check states 0 of 1 instance healthy
if change the health check from http to https and back again for the http back end service it works for a short period of time but after a few minutes it states 0 of 1 instance healthy again.
What must i change to keep it healthy?
TL;DR - Use the same HTTPS health check for both the backend services.
Health Checking and response codes
You will need to respond with 200 response code and close the connection normally within the configured period.
HTTP and HTTPS health checks
If traffic from the load balancer to your instances uses the HTTP or
HTTPS protocol, then HTTP or HTTPS health checks verify that the
instance is healthy and the web server is up and serving traffic.
For an HTTP(S) health check probe to be deemed successful, the
instance must return a valid HTTP response with code 200 and close the
connection normally within the configured period. If it does this a
specified number of times in a row, the health check returns a status
of HEALTHY for that instance. If an instance fails a specified number
of health check probes in a row, it is marked UNHEALTHY without any
notification being sent. UNHEALTHY instances do not receive new
connections, but existing connections are allowed to continue.
UNHEALTHY instances continue to receive health check probes. If an
instance later passes a health check by successfully responding to a
specified number of consecutive health check probes, it is marked
HEALTHY and starts receiving new connections, again without any
notification.
Since you have 2 separate backend services (one for HTTP and other for HTTPS), you will need 2 health checks (although backend services allows reusing the same health check too if needed - read on) since the load balancer considers them independent services.
As you have already confirmed, using the HTTPS health check will work with the HTTPS based service, but using the HTTP health check will not. The reason is you are actually returning a HTTP 301 response code for permanent URL redirection instead of the expected HTTP 200 response code.
Possible Solution(s)
One way to fix this is to use HTTPS health checks for both the backend services, since your underlying service is still the same. You lose the ability to health check the redirection, but that unfortunately is not supported by the Google Cloud Load Balancer. You can share the same HTTPS health check resource too for both the backend services.
The solution posted by CharlesB will also work, but I feel you're adding additional redirection rules just to satisfy health checks and are not used on your service path anyway. You will also need a separate HTTP health check resource. Using just HTTPS health checks for both the backend services I feel is much simpler and also verifies that your service is alive to handle new requests.
Redirect everything except your health check page to HTTPS. The How to force rewrite to HTTPS except for a few pages in Apache? question explains how you can do that. GCE Network load balancing mentions this requirement, saying "Even if your service does not use HTTP, you'll need to at least run a basic web server on each instance that the health check system can query."
Related
I have a web application running on AWS EC2 using a load balancer which only listens for HTTPS requests on port 443 along with a security group which only allows for traffic on port 443 so my application cannot be reached via HTTP (as far as I am aware).
The application has recently been pen tested which resulted in the recommendation to implement HSTS. From what I can see one of the requirements for implementing HSTS is to redirect ALL HTTP links to HTTPS with a 301 Permanent Redirect. Upon trying to do this through another listener on the load balancer redirecting traffic from port 80 to port 443 I get a warning that the security group attached "does not allow traffic on this listener port."
My question is should I consider enabling HTTP traffic so that I can implement the permanent redirect and then implement HSTS or am I better off leaving things the way they are? I'm inclined to think that because my load balancer does not allow HTTP traffic that I'm better off leaving my current set up as is but not 100% sure. Can anyone explain which is the safer option and why? Thanks in advance.
It totally depends on whether you want to allow and redirect HTTP traffic to HTTPS. For example if it's a user facing application then generally HTTP traffic allowed at gateway/load balancer and then redirected to HTTPS as end user shouldn't get any error on accessing on HTTP. Then in this case HSTS header plays important role.
But it's not end-user facing application and you know consuming entity will always access using HTTPS then don't allow HTTP traffic. So here you won't need to have HSTS header set.
I need your help.
I have implemented a haproxy configuration which correctly manages both http and websocket backends, except in one specific scenario.
Here below a summary about how this stuff works:
When I connect to :2703/webapp, haproxy correctly redirects to one of the two http configured backends (webapp-lb1 or webapp-lb2).
When I connect to :2703/webapp/events, haproxy correctly redirects to one of the two websocket configured backends (websocket-lb1 or websocket-lb2)
Webapp is a servlet running in apache tomcat.
When I stop one of the two backend tomcats, haproxy correctly switches to the other one (for both the http and the websocket).
On the contrary, when I try to simulate an outage of one of the http backends by stopping the webapp via the tomcat manager, haproxy reports a HTTP Status 404 error but does not switch to the other backend.
Being that I explicitly configured the http-check expect status 302 directive, I would expect that - in case of a 404 status - haproxy switches to the other backend.
I had a look at the haproxy official documentation and I also tested the http-check disable-on-404 configuration but this is not what I need, as the haproxy behavior remains exactly the same of the above one.
For info, activating the http-check disable-on-404, haproxy detects the DOWN of the backend I stopped but does nothing (which as far as I understand, is exactly what we have to expect from the http-check disable-on-404 configuration in case of 404 status); here below the haproxy log when this option is enabled:
Jul 23 14:19:23 localhost haproxy[4037]: Server webapp-lb/webapp-lb2 is stopping, reason: Layer7 check conditionally passed, code: 404, info: "Not Found", check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
Here below an extract of my haproxy configuration:
frontend haproxy-webapp
bind *:2703
monitor-uri /haproxy_check
stats enable
stats uri /haproxy_stats
stats realm Strictly Private
stats auth admin:zqxwcevr
acl is_websocket url_beg /webapp/events
use_backend websocket-lb if is_websocket
default_backend webapp-lb
log global
backend webapp-lb
server webapp-lb1 192.168.136.129:8888 maxconn 400 check cookie webapp-lb1
server webapp-lb2 192.168.136.130:8888 maxconn 400 check cookie webapp-lb2
balance roundrobin
cookie JSESSIONID prefix nocache
log global
#http-check disable-on-404
option httpchk GET /webapp
http-check expect status 302
backend websocket-lb
server websocket-lb1 192.168.136.129:8888 maxconn 400 check
server websocket-lb2 192.168.136.130:8888 maxconn 400 check
balance roundrobin
log global
Please give me a hint as I am spending ages in reading documentation and forums with no success.
Thanks!
I need to use http health checks on a Elastic Beanstalk application, with proxy protocol turned on. That is currently not possible, and the health check fails with a an error --> *58 broken header while reading PROXY protocol
I figured I have two options
Perform the health check on another port, and setup nginx to listen to http requests on that port and proxy to my app.
If it is possible to catch the broken header errors, or detect regular http requests in the proxy_protocol server block, then redirect those requests to a port that listens to http.
I would prefer the latter(#2), if possible. So is there any way to do this?
Ideally, I would prefer not to have to do any of this. A feature request to fix this has been submitted to AWS, but it has no ETA.
The proxy protocol specification says:
The receiver MUST be configured to only receive the protocol described in this
specification and MUST not try to guess whether the protocol header is present
or not. This means that the protocol explicitly prevents port sharing between
public and private access. Otherwise it would open a major security breach by
allowing untrusted parties to spoof their connection addresses.
I think this means that option 2 is a sufficiently bad idea that it's not even supported by conforming implementations of the proxy protocol.
Option 1, on the other hand, seems pretty reasonable. You can set up a security group so that only legitimate health checks can come in on the port without proxy protocol enabled.
Another couple of options spring to mind too:
Simply point your health checks at the thing that's adding the header (i.e. ELB?), rather than directly at your Nginx instance. Not sure if this is possible with Elastic Beanstalk, it's not a service I use.
Use something else to add the proxy protocol header before forwarding the health-check traffic on to your Nginx, which would avoid having to duplicate your Nginx config. For instance a HAProxy running on the same machine as your Nginx could do this. Again, use security groups to ensure that only legitimate traffic gets through.
Before XYZ.com was down I noticed that my request was being routed to IP address 192.33.31.xxx and when it came up I noticed that my request was routed to IP 50.17.196.xxx , is it some sort of server switching? Isn't Dynamic server switching in case of fault is a way to make fault tolerant system?
Most of the heavy used websites works with load balancers directing the request to multiple servers. There are very simple to very complex logic based on which the routing is defined.
For e.g. the logic can be
Once the request is handled by one particular server the request from the same IP is handled by the same server OR
Use round robing way of handling request ( In this case IP has role to play)
route request to the least used server etc.
Website can also use hybrid approach of all the above.
Now if the website you were accessing was using the IP based routing then the case would have been
This is your first request. Then you would be assigned a server which will handle your request. (This can be round robin or other logic (for e.g. the server which has less load)
Now the load balancer might have logic that next X (say 100) requests or for Y (say 1 hour) time period, all requests from the same IP will be routed to the same server or both
So in your case if you make a request to XYZ it routes to some a.b.c.d server and then if you make another call after 1 hour or you make 101'th call you might get a different server which handles your request.
Now if your server went down, then you can have a fallback mechanisms like,
If a server do not respond in some pre configured time period, remove that server from live server List in load balancer configuration and reroute the request.
Or we can have an idle sitting pre running servers (it is costly but to be followed when each request has to served without fail). As soon as a server goes down, invoke this new server and assign it the ip which was assigned to the old server which went down.
There can be better and much more complex solution for this problem though.
I'm trying to understand the best way to handle SOA on heroku, i've got it into my head that making requests to custom domains will somehow be slower, or would all requests go "out" via the internet?
On previous projects which are SOA in nature we've had dedicated hosting so could make requests like http://blogs/ (obviously on the internal network) I'm wondering if heroku treats *.herokuapp.com requests as "internal"... Or is it clever enough to know the myapp.com is actually myapp.herokuapp.com and route locally, or am i missing the point completely, and in fact all requests are "external"
What you are asking about is general knowledge of how internet requests are working.
Whenever you do request from your application to lets say example.com, domain name will first be translated into IP address using so called DNS servers.
So this how it works: does not matter you request myapp.com or myapp.heroku.com you will always request infromation from specific IP address, and domain name you have requested will be passed as part of request headers.
Server which receives this request will try to find in its internal records this domain name and handle request accordingly.
So conclusion is that does not matter you put myapp.com or myapp.heroku.com, the speed of request will always be same.
PS: As heroku will load balance your requests between different instances of your running myapp.com, the speed here will depend on several factors: how quickly your application will respond, how many instances you have running and load average per instance, how much is load balancer loaded at the moment. But surely it will not depend on which domain name you use.