I'm using kubernetes ingress-nginx and this is my Ingress spec. http://example.com works fine as expected. But when I go to https://example.com it still works, but pointing to default-backend with Fake Ingress Controller certificate. How can I disable this behaviour? I want to disable listening on https at all on this particular ingress, since there is no TLS configured.
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: http-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: my-deployment
servicePort: 80
I've tried this nginx.ingress.kubernetes.io/ssl-redirect: "false" annotation. However this has no effect.
I'm not aware of an ingress-nginx configmap value or ingress annotation to easily disable TLS.
You could remove port 443 from your ingress controllers service definition.
Remove the https entry from the spec.ports array
apiVersion: v1
kind: Service
metadata:
name: mingress-nginx-ingress-controller
spec:
ports:
- name: https
nodePort: NNNNN
port: 443
protocol: TCP
targetPort: https
nginx will still be listening on a TLS port, but no clients outside the cluster will be able to connect to it.
Redirection is not involved in your problem.
ingress-controller is listening on both port, 80 and 443. When you configure an ingress with only 80 port, if you reach the 443 port you are redirected to the default backend, which is expected behaviour.
A solution is to add an other nginx-controller, that will only listen on 80 port. And then you can configure your ingresses with kubernetes.io/ingress.class: myingress.
When creating the new nginx-controller, change the command --ingress-class=myingress of the daemonset. It will then handle only ingress annotated with this class.
If you use helm to deploy it, simply override the controller.ingressClass value.
Related
I have defined my service app running on port 9000. It is not web/http server it is simply just a service application running as windows service on that port to which other apps connect to (outside the container).
So I have defined Port 9000 in my service definition and in my config map definition. We are using NGINX as a proxy for accessing from outside and everything works.
Nginx Service:
- name: 9000-tcp
nodePort: 30758
port: 9000
protocol: TCP
targetPort: 9000
Config Map:
apiVersion: v1
data:
"9000": default/frontarena-ads-aks-test:9000
kind: ConfigMap
Service definition:
apiVersion: v1
kind: Service
metadata:
name: frontarena-ads-aks-test
spec:
type: ClusterIP
ports:
- protocol: TCP
port: 9000
selector:
app: frontarena-ads-aks-test
Ingress definition:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ads-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: frontarena-ads-aks-test
servicePort: 9000
As mentioned everything works. I know that TCP is used for L4 layer and HTTP for L7 Application Layer.
I need to access my app from another app solely by its hostname and port. Without any HTTP Url.
So basically does it mean that I do NOT need actually my Ingress Controller definition at all?
I do not need to deploy it at all?
I would only need it if I need HTTP access with some URL for example: hostname:port/pathA or hostname:port/pathB
Is that correct? For regular TCP connection we do not need at all our Ingress YAML definition? Thank you
Yes, you don't need ingress at all in this case. According to kubernetes official doc, Ingress is:
An API object that manages external access to the services in a cluster, typically HTTP.
So, if you don't need any external access via http, you can omit ingress.
Ref: https://kubernetes.io/docs/concepts/services-networking/ingress/
I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routing¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.
I'm using ingress-nginx-controller (0.32.0) and am attempting to point an ExternalName service at a URL and yet it’s stuck in a loop of 308 Redirects. I've seen plenty of issues out there and I figure there’s just one thing off with my configuration. Is there something really small that I'm missing here?
ConfigMap for NGINX Configuration:
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
data:
use-proxy-protocol: "true"
use-forwarded-headers: "true"
proxy-real-ip-cidr: "0.0.0.0/0" # restrict this to the IP addresses of ELB
proxy-read-timeout: "3600"
proxy-send-timeout: "3600"
backend-protocol: "HTTPS"
ssl-redirect: "false"
http-snippet: |
map true $pass_access_scheme {
default "https";
}
map true $pass_port {
default 443;
}
server {
listen 8080 proxy_protocol;
return 308 https://$host$request_uri;
}
NGINX Service:
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "XXX"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600"
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
spec:
type: LoadBalancer
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
targetPort: 8080
- name: https
port: 443
targetPort: http
Ingress Definition:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-members-portal
namespace: dev
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: foo-111.dev.bar.com
http:
paths:
- path: /login
backend:
serviceName: foo-service
servicePort: 80
ExternalName Service:
apiVersion: v1
kind: Service
metadata:
name: foo-service
spec:
type: ExternalName
externalName: foo.staging.bar.com
selector:
app: foo
EDIT
I figured it out! I wanted to point to a service in another namespace, so I changed the ExternalName service to this:
apiVersion: v1
kind: Service
metadata:
name: foo-service
spec:
type: ExternalName
externalName: foo-service.staging.svc.cluster.local
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: foo
I believe the issue you're seeing is due to the fact that your external service isn't working as you think it is. In your ingress definition, you are defining the service to utilize port 80 on your foo-service. In theory, this would redirect you back to your ingress controller's ELB, redirect your request to the https://foo.staging.bar.com address, and move on.
However, external services don't really work that way. Essentially, all externalName will do is run a DNS check with KubeDNS/CoreDNS, and return the CNAME information on that request. It doesn't handle redirects of any kind.
For example, in this case, foo.staging.bar.com:80 would return foo.staging.bar.com:443. You are directing the request for that site to port 80, which in itself directs the request to port 8080 in the ingress controller, which then redirects that request back out to the ELB's port 443. That redirect logic doesn't coexist with the external service.
The problem here, then, is that your app will essentially then try to do this:
http://foo-service:80 --> http://foo.staging.bar.com:80/login --> https://foo.staging.bar.com:443
My expectation on this is that you never actually reach the third step. Why? Well because foo-service:80 is not directing you to port 443, first of all, but second of all...all coreDNS is doing in the backend is running a DNS check against the foo-service's external name, which is foo.staging.bar.com. It's not handling any kind of redirection. So depending on how your host from your app is returned, and handled, your app may never actually get to that site and port. So rather than reach that site, you just have your app keep looping back to http://foo-service:80 for those requests, which will always result in a 308 loopback.
The key here is that foo-service is the host header being sent to the NGINX Ingress controller, not foo.staging.bar.com. So on a redirect to 443, my expectation is that all that is happening, then, is you are hitting foo-service, and any redirects are being improperly sent back to foo-service:80.
A good way to test this is to run curl -L -v http://foo-service:80 and see what happens. That will follow all redirects from that service, and provide you context as to how your ingress controller is handling those requests.
It's really hard to give more information, as I don't have access to your setup directly. However, if you know that your app is always going to be hitting port 443, it would probably be a good fix, in this case, to change your ingress and service to look something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-members-portal
namespace: dev
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: foo-111.dev.bar.com
http:
paths:
- path: /login
backend:
serviceName: foo-service
servicePort: 443
---
apiVersion: v1
kind: Service
metadata:
name: foo-service
spec:
type: ExternalName
externalName: foo.staging.bar.com
That should ensure you don't have any kind of https redirect. Unfortunately, this may also cause issues with ssl validation, but that would be another issue all together. The last piece of I can recommend is to possibly just use foo.staging.bar.com itself, rather than an external service in this case.
For more information, see: https://kubernetes.io/docs/concepts/services-networking/service/#externalname
Hope this helps!
I'm having a problem where my NGINX headers are not showing when I connect to the ingress controller's IP. For example, let's say I have a HTTP header called "x-Header" setup in my NGINX config map. If I go to the ingress controller's IP I don't see the header, but when I go to the NGINX load balancer IP, I see the headers. Also when I go to the ingress IP, I get the correct cert, but the NGINX IP gives me a self signed cert. I don't think I have the controller setup right.
Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/ingress.global-static-ip-name: staticip
labels:
app: exam
spec:
tls:
- hosts:
- www.example.com
secretName: tls-secret
backend:
serviceName: example-app
servicePort: 80
Ingress node:
apiVersion: v1
kind: Service
metadata:
name: exam-node
labels:
app: exam
spec:
type: NodePort
selector:
app: example-app
tier: backend
ports:
- port: 80
targetPort: 80
To create the NGINX controller I used the command helm install stable/nginx-ingress --name nginx-ingress --set rbac.create=true
My nginx controller logs say this:
2018-07-29 19:57:11.000 CDT
updating Ingress default/example-ingress status to [{12.346.151.45 }]
It seems the nginx controller knows about ingress but ingress doesn't know about the controller. I am probably confused on how Ingress and NGINX work together.
I'm trying to connect have NGINX direct traffic to different parts of my app through the config file, but I can't figure it out for the life of me. Here is my current setup:
http-service (loadbalancer)
NGINX (port 80)
website-service (10.27.246.107, port 8000, targetPort 8000, selector 'run: website')
website (label 'run: website', containerPort 8000)
NGINX Conf
upstream website{
server 10.27.246.107:8000
}
This is a normal nginx pod using containerPort 80 at the moment.
Am I going about this the right way?
The best way to rout traffic to different part of your application is using Ingress. In Ingress, you can describe all your paths into all parts of your application. It will look like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: website1
servicePort: 80
- path: /bar
backend:
serviceName: website2
servicePort: 3368
Actually, Ingress-controller is based on Nginx, but anyway you can choose another engine, for example, HAproxy.
Ingress was designed for using in Kubernetes and it has more features in Kubernetes. For example, your website upstream should be described as a service in Kubernetes:
kind: Service
apiVersion: v1
metadata:
name: website1
spec:
selector:
app: python-web-site
ports:
- protocol: TCP
port: 80
targetPort: 8080
Anyway, you can route traffic by Nginx and expose it to the world, but the best practice is to uses Ingress in Kubernetes.