ingress configuration for dashboard - nginx

Question

I did nginx ingress controller tutorial from github and
exposed kubernetes dashboard
kubernetes-dashboard NodePort 10.233.53.77 <none> 443:31925/TCP 20d
created ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/ssl-passthrough: "true"
nginx.org/ssl-backends: "kubernetes-dashboard"
kubernetes.io/ingress.allow-http: "false"
name: dashboard-ingress
namespace: kube-system
spec:
tls:
- hosts:
- serverdnsname
secretName: kubernetes-dashboard-certs
rules:
- host: serverdnsname
http:
paths:
- path: /dashboard
backend:
serviceName: kubernetes-dashboard
servicePort: 443
ingress-nginx ingress-nginx NodePort 10.233.21.200 <none> 80:30827/TCP,443:32536/TCP 5h
https://serverdnsname:32536/dashboard
but dashboard throws error
2018/01/18 14:42:51 http: TLS handshake error from ipWhichEndsWith.77:52686: tls: first record does not look like a TLS handshake
and ingress controller logs
2018/01/18 14:42:51 [error] 864#864: *37 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 10.233.82.1, server: serverdnsname, request: "GET /dashboard HTTP/2.0", upstream: "http://ipWhichEndsWith.249:8443/dashboard", host: "serverdnsname:32536"
10.233.82.1 - [10.233.82.1] - - [18/Jan/2018:14:42:51 +0000] "GET /dashboard HTTP/2.0" 009 7 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64" 25 0.001 [kube-system-kubernetes-dashboard-443] ipWhichEndsWith.249:8443 7 0.001 200
On my mind it is related with nginx redirection to upstream: "http://ipWhichEndsWith.249:8443/dashboard" .
tried to update controller image version to 0.9.0-beta.19 - didnt help
Thank you for any help.

Answer 1

As you pointed out, looks like nginx is proxying your https request to ipWhichEndsWith.249:8443, which is an HTTPS endpoint, using http as protocol.
You should add the following annotation to your PodSpec:
LATEST
This annotation was added to replace the deprecated annotation since 0.18.0
#2871 Add support for AJP protocol
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
DEPRECATED
This annotation was deprecated in 0.18.0 and removed after the release of
0.20.0
#3203 Remove annotations grpc-backend and secure-backend already deprecated
nginx.ingress.kubernetes.io/secure-backends: "true"
This should make nginx forward your request to the pods with https.
Source: https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md#backend-protocol
Docs: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#backend-protocol

Answer 2

Just for code reference. There are 2 gtochas. Setting the proper annotations since the dashboard talks https and using the correct namepace for the ingress. tls config is optional.
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dashboard-google
namespace: kube-system
annotations:
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
tls:
- hosts:
- kube.mydomain.com
secretName: tls-secret
rules:
- host: kube.mydomain.com
http:
paths:
- path: /
backend:
serviceName: kubernetes-dashboard
servicePort: 443

Answer 3

To keep this ticket updated (if user uses nginx ingress) in order to reach the Kubernetes Dashboard you need to apply the following annotations:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
Do not use secure-backends on later versions than image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1. It is replaced by backend-protocol.
If the user is using ingress in non https port e.g. 80 can be done as documented here TLS termination (nging ingress documentation).
Sample of complete code with subdomain:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: kubernetes-dashboard-ingress
namespace: kubernetes-dashboard
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
tls:
- hosts:
- "dashboard.my.example.com"
secretName: kubernetes-dashboard-secret
rules:
- host: "dashboard.my.example.com"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
port:
number: 443
Hope this help other beginners like me not to spend so much time to figure out how to do it. Also the user should take in consideration the external load balancer configuration towards the ingress controller. Remember to set it up as SSL Pass-Through for the port that you will be forwarding.
Update: In case the user wants to use another ingress provider e.g. Kubernetes Ingress Controller Documentation/HAProxy Kubernetes Ingress/Controller 1.4
Sample of code with annotations:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: kubernetes-dashboard-ingress
namespace: kubernetes-dashboard
annotations:
haproxy.org/server-ssl: "true"
spec:
tls:
- hosts:
- "dashboard.my.example.com"
secretName: kubernetes-dashboard-secret
rules:
- host: "dashboard.my.example.com"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
port:
number: 443
The user should not forget that the secrets are unique per namespace.

Answer 4

You could also use the helm charts available here
helm-chart/kubernetes-dashboard
Then setup your values.yaml file in order to override ingress parts like enable it, and adding hosts are available.

Answer 5

Here is an ingress for the dashboard that works for me.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: kubernetes-dashboard
namespace: kube-system
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite ^(/dashboard)$ $1/ redirect;
spec:
ingressClassName: nginx
tls:
- hosts:
- yourdomain.com
secretName: kubernetes-dashboard-tls
rules:
- host: yourdomain.com
http:
paths:
- path: /dashboard(/|$)(.*)
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
port:
number: 443