The External IP of my Nginx load balancer dones not work - nginx

Ok lets to explain my probrem...
I have deployed a Kind Kubernetes. This is my script:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: kind
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
# Mongo
- containerPort: 30005
hostPort: 27017
protocol: TCP
- role: worker
extraMounts:
- hostPath: C:/Kind
containerPath: /data
- role: worker
extraMounts:
- hostPath: C:/Kind
containerPath: /data
- role: worker
extraMounts:
- hostPath: C:/Kind
containerPath: /data
The next step is deploy MetalLB (the Load Balancer). I have used thoose yamls:
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml
To configure the layer 2, I set a ip range, inside the kind network. For know it:
docker network inspect -f '{{.IPAM.Config}}' kind
This commmnad show this:
[{172.18.0.0/16 172.18.0.1 map[]} {fc00:f853:ccd:e793::/64 fc00:f853:ccd:e793::1 map[]}]
So, I set the following configmap:
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 172.18.255.200-172.18.255.250
Ok, the last step is install Nginx controller, I did with the following commnad:
helm install nginx-ingress-controller bitnami/nginx-ingress-controller
All deployed ok and with this command I can see all:
kubectl get all
This command show:
NAME READY STATUS RESTARTS AGE
pod/ddclient-deployment-fcbf95d66-ndldk 1/1 Running 0 51m
pod/nginx-ingress-controller-6b9cf4684f-7hsw2 1/1 Running 0 64s
pod/nginx-ingress-controller-default-backend-6798d86668-7b552 1/1 Running 0 64s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17h
service/nginx-ingress-controller LoadBalancer 10.96.49.179 172.18.255.200 80:30307/TCP,443:31387/TCP 64s
service/nginx-ingress-controller-default-backend ClusterIP 10.96.247.49 <none> 80/TCP 64s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ddclient-deployment 1/1 1 1 51m
deployment.apps/nginx-ingress-controller 1/1 1 1 64s
deployment.apps/nginx-ingress-controller-default-backend 1/1 1 1 64s
NAME DESIRED CURRENT READY AGE
replicaset.apps/ddclient-deployment-fcbf95d66 1 1 1 51m
replicaset.apps/nginx-ingress-controller-6b9cf4684f 1 1 1 64s
replicaset.apps/nginx-ingress-controller-default-backend-6798d86668 1 1 1 64s
Well, here is the problem. In theory, if you put the load balacer external ip:
service/nginx-ingress-controller LoadBalancer 10.96.49.179 172.18.255.200 80:30307/TCP,443:31387/TCP 64s
in the browser, you should see the nginx web page. I cant, just see an error message saying
"ERR_CONNECTION_TIMED_OUT".
I dont know what I am missing...
Thanks for the help!

Related

Access K8s Services via Ingress

We have configured MetalLB since our K8s cluster is hosted on bare metal infrastructure. It seems to be running fine with all pods up and running.
[~]# kubectl get all -n metallb-system
NAME READY STATUS RESTARTS AGE
pod/controller-b78574c59-47qfv 1/1 Running 0 24h
pod/speaker-4q2vm 1/1 Running 0 24h
pod/speaker-m8kwk 1/1 Running 0 24h
pod/speaker-t4rvs 1/1 Running 0 24h
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 3 3 3 3 3 kubernetes.io/os=linux 24h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 24h
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-b78574c59 1 1 1 24h
We have configured ingress controller via helm from https://github.com/kubernetes/ingress-nginx/releases/tag/helm-chart-3.29.0 and updating hostNetwork,ingressClass,kind to true,ingress-nginx,DaemonSet respectively in file values.yaml. The helm installation seems to have worked fine with all daemonset pods running and an LB ip provided to created ingress controller service.
[~]# kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/devingress-ingress-nginx-controller-c2x42 1/1 Running 0 18h
pod/devingress-ingress-nginx-controller-wtmgw 1/1 Running 0 18h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/devingress-ingress-nginx-controller LoadBalancer x.x.x.x 1.2.3.40 80:32386/TCP,443:30020/TCP 18h
service/devingress-ingress-nginx-controller-admission ClusterIP x.x.x.x <none> 443/TCP 18h
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/devingress-ingress-nginx-controller 2 2 2 2 2 kubernetes.io/os=linux 18h
Now we have deployed two pods namely nginx with LoadBalancer service type & nginx-deploy-main with ClusterIP service type.
[~]# kubectl get all -n default
NAME READY STATUS RESTARTS AGE
pod/nginx-854cf6b4d7-lv5ss 1/1 Running 0 18h
pod/nginx-deploy-main-6b5457fbb5-7tg9z 1/1 Running 0 18h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx LoadBalancer x.x.x.x 1.2.3.41 8080:31101/TCP 18h
service/nginx-deploy-main ClusterIP x.x.x.x <none> 80/TCP 18h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 18h
deployment.apps/nginx-deploy-main 1/1 1 1 18h
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-854cf6b4d7 1 1 1 18h
replicaset.apps/nginx-deploy-main-6b5457fbb5 1 1 1 18h
Below is the ingress resource setup to access nginx-deploy-main.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: nginx
spec:
ingressClassName: nginx
rules:
- host: nginx-main.int.org.com
http:
paths:
- path: /
backend:
serviceName: nginx-deploy-main
servicePort: 80
And the ingress resource seems to be created correctly pointing to nginx-deploy-main service.
[~]# kubectl get ing -n default
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-resource nginx nginx-main.int.org.com 80 19h
[~]# kubectl describe ing/ingress-resource -n default
Name: ingress-resource
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
nginx-main.int.org.com
/ nginx-deploy-main:80 (x.x.x.x:80)
Annotations: kubernetes.io/ingress.class: nginx
Events: <none>
Outside of K8s cluster, we have nginx set up serving as reverse proxy with domain int.org.com resolution.
Below is the nginx configuration which should help me hit url http://nginx-main.int.org.com and get response but the response returned is 404.
upstream nginx-main.int.org.com {
server 1.2.3.40:80; ## Ingress Controller Service IP
}
server {
listen 80;
server_name nginx-main.int.org.com;
location / {
proxy_pass http://nginx-main.int.org.com;
}
}
Now when I try to access nginx pod (not nginx-main) using its LoadBalancer Service IP with below configuration , its able to provide response and works just fine
upstream nginx.int.org.com {
server 1.2.3.41:8080;
}
server {
listen 80;
server_name nginx.int.org.com;
location / {
proxy_pass http://nginx.int.org.com;
}
}
Am I missing something here with regards to Ingress Controller or Resource. Port Forwarding works fine and am able to access services using the same.
This really is a blocker and any help or documentation reference would be really useful .
We tried with another Ingress Controller i.e. https://github.com/nginxinc/kubernetes-ingress and were able to make it work .
Below were the steps done .
[~] git clone https://github.com/nginxinc/kubernetes-ingress/
[~] cd kubernetes-ingress/deployments
[~] git checkout v1.11.1
[~] kubectl apply -f common/ns-and-sa.yaml
[~] kubectl apply -f rbac/rbac.yaml
[~] kubectl apply -f common/default-server-secret.yaml
[~] kubectl apply -f common/nginx-config.yaml
[~] kubectl apply -f common/ingress-class.yaml
Created daemon-set pods with extra environment argument i.e. --enable-custom-resources=false added in yaml due to below issue in controller logs
Refer : Kubernetes cluster working but getting this error from the NGINX controller
[~] kubectl apply -f daemon-set/nginx-ingress.yaml
[~] kubectl get all -n nginx-ingress -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/nginx-ingress-gd8gw 1/1 Running 0 3h55m x.x.x.x worker1 <none> <none>
pod/nginx-ingress-kr9lx 1/1 Running 0 3h55m x.x.x.x worker2 <none> <none>
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR
daemonset.apps/nginx-ingress 2 2 2 2 2 <none> 5h14m nginx-ingress nginx/nginx-ingress:1.11.1 app=nginx-ingress
Hit respective worker nodes at port 80 and a 404 response means its working fine.
Deployed a sample application using github link https://github.com/vipin-k/Ingress-Controller-v1.9.0/blob/main/hotel.yml and updated host entry within Ingress object to hotel.int.org.com
[~] kubectl create -f hotel.yaml
[~] kubectl get all -n hotel -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/hotel-65d644c8f7-bj597 1/1 Running 0 3h51m x.x.x.x worker1 <none> <none>
pod/hotel-65d644c8f7-csvgp 1/1 Running 0 3h51m x.x.x.x worker2 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/hotel-svc ClusterIP x.x.x.x <none> 80/TCP 3h51m app=hotel
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/hotel 2/2 2 2 3h51m hotel nginxdemos/hello:plain-text app=hotel
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/hotel-65d644c8f7 2 2 2 3h51m hotel nginxdemos/hello:plain-text app=hotel,pod-template-hash=65d644c8f7
[~] kubectl get ing -n hotel
NAME CLASS HOSTS ADDRESS PORTS AGE
hotel-ingress nginx hotel.int.org.com 80 3h52m
[~] kubectl describe ing hotel-ingress -n hotel
Name: hotel-ingress
Namespace: hotel
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
hotel.int.org.com
/ hotel-svc:80 (x.x.x.x:80,x.x.x.x:80)
Annotations: Events: <none>
Updated external nginx configuration with enabled domain resolution .
upstream hotel.int.org.com {
server 1.2.3.41:80; #worker1
server 1.2.3.42:80; #worker2
}
server {
listen 80;
server_name hotel.int.org.com;
location / {
proxy_pass http://hotel.int.org.com;
}
}
Restart nginx and verify able to access via browser its serving response from respective running hotel namespace daemonset pods.
[~]# curl hotel.int.org.com
Server address: x.x.x.x:80
Server name: hotel-65d644c8f7-bj597
Date: 28/Apr/2021:05:47:15 +0000
URI: /
Request ID: 28d5cfab4ea28beea49879422b7e8f4c
[~]# curl hotel.int.org.com
Server address: x.x.x.x:80
Server name: hotel-65d644c8f7-csvgp
Date: 28/Apr/2021:05:52:06 +0000
URI: /
Request ID: 4135cacf83f8bf41c9677104500e610b
Exploring with MetalLB too and will post solution once its works

How to access simple nginx deployment on kubernetes?

I want to deploy a simple nginx app on my own kubernetes cluster.
I used the basic nginx deployment. On the machine with the ip 192.168.188.10. It is part of cluster of 3 raspberries.
NAME STATUS ROLES AGE VERSION
master-pi4 Ready master 2d20h v1.18.2
node1-pi4 Ready <none> 2d19h v1.18.2
node2-pi3 Ready <none> 2d19h v1.18.2
$ kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
$ kubectl create service nodeport nginx --tcp=80:80
service/nginx created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
my-nginx-8fb6d868-6957j 1/1 Running 0 10m
my-nginx-8fb6d868-8c59b 1/1 Running 0 10m
nginx-f89759699-n6f79 1/1 Running 0 4m20s
$ kubectl describe service nginx
Name: nginx
Namespace: default
Labels: app=nginx
Annotations: <none>
Selector: app=nginx
Type: NodePort
IP: 10.98.41.205
Port: 80-80 80/TCP
TargetPort: 80/TCP
NodePort: 80-80 31400/TCP
Endpoints: <none>
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
But I always get a time out
$ curl http://192.168.188.10:31400/
curl: (7) Failed to connect to 192.168.188.10 port 31400: Connection timed out
Why is the web server nginx not reachable? I tried to run it from the same machine I deployed it to? How can I make it accessible from an other machine from the network on port 31400?
As mentioned by #suren, you are creating a stand-alone service without any link with your deployment.
You can solve using the command from suren answer, or creating a new deployment using the follow yaml spec:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- name: http
containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
spec:
type: NodePort
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
After, type kubectl get svc to get the nodeport to access your service.
nginx-svc NodePort 10.100.136.135 <none> 80:31816/TCP 34s
To access use http://<YOUR_NODE_IP>:31816
so is 192.168.188.10 your host ip / your vm ip ?
you have to check it first if any service using that port or maybe you haven't add it into your security group if you using cloud platform.
just to make sure you can create a pod and access it using fqdn like my-svc.my-namespace.svc.cluster-domain.example

Error 404 after deploying and exposing Nginx pod

I have deployed and exposed Nginx with the following commands:
sudo kubectl create deployment mynginx1 --image=nginx
sudo kubectl expose deployment mynginx1 --type NodePort --port 8080
I access using http://<master node IP>:<port> or http://172.17.135.42:31788
But I am getting Error 404. Help appreciated.
gtan#master:~$ kubectl get pods -owide -A
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default mynginx1-f544c49cb-g92w2 1/1 Running 0 3m19s 172.168.10.2 slave1 <none> <none>
kube-system coredns-66bff467f8-92r4n 1/1 Running 0 7m56s 172.168.10.2 master <none> <none>
kube-system coredns-66bff467f8-gc7tc 1/1 Running 0 7m56s 172.168.10.3 master <none> <none>
kube-system etcd-master 1/1 Running 0 8m6s 172.17.82.100 master <none> <none>
kube-system kube-apiserver-master 1/1 Running 0 8m6s 172.17.82.100 master <none> <none>
kube-system kube-controller-manager-master 1/1 Running 0 8m6s 172.17.82.100 master <none> <none>
kube-system kube-flannel-ds-amd64-24pwc 1/1 Running 3 4m58s 172.17.82.110 slave1 <none> <none>
kube-system kube-flannel-ds-amd64-q5qwg 1/1 Running 0 5m28s 172.17.82.100 master <none> <none>
kube-system kube-proxy-hf59b 1/1 Running 0 4m58s 172.17.82.110 slave1 <none> <none>
kube-system kube-proxy-r7pz6 1/1 Running 0 7m56s 172.17.82.100 master <none> <none>
kube-system kube-scheduler-master 1/1 Running 0 8m5s 172.17.82.100 master <none> <none>
gtan#master:~$
gtan#master:~$ curl -IL http://172.17.82.100:30131
curl: (7) Failed to connect to 172.17.82.100 port 30131: Connection refused where "172.17.82.100" is the master node ip address.
gtan#master:~$ kubectl get services -o wide -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 15m <none>
default mynginx1 NodePort 10.102.106.240 <none> 80:30131/TCP 10m app=mynginx1
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 15m k8s-app=kube-dns
What is the architecture of your setup? do you have worker node and master node on same machine?
check the nginx pod status with :
kubectl get pods
If the pod is running without issue then hit your worker machine IP with NodePort http:/Workernode_IP:Nodeport
The default nginx container port is 80 as you can see here. Just change the container port from 8080 to 80 in your second command:
sudo kubectl expose deployment mynginx1 --type NodePort --port 80
and try to reach the service using the NodePort showed in the output of the command, for example:
$kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mynginx1 NodePort 10.97.142.170 <none> 80:31591/TCP 8m9s
Altenatively, you can use this yaml spec to configure your pod and service:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- name: http
containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
type: NodePort
Testing using curl:
$ curl -IL http://localhost:31591
HTTP/1.1 200 OK
Server: nginx/1.17.10
Date: Tue, 12 May 2020 10:05:04 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 14 Apr 2020 14:19:26 GMT
Connection: keep-alive
ETag: "5e95c66e-264"
Accept-Ranges: bytes
Also, I recommend you to reserve a time to take a look in these documentations pages:
Kuberentes Concepts
Services

Kubernetes services - Portforwarding to all pods through deployment

I want to showcase kubernetes load balancing capabilities. On my local system, I have one node in the cluster. Want to deploy nginx container in 3 pods and replace the index.html (default) with my modified index.html (having some variances). I am creating a service and assigning a port to forward all requests to port 80 of the containers. I want to access my pod as http://localhost:3030. Depending on the pod the request hits, the index.html will display the content. However with the below deployment and service code I could not hit any pod. If I do port-forward to an individual pod, I can reach it though.
I followed the approach explained here but no luck. Any idea what I am missing.
Here is what I see when get all.
$ k get all
NAME READY STATUS RESTARTS AGE
pod/app-server-6ccf5d55db-2qt2r 1/1 Running 0 3d20h
pod/app-server-6ccf5d55db-96lkb 1/1 Running 0 3d20h
pod/app-server-6ccf5d55db-ljsc4 1/1 Running 0 3d20h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 19d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/app-server 3/3 3 3 3d20h
apiVersion: v1
kind: Service
metadata:
name: app-service
spec:
type: NodePort
ports:
- name: http
protocol: TCP
port: 80
targetPort: 3030
selector:
app: app-server
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-server
labels:
app: app-server
spec:
replicas: 3
selector:
matchLabels:
app: app-server
template:
metadata:
labels:
app: app-server
spec:
containers:
- name: web-server
image: nginx:latest
ports:
- containerPort: 80
Ok, I did two mistakes.
Both service and app server deployment is in single file.
I messed up the port and servicePort values
Here are the changes I made which worked.
Service.yml
apiVersion: v1
kind: Service
metadata:
name: app-service
spec:
type: NodePort
ports:
- name: httpport
protocol: TCP
port: 32766
nodePort: 32766
targetPort: 80
selector:
app: app-server
deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-server
labels:
app: app-server
spec:
replicas: 3
selector:
matchLabels:
app: app-server
template:
metadata:
labels:
app: app-server
spec:
containers:
- name: web-server
image: nginx:latest
ports:
- containerPort: 80
I deployed the server first and then the service. Then I was able to reach the nginx server with http://localhost:32766
Here is the output of my k get all
$ k get all -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/app-server-6ccf5d55db-9xjwh 1/1 Running 0 60s 10.1.0.201 docker-desktop <none> <none>
pod/app-server-6ccf5d55db-mdtrx 1/1 Running 0 60s 10.1.0.200 docker-desktop <none> <none>
pod/app-server-6ccf5d55db-smmcg 1/1 Running 0 60s 10.1.0.199 docker-desktop <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/app-service NodePort 10.110.72.85 <none> 32766:32766/TCP 54s app=app-server
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20d <none>
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/app-server 3/3 3 3 60s web-server nginx:latest app=app-server
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/app-server-6ccf5d55db 3 3 3 60s web-server nginx:latest app=app-server,pod-template-hash=6ccf5d55db

Kubernetes nginx ingress controller as loadbalancer gets random ports

I'm trying to expose kubernetes dashboard publicly via an ingress on a single master bare-metal cluster. The issue is that the LoadBalancer (nginx ingress controller) service I'm using is not opening the 80/443 ports which I would expect it to open/use. Instead it takes some random ports from the 30-32k range. I know I can set this range with --service-node-port-range but I'm quite certain I didn't have to do this a year ago on another server. Am I missing something here?
Currently this is my stack/setup (clean install of Ubuntu 16.04):
Nginx Ingress Controller (installed via helm)
MetalLB
Kubernetes Dashboard
Kubernetes Dashboard Ingress to deploy it publicly on <domain>
Cert-Manager (installed via helm)
k8s-dashboard-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
# add an annotation indicating the issuer to use.
cert-manager.io/cluster-issuer: letsencrypt-staging
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
name: kubernetes-dashboard-ingress
namespace: kubernetes-dashboard
spec:
rules:
- host: <domain>
http:
paths:
- backend:
serviceName: kubernetes-dashboard
servicePort: 443
path: /
tls:
- hosts:
- <domain>
secretName: kubernetes-dashboard-staging-cert
This is what my kubectl get svc -A looks like:
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cert-manager cert-manager ClusterIP 10.101.142.87 <none> 9402/TCP 23h
cert-manager cert-manager-webhook ClusterIP 10.104.104.232 <none> 443/TCP 23h
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6d6h
ingress-nginx nginx-ingress-controller LoadBalancer 10.100.64.210 10.65.106.240 80:31122/TCP,443:32697/TCP 16m
ingress-nginx nginx-ingress-default-backend ClusterIP 10.111.73.136 <none> 80/TCP 16m
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 6d6h
kubernetes-dashboard cm-acme-http-solver-kw8zn NodePort 10.107.15.18 <none> 8089:30074/TCP 140m
kubernetes-dashboard dashboard-metrics-scraper ClusterIP 10.96.228.215 <none> 8000/TCP 5d18h
kubernetes-dashboard kubernetes-dashboard ClusterIP 10.99.250.49 <none> 443/TCP 4d6h
Here are some more examples of what's happening:
curl -D- http://<public_ip>:31122 -H 'Host: <domain>'
returns 308, as the protocol is http not https. This is expected
curl -D- http://<public_ip> -H 'Host: <domain>'
curl: (7) Failed to connect to <public_ip> port 80: Connection refused
port 80 is closed
curl -D- --insecure https://10.65.106.240 -H "Host: <domain>"
reaching the dashboard through an internal IP obviously works and I get the correct k8s-dashboard html.
--insecure is due to the let's encrypt not working yet as the acme challenge on port 80 is unreachable.
So to recap, how do I get 2. working? E.g. reaching the service through 80/443?
EDIT: Nginx Ingress Controller .yaml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2020-02-12T20:20:45Z"
labels:
app: nginx-ingress
chart: nginx-ingress-1.30.1
component: controller
heritage: Helm
release: nginx-ingress
name: nginx-ingress-controller
namespace: ingress-nginx
resourceVersion: "1785264"
selfLink: /api/v1/namespaces/ingress-nginx/services/nginx-ingress-controller
uid: b3ce0ff2-ad3e-46f7-bb02-4dc45c1e3a62
spec:
clusterIP: 10.100.64.210
externalTrafficPolicy: Cluster
ports:
- name: http
nodePort: 31122
port: 80
protocol: TCP
targetPort: http
- name: https
nodePort: 32697
port: 443
protocol: TCP
targetPort: https
selector:
app: nginx-ingress
component: controller
release: nginx-ingress
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 10.65.106.240
EDIT 2: metallb configmap yaml
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 10.65.106.240-10.65.106.250
So, to solve the 2nd question, as I suggested, you can use hostNetwork: true parameter to map container port to the host it is running on. Note that this is not a recommended practice, and you should always avoid to do this, unless you have a reason.
Example:
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
hostNetwork: true
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
hostPort: 80 # this parameter is optional, but recommended when using host network
name: nginx
When I deploy this yaml, I can check where the pod is running and curl that host's port 80.
root#v1-16-master:~# kubectl get po -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx 1/1 Running 0 105s 10.132.0.50 v1-16-worker-2 <none> <none>
Note: now I know the pod is running on worker node 2. I just need its IP address.
root#v1-16-master:~# kubectl get no -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
v1-16-master Ready master 52d v1.16.4 10.132.0.48 xxxx Ubuntu 16.04.6 LTS 4.15.0-1052-gcp docker://19.3.5
v1-16-worker-1 Ready <none> 52d v1.16.4 10.132.0.49 xxxx Ubuntu 16.04.6 LTS 4.15.0-1052-gcp docker://19.3.5
v1-16-worker-2 Ready <none> 52d v1.16.4 10.132.0.50 xxxx Ubuntu 16.04.6 LTS 4.15.0-1052-gcp docker://19.3.5
v1-16-worker-3 Ready <none> 20d v1.16.4 10.132.0.51 xxxx Ubuntu 16.04.6 LTS 4.15.0-1052-gcp docker://19.3.5
root#v1-16-master:~# curl 10.132.0.50 2>/dev/null | grep title
<title>Welcome to nginx!</title>
root#v1-16-master:~# kubectl delete po nginx
pod "nginx" deleted
root#v1-16-master:~# curl 10.132.0.50
curl: (7) Failed to connect to 10.132.0.50 port 80: Connection refused
And of course it also works if I go to the public IP on my browser.
update:
i didn't see the edit part of the question when I was writing this answer. it doesn't make sense given the additional info provided. please disregard.
original:
apparently the cluster you are using now has its ingress controller setup over a node-port type service instead of a load-balancer. in order to get desired behavior you need to change configuration of ingress-controller. refer to nginx ingress controller documentation for metalLB cases how to do this.

Resources