Using nginx-ingress controller and metallb for my loadbalancer in my k3s raspberry pi cluster. Trying to access my blog site but I get white page with 400: Bad Request.
I'm using Cloudflare to managed my domain and SSL/TLS mode is on "Full". Created an A name "Blog" and pointed the content to my public external IP. I opened the Loadbalancer IP address on my router exposing 80 and 433. What am I missing. I've been pulling my hair with his issues for days now. Here's the example of my k3s entire deployment
apiVersion: v1
kind: Service
metadata:
namespace: nginx
name: nginx-web
labels:
app: nginx-web
spec:
ports:
# the port that this service should serve on
- port: 8000
targetPort: 80
protocol: TCP
selector:
app: nginx-web
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: nginx
labels:
app: nginx-web
name: nginx-web
spec:
replicas: 1
selector:
matchLabels:
app: nginx-web
template:
metadata:
namespace: nginx
labels:
app: nginx-web
name: nginx-web
spec:
containers:
- name: nginx-web
image: nginx:latest
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
labels:
app: nginx-web
namespace: nginx
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- hosts:
- blog.example.com
secretName: blog-example-com-tls
rules:
- host: blog.example.com
http:
paths:
- backend:
service:
name: nginx-web
port:
number: 80
path: /
pathType: Prefix
Related
I have setup in my GKE cluster an nginx ingress as follows:
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx --namespace nginx-ingress
A load balancer with its IP came up.
Now I added two DNS pointing to that domain at Cloudflare:
In addition I created a namespace app-a
kubectl create namespace app-a
kubectl label namespace app-a project=a
and deployed an app there:
apiVersion: v1
kind: Service
metadata:
name: echo1
namespace: app-a
spec:
ports:
- port: 80
targetPort: 5678
selector:
app: echo1
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: echo1
namespace: app-a
spec:
selector:
matchLabels:
app: echo1
replicas: 2
template:
metadata:
labels:
app: echo1
spec:
containers:
- name: echo1
image: hashicorp/http-echo
args:
- "-text=echo1"
ports:
- containerPort: 5678
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: echo-ingress-global
namespace: app-a
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: "test.my-domain.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: echo1
port:
number: 80
Things look good in Lens, so I thought to test it out.
When I was entering eu1.my-domain.com, I get
which is intended, of course.
but when I entered test.my-domain.com, I get that the website is unreachable: DNS_PROBE_FINISHED_NXDOMAIN, although I expected to see the dummy output of the dummy app.
Even more strangely, no matter if I get the well-responding result or the non-responding one, in the logs of the nginx controller there is nothing showing up for any of the calls.
Can you help me, such that I can access the test.my-domain.com homepage?
Ingress is not forwarding traffic to pods.
Application is deployed on Azure Internal network.
I can access app successfully using pod Ip and port but when trying Ingress IP/ Host I am getting 404 not found. I do not see any error in Ingress logs.
Bellow are my config files.
Please help me if I am missing anything or a how I can troubleshoot to find issue.
Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: aks-helloworld-one
spec:
replicas: 1
selector:
matchLabels:
app: aks-helloworld-one
template:
metadata:
labels:
app: aks-helloworld-one
spec:
containers:
- name: aks-helloworld-one
image: <image>
ports:
- containerPort: 8290
protocol: "TCP"
env:
- name: env1
valueFrom:
secretKeyRef:
name: configs
key: env1
volumeMounts:
- mountPath: "mnt/secrets-store"
name: secrets-mount
volumes:
- name: secrets-mount
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "azure-keyvault"
imagePullSecrets:
- name: acr-secret
---
apiVersion: v1
kind: Service
metadata:
name: aks-helloworld-one
spec:
type: ClusterIP
ports:
- name: http
protocol: TCP
port: 8080
targetPort: 8290
selector:
app: aks-helloworld-one
Ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ingress
namespace: ingress-basic
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: aks-helloworld
port:
number: 80
Correct your service name and service port in ingress.yaml.
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
# wrong: name: aks-helloworld
name: aks-helloworld-one
port:
# wrong: number: 80
number: 8080
Actually, you can use below command to confirm if ingress has any endpoint.
kubectl describe ingress hello-world-ingress -n ingress-basic
You have mentioned the wrong service name under the ingress definition. Service name should be aks-helloworld-one as per the service definition.
I would like to have a script that produces Kubernetes manifests to deploy a bare nginx container with service port 80 and ingress for host as example.nginx.com. I will deploy it into EKS cluster. Can someone give me clue?
Recently I started configuring NLB with Nginx controller on EKS. So documenting the complete flow with the script you needed.
I tried other approached like cloud provider based Nginx deployment but it didn't work as expected ( instead of ELB it was creating Classic LB).
Ref- https://github.com/kubernetes/ingress-nginx/issues/6292
In short below approach is the best so far.
Install Nginx controller- This will create a deployment and a NodePort service for HTTP say port- 31848, HTTPS- 30099
#kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.1/deploy/static/provider/baremetal/deploy.yaml
Create Production deployment, service and ingress resource.
apiVersion: apps/v1
kind: Deployment
metadata:
name: production
labels:
app: production
namespace: app
spec:
replicas: 1
selector:
matchLabels:
app: production
template:
metadata:
labels:
app: production
spec:
containers:
- name: production
image: mirrorgooglecontainers/echoserver:1.10
ports:
- containerPort: 8080
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
---
apiVersion: v1
kind: Service
metadata:
name: production
labels:
app: production
namespace: app
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: production
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: production
annotations:
kubernetes.io/ingress.class: nginx
namespace: app
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
serviceName: production
servicePort: 80
Create Canary Deployment.
apiVersion: apps/v1
kind: Deployment
metadata:
name: canary
labels:
app: canary
namespace: app
spec:
replicas: 1
selector:
matchLabels:
app: canary
template:
metadata:
labels:
app: canary
spec:
containers:
- name: canary
image: mirrorgooglecontainers/echoserver:1.10
ports:
- containerPort: 8080
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
---
apiVersion: v1
kind: Service
metadata:
name: canary
labels:
app: canary
namespace: app
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: canary
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: canary
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "30"
namespace: app
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
serviceName: canary
servicePort: 80
Create a NLB type Load Balancer on EKS. If you are choosing "internet-facing".
Create a Target Group with "Target Type" as instance and Port/Health Check Port- 31848 (HTTP).
Attach Target Group to Autoscaling group.
Create a listener on NLB (TLS- Secure TCP) and forward it to the Target Group.
Although we would be launching worker nodes on Private subnets but we need to open port "31848" for all the IP. This is how EC2 would be able to communicate with NLB.
Hope I am able to provide you clear idea on this. Please do let me know in case you face any issue.
You must have deployed nginx-ingress to your cluster. Then run the following script at your command prompt to deploy a bare nginx container with service port 80 and ingress for host as example.nginx.com:
cat << EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
selector:
app: nginx
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: example.nginx.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx
port:
number: 80
EOF
I'm trying to follow this suggestion to use Hetzner load balancer as a Nginx ingress controller.
helm install ingress-nginx with these configurations:
controller:
config:
use-proxy-protocol: "true"
replicaCount: 3
service:
type: LoadBalancer
annotations:
load-balancer.hetzner.cloud/name: nginx-controller-new
load-balancer.hetzner.cloud/location: hel1
load-balancer.hetzner.cloud/use-private-ip: true
load-balancer.hetzner.cloud/algorithm-type: least_connections
load-balancer.hetzner.cloud/uses-proxyprotocol: true
load-balancer.hetzner.cloud/hostname: somehost.com
Deployments:
apiVersion: apps/v1
kind: Deployment
metadata:
name: echo1
spec:
selector:
matchLabels:
app: echo1
replicas: 3
template:
metadata:
labels:
app: echo1
spec:
containers:
- name: echo1
image: hashicorp/http-echo
args:
- "-text=echo1"
ports:
- containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
name: echo-service
spec:
selector:
app: echo1
ports:
- name: http
protocol: TCP
port: 80
targetPort: 5678
ipFamilyPolicy: PreferDualStack
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx
spec:
ingressClassName: nginx
rules:
- http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: echo-service
port:
number: 80
host: somehost.com
After installation, a Hetzner load balancer is successfully provisioned, however, it isn't able to detect the services:
I'm at a loss here. How can I connect the echo1 app to the ingress-nginx-controller service? I check out all of the available helm values but I cannot find something like service.selector to target echo1's service and make it publicly available. Can someone help me? Are there any alternatives?
I am not the Kubernetes master (more a noob) but I got it working with a L4-loadbalancer.
An annotation has to be set to your Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
And to use it without nginx-ingress, that worked for me (not tested with your labels)
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
name: "nginx-hello-world"
labels:
app: "hello-world"
spec:
selector:
matchLabels:
app: "hello-world"
strategy:
type: "Recreate"
template:
metadata:
labels:
app: "hello-world"
spec:
containers:
- image: "rancher/hello-world"
name: "nginx-hello-world"
imagePullPolicy: "Always"
ports:
- containerPort: 80
name: "http"
Service
for HTTP (to test your deployment)
---
apiVersion: "v1"
kind: Service
metadata:
name: "nginx-hello-world"
labels:
app: "hello-world"
annotations:
load-balancer.hetzner.cloud/name: lb-development
load-balancer.hetzner.cloud/hostname: somehost.com
load-balancer.hetzner.cloud/protocol: http
load-balancer.hetzner.cloud/health-check-port: 10254
spec:
type: LoadBalancer
selector:
app: "hello-world"
ports:
- name: "http"
port: 80
targetPort: 80
for SSL
apiVersion: v1
kind: Service
metadata:
name: nginx-hello-world
labels:
app: hello-world
annotations:
load-balancer.hetzner.cloud/hostname: somehost.com
load-balancer.hetzner.cloud/http-certificates: managed-certificate-1-wildcard-somehost.com
load-balancer.hetzner.cloud/name: lb-development
load-balancer.hetzner.cloud/protocol: https
spec:
ports:
- name: https
nodePort: 32725
port: 443
protocol: TCP
targetPort: 80
selector:
app: hello-world
type: LoadBalancer
I have an issue I can't figure out. I have setup Nginx Ingress Controller on my managed k8s cluster. I'm trying to reach an SSL enabled pod behind and it does not work. I have 404 not found from Nginx and the certificate which is presented is the Nginx one. I have deployed the controller using their github repo and the default files following their doc.
I have setup a clear http pod for purpose tests and it works. It seems to be related to ssl.
I have tried many things to no avail. How can I reach an SSL pod behind nginx ?
Here's the Deployment + service (for the https one) resource I have setup :
apiVersion: apps/v1
kind: Deployment
metadata:
name: moulip-https
spec:
selector:
matchLabels:
app: moulip-https
replicas: 2
template:
metadata:
labels:
app: moulip-https
spec:
containers:
- name: "wabam"
image: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
ports:
- containerPort: 443
imagePullSecrets:
- name: regcrd
---
apiVersion: v1
kind: Service
metadata:
name: https-svc
labels:
app: moulip-https
spec:
ports:
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: moulip-https
and my Ingress :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/rewrite-target: /
namespace: default
spec:
rules:
- host: https.moulip.lan
http:
paths:
- backend:
serviceName: https-svc
servicePort: 443
- host: test.moulip.lan
http:
paths:
- backend:
serviceName: hostname-svc
servicePort: 80
Many thanks for any guidance you could provide me with.
You are missing tls configuration in the ingress. follow sample below
apiVersion: v1
kind: Secret
metadata:
name: testsecret-tls
namespace: default
data:
tls.crt: base64 encoded cert
tls.key: base64 encoded key
type: kubernetes.io/tls
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: tls-example-ingress
spec:
tls:
- hosts:
- sslexample.foo.com
secretName: testsecret-tls
rules:
- host: sslexample.foo.com
http:
paths:
- path: /
backend:
serviceName: service1
servicePort: 80