I have a kubernetes setup with the configuration like below:
I am using this mandatory file:
https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.26.1/deploy/static/mandatory.yaml
https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.26.1/deploy/static/provider/cloud-generic.yaml
My ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: api-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- api.service.com
secretName: api-tls
rules:
- host: api.service.com
http:
paths:
- backend:
serviceName: api-service
servicePort: 80
My service:
#########################################################
# Service for API Gateway service
#########################################################
apiVersion: v1
kind: Service
metadata:
name: api-service
labels:
name: api
spec:
selector:
app: api
ports:
- name: http
port: 80
targetPort: 3000
nodePort: 30000
protocol: TCP
- name: https
port: 443
targetPort: 3000
nodePort: 30001
protocol: TCP
type: NodePort
sessionAffinity: ClientIP
My deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: api
name: api
spec:
replicas: 1
selector:
matchLabels:
name: api
template:
metadata:
labels:
name: api
app: api
spec:
containers:
- env:
- name: CACHER
value: redis://redis:6379
- name: LOGLEVEL
value: info
- name: NAMESPACE
value: myName
- name: PORT
value: "3000"
- name: SERVICEDIR
value: services
- name: SERVICES
value: api
- name: TRANSPORTER
value: nats://nats:4222
ports:
- containerPort: 3000
image: registry.digitalocean.com/my-registry/my-image:latest
imagePullPolicy: ""
name: api
resources: {}
imagePullSecrets:
- name: my-registry
restartPolicy: Always
serviceAccountName: ""
volumes: null
status: {}
If I use Service NodePort with port 30001 and its own IP, I don't have any problem, but with LoadBalancer always throws a 502 Bad gateway.
Any idea?
Thanks!
Please, avoid using these files manually. This file seems outdated too. Use helm if you don't like surprises. Because these are managed services.
First, install Helm on your laptop. Then log in to your Digitalocean in the command panel. Delete existing Nginx ingress implementations. Then run these commands one by one.
At first add the ingress controller to the default namespace
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
Then update the helm repo
helm repo update
Then finally run this command
helm install nginx-ingress ingress-nginx/ingress-nginx --set controller.publishService.enabled=true
To check the installation run this command
kubectl --namespace default get services -o wide -w nginx-ingress-ingress-nginx-controller
There is also a digital ocean recommended approach. You can use Digital Ocean Marketplace to install Nginx-Ingress too. Digitalocean will then automatically run these aforementioned commands for you! If you check their Github account, you will find that they are also using helm for their marketplace services. It's time to adopt Helm.
Related
I am experiencing exactly this issue: Nginx-ingress-controller fails to start after AKS upgrade to v1.22, with the exception that none of the proposed solutions is working for my case.
I am running a Kubernetes Cluster on Oracle Cloud and I accidentally upgraded the cluster and now I cannot connect anymore to the services through nginx-controller. After reading the official nginx documentation, I am aware of the new version of nginx, so I checked the documentation and re-installed the nginx-controller following Oracle Cloud official documentation.
I am able to perform step by step as I run:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/cloud/deploy.yaml
And then an ingress-nginx namespace is created and a LoadBalancer is created. Then as in the guide I have created a simple hello application (though not running on port 80):
apiVersion: apps/v1
kind: Deployment
metadata:
name: docker-hello-world
labels:
app: docker-hello-world
spec:
selector:
matchLabels:
app: docker-hello-world
replicas: 1
template:
metadata:
labels:
app: docker-hello-world
spec:
containers:
- name: docker-hello-world
image: scottsbaldwin/docker-hello-world:latest
ports:
- containerPort: 8088
---
apiVersion: v1
kind: Service
metadata:
name: docker-hello-world-svc
spec:
selector:
app: docker-hello-world
ports:
- port: 8088
targetPort: 8088
type: ClusterIP
and then the ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ing
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- secretName: tls-secret
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: docker-hello-world-svc
port:
number: 8088
But when running the curl commands I only get a curl: (56) Recv failure: Connection reset by peer.
So I then tried to connect to some python microservices that are already running by simply editing the ingress, but whatever I do I get the same error message. And when setting the host as the following:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ing
namespace: ingress-nginx
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: SUBDOMAIN.DOMAIN.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: ANY_MICROSERVICE_RUNNING_IN_CLUSTER
port:
number: EXPOSED_PORT_BY_MICROSERVICE
Then, by setting the subdomain on CloudFlare I only get a 520 Bad Gateway.
Can you help me find what is that I do not see?
This may be related to your Ingress resource.
In Kubernetes versions v1.19 and above, Ingress resources should use ingressClassName instead of the older annotation syntax. Additional information on what should be done when upgrading can be found on the official Kubernetes documentation.
However, with the changes it requires at face value, from the information you're provided so far, your Ingress resource should look this:
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ing
spec:
ingressClassName: nginx
rules:
- host: SUBDOMAIN.DOMAIN.com
http:
paths:
- backend:
service:
name: docker-hello-world-svc
port:
number: 8088
path: /
pathType: Prefix
tls:
- hosts:
- SUBDOMAIN.DOMAIN.com
secretName: tls-secret
Additionally, please provide the deployment Nginx-ingress logs if you still have issues, as the Cloudflare error does not detail what could be wrong apart from providing a starting point.
As someone who uses Cloudflare and Nginx, there are multiple reasons why you're receiving a 520 error, so it'd be better if we could reduce the scope of what could be the main issue. Let me know if you have any questions.
I have deployed my API to Kubernetes on AKS through kubectl command from my local machine. But the nginx ingress is not able to resolve the backend endpoint. The ingress logs has an error The service 'hello-world/filter-api' does not have any active endpoint
Steps followed:
Install dapr on AKS
dapr init -k --set global.tag=1.1.2
Install nginx ingress on AKS
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx -f ...\dapr\components\dapr-annotations.yaml --set image.tag=1.11.1 -n ingress-nginx
Apply manifest
kubectl apply -f .\services\filter.yaml
What did I try?
Verified the selectors and labels
Followed the steps mentioned Troublshooting nginx ingress
I tried to deploy this to local Kubernetes cluster on windows with docker desktop. This works fine. What am I missing?
filter.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: filter-cm
namespace: hello-world
labels:
app: hello-world
service: filter
data:
ASPNETCORE_ENVIRONMENT: Development
ASPNETCORE_URLS: http://0.0.0.0:80
PATH_BASE: /filter
PORT: "80"
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: filter
namespace: hello-world
labels:
app: hello-world
service: filter
spec:
replicas: 1
selector:
matchLabels:
service: filter
template:
metadata:
labels:
app: hello-world
service: filter
annotations:
dapr.io/enabled: "true"
dapr.io/app-id: "filter-api"
dapr.io/app-port: "80"
dapr.io/config: "dapr-config"
spec:
containers:
- name: filter-api
image: client/hello-world-filter-api:0.0.1
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
protocol: TCP
envFrom:
- configMapRef:
name: filter-cm
imagePullSecrets:
- name: regcred
---
apiVersion: v1
kind: Service
metadata:
name: filter-api
namespace: hello-world
labels:
app: hello-world
service: filter
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 30001
protocol: TCP
name: http
selector:
service: filter
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: filter-ingress
namespace: hello-world
labels:
app: hello-world
service: filter
spec:
rules:
- http:
paths:
- path: /filter
pathType: Prefix
backend:
service:
name: filter-api
port:
number: 80
In the service selector, use matchLabels for the service to find the backend pods
example:
selector:
matchLabels:
service: filter
I am running a simple pod with an image from local image registry in a minikube cluster on Windows 10. I am also running a simple nodeport service. The container is available when I try accessing it from the browser with <minikube_ip>:30080.
However, now I want to set an ingress controller because I want to set up a domain and not access it using the IP. The ingress works for something simple like a basic nginx pod, but does not work for this pod that I'm trying to use. I was previously using jwilder/nginx-proxy in docker-compose and it had some conf files that needed to be attached in the conf.d directory. However, since I am moving to Kubernetes, I thought to totally omit the conf files and the reverse proxy image.
Now after the hosts fie is updated, the domain is reachable via curl, the domain is also pingable, however, it simply cannot be reached on the browser.
pod-yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
io.kompose.service: api
name: api
spec:
replicas: 1
selector:
matchLabels:
io.kompose.service: api
strategy:
type: RollingUpdate
template:
metadata:
labels:
io.kompose.service: api
spec:
containers:
- env:
- name: DEV_PORT
value: "80"
image: localhost:5000/api:2.3
imagePullPolicy: "IfNotPresent"
name: api
resources: {}
restartPolicy: Always
serviceAccountName: ""
status: {}
Service.yaml
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: C:\Users\***kompose.exe
convert
kompose.version: 1.21.0 (992df58d8)
creationTimestamp: null
labels:
io.kompose.service: api
name: api
spec:
selector:
io.kompose.service: api
type: NodePort
ports:
- name: "http"
port: 80
targetPort: 80
nodePort: 30080
Ingress.yaml
apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
kind: Ingress
metadata:
name: tls-ingress
spec:
tls:
- secretName: oaky-tls
hosts:
- api.localhost
rules:
- host: api.localhost
http:
paths:
- path: /
backend:
serviceName: api
servicePort: 80
I have checked and the TLS secret is available, I am not understanding the issue here and would really appreciate some help.
Solved:
Chrome was overlooking the etc hosts file, so I did the following:
Switched to Firefox and instantly the URLs were working.
Added annotations to denote the class:
kubernetes.io/ingress.class: nginx
Added annotations to make sure requests are redirected to ssl
nginx.ingress.kubernetes.io/ssl-redirect: "true"
I have 2 nodes on GCP in a kubernetes cluster. I also have a load balancer in GCP as well. this is a regular cluster (not GCK). I am trying to expose my front-end-service to the world. I am trying nginx-ingress type:nodePort as a solution. Where should my loadbalancer be pointing to? is this a good architecture approach?
world --> GCP-LB --> nginx-ingress-resource(GCP k8s cluster) --> services(pods)
to access my site I would have to point LB to worker-node-IP where nginx pod is running. Is this bad practice. I am new in this subject and trying to understand.
Thank you
deployservice:
apiVersion: v1
kind: Service
metadata:
name: mycha-service
labels:
run: mycha-app
spec:
ports:
- port: 80
targetPort: 3000
protocol: TCP
selector:
app: mycha-app
nginxservice:
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress
labels:
app: nginx-ingress
spec:
type: NodePort
ports:
- nodePort: 31000
port: 80
targetPort: 80
protocol: TCP
name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
name: nginx-ingress
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress
labels:
run: nginx-ingress
spec:
type: NodePort
ports:
- nodePort: 31000
port: 80
targetPort: 3000
protocol: TCP
selector:
app: nginx-ingress
nginx-resource:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: mycha-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: mycha-service
servicePort: 80
This configuration is not working.
When you use ingress in-front of your workload pods the service type for workload pods will always be of type clusterIP because you are not exposing pods directly outside the cluster.
But you need to expose the ingress controller outside the cluster either using NodePort type service or using Load Balancer type service and for production its recommended to use Loadbalancer type service.
This is the recommended pattern.
Client -> LoadBalancer -> Ingress Controller -> Kubernetes Pods
Ingress controller avoids usage of kube-proxy and load balancing provided by kube-proxy. You can configure layer 7 load balancing in the ingress itself.
The best practise of exposing application is:
World > LoadBalancer/NodePort (for connecting to the cluster) > Ingress (Mostly to redirect traffic) > Service
If you are using Google Cloud Platform, I would use GKE as it is optimized for containers and configure many things automatically for you.
Regarding your issue, I also couldn't obtain IP address for LB <Pending> state, however you can expose your application using NodePort and VMs IP. I will try few other config to obtain ExternalIP and will edit answer.
Below is one of examples how to expose your app using Kubeadm on GCE.
On GCE, your VM already have ExternalIP. This way you can just use Service with NodePort and Ingress to redirect traffic to proper services.
Deploy Nginx Ingress using Helm 3 as tiller is not required anymore ($ helm install nginx stable/nginx-ingress).
As Default it will deploy service with LoadBalancer type but it won't get externalIP and it will stuck in <Pending> state. You have to change it to NodePort and apply changes.
$ kubectl edit svc nginx-nginx-ingress-controller
Default it will open Vi editor. If you want other you need to specify it
$ KUBE_EDITOR="nano" kubectl edit svc nginx-nginx-ingress-controller
Now you can deploy service, deployment and ingress.
apiVersion: v1
kind: Service
metadata:
name: fs
spec:
selector:
key: app
ports:
- port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: fd
spec:
replicas: 1
selector:
matchLabels:
key: app
template:
metadata:
labels:
key: app
spec:
containers:
- name: hello1
image: gcr.io/google-samples/hello-app:1.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mycha-deploy
labels:
app: mycha-app
spec:
replicas: 1
selector:
matchLabels:
app: mycha-app
template:
metadata:
labels:
app: mycha-app
spec:
containers:
- name: mycha-container
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: mycha-service
labels:
app: mycha-app
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: mycha-app
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: my.pod.svc
http:
paths:
- path: /mycha
backend:
serviceName: mycha-service
servicePort: 80
- path: /hello
backend:
serviceName: fs
servicePort: 80
service/fs created
deployment.apps/fd created
deployment.apps/mycha-deploy created
service/mycha-service created
ingress.extensions/two-svc-ingress created
$ kubectl get svc nginx-nginx-ingress-controller
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-nginx-ingress-controller NodePort 10.105.247.148 <none> 80:31143/TCP,443:32224/TCP 97m
Now you should use your VM ExternalIP (slave VM) with port from NodePort service. My VM ExternalIP: 35.228.133.12, service: 80:31143/TCP,443:32224/TCP
IMPORTANT
If you would curl your VM with port you would get response:
$ curl 35.228.235.99:31143
curl: (7) Failed to connect to 35.228.235.99 port 31143: Connection timed out
As you are doing this manually, you also need add Firewall rule to allow traffic from outside on this specific port or range.
Information about creation of Firewall Rules can be found here.
If you will set proper values (open ports, set IP range (0.0.0.0/0), etc) you will be able to get service from you machine.
Curl from my local machine:
$ curl -H "HOST: my.pod.svc" http://35.228.235.99:31143/mycha
<!DOCTYPE html>
...
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
$ curl -H "HOST: my.pod.svc" http://35.228.235.99:31143/hello
Hello, world!
Version: 1.0.0
Hostname: fd-c6d79cdf8-dq2d6
I encountered a problem when integrating K8S nginx ingress. I installed the nginx ingress controller and established the testing ingress resources according to the instructions on the document, but I was not able to jump to the normal path. The test serive was normal and Accessible via cluster IP. Am I missing something?
Install script
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
Bare-metal Using NodePort
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
Ingress controller is OK
Testing ingress resource
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
app: my-nginx
template:
metadata:
labels:
app: my-nginx
spec:
containers:
- name: my-nginx
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
app: my-nginx
spec:
ports:
- port: 80
protocol: TCP
name: http
selector:
app: my-nginx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-nginx
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: nginx1.beencoding.com
http:
paths:
- path: /
backend:
serviceName: nginx-1
servicePort: 80
We can see the test nginx pod raised and works fine, I can access the nginx index page by cluster IP
But I can't access nginx1.beencoding.com
Can't access via browser
I have solved the problem by setting hostnetwork: true
It says can't resolve.
Either put the domain in /etc/hosts/ file, or do the curl as follows:
curl -H "Host: nginx1.beecoding.com" IP_ADDRESS
Should work.