why nginx ingress controller is deployed as a container(pod)? - nginx

my natural thought is that if nginx is just a daemon process on the k8s node, but not a pod(container) in the k8s cluster, looks like it still can fullfill ingress controller jobs. because:
if it's a process, because it is on the k8s node, it still can talk to apiserver to fetch service backend pods information, like IP addresses, so it's still can be used as a http proxy server to direct traffic to different services.
so 2 questions,
why nginx ingress controller has to be a pod?
why nginx ingress controller only got 1 replica? and on which node? if nginx controller pod is dead, things will go unstable.
Thanks!

why Nginx ingress controller has to be a pod?
it is possible to run the Nginx controller as a daemon set in Kubernetes however I am not sure about the running on the node.
Manging the POD using daemon set and deployment of Kubernetes easy compare to process on Node.
By default Nginx daemon process is not part of any Kubernetes node, if you cluster autoscale will you install the Nginx process manually on Node?
If you thinking to create own AMI with Nginx process inside and use it inside the Node pool and scale that pool, it's possible but how about OS patching and maintenance ?
why nginx ingress controller only got 1 replica? and on which node? if
nginx controller pod is dead, things will go unstable.
Running replicas 1 is the default configuration but you can implement the HPA and increase the replicas as per need. Nginx is lightweight so handling a large volume of traffic not require more replicas.
Still, as per need, you can run multiple replicas with HPA or increase manually replicas to get high availability.

Because Pods are how you run daemon processes (or really, all processes) inside Kubernetes. That's just how you run stuff. I suppose there is nothing stopping you from running it outside the cluster, manually setting up API configuration and authentication, doing all the needed networking bits yourself. But ... why?
As for replicas, you should indeed generally have more than one across multiple physical nodes for redundancy. A lot of the tutorials show it with replicas: 1 because either it's for a single-node dev cluster like Minikube or it's only an example.

Related

Kubernetes - Is Nginx Ingress, Proxy part of k8s core?

I understand there are various ways to get external traffic into the cluster - Ingress, cluster IP, node port and load balancer. I am particularly looking into the Ingress and k8s and from the documentation k8s supports AKS, EKS & Nginx controllers.
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
To implement Ingress, understand that we need to configure an Ingress Controller in the cluster. My query is whether Nginx ingress & proxy are an offering of core k8s (packaged / embedded)? Might have overlooked, did not find any documentation where it is mentioned. Any insight or pointer to documentation if stated above is true is highly appreciated.
Just reading the first rows of the page you linked, it states that no controller are started automatically with a cluster and that you must choose the one of your preference, depending on your requirements
Ingress controllers are not started automatically with a cluster. Use
this page to choose the ingress controller implementation that best
fits your cluster.
Kubernetes defines Ingress, IngressClass and other ingress-related resources but a fresh installation does not come with any default.
Some prepackaged installation of Kubernetes (like microk8s, minikube etc...) comes with ingress controller that, usually, needs to be enabled manually during the installation/configuration phase.

Kubernetes on-premise Ingress traffic policy local

Kubernetes installed on premise,
nginx-ingress
a service with multiple pods on multiple nodes
All this nodes are working as an nginx ingress.
The problem is when a request come from a load balancer can jump to another worker that have a pod, this cause unecesary trafic inside the workers network, I want to force when a request come from outside to the ingress,
the ingress always choice pods on the same node, in case no pods then
can forward to other nodes.
More or less this image represent my case.
example
I have the problem in the blue case, what I expect is the red case.
I saw exist the "externalTrafficPolicy: Local" but this only work for
serviceType nodePort/loadBalancer, nginx ingress try to connect using the "clusterIP" so it skips this functionality.
There are a way to have this feature working for clusterIP or something similar? I started to read about istio and linkerd, they seem so powerful but I don't see any parameter to configure this workflow.
You have to deploy an Ingress Controller using a NodeSelector to deploy it to specific nodes, named ingress or whatever you want: so you can proceed to create an LB on these node IPs using simple health-checking on port 80 and 443 (just to update the zone in case of node failure) or, even better, with a custom health-check endpoint.
As you said, the externalTrafficPolicy=Local works only for Load-Balancer services: dealing with on-prem clusters is tough :)

Should I run nginx in every Kubernetes pod?

I have a kubernetes cluster with 20 worker nodes. My main application is a Flask API that serves thousands of android/ios requests per minute. The way my Kubernetes deployment is configured is that each pod has 2 containers - flask/python server and nginx. The flask app runs on-top of gunicorn with meinheld workers (20 workers per pod).
My question is: do I need to be running nginx in each of the pods alongside the flask app or can I just use a main nginx ingress controller as a proxy buffering layer?
NOTE:
I am using ELB to route external traffic to my internal k8s cluster.
Is not too strange to have a proxy on every pod, in fact, istio injects one envoy container per pod as a proxy to control de ingress and egress traffic and also to having more accurate metrics.
Check de documentation https://istio.io/
But if you don't want to manage a service mesh by the moment you can avoid the nginx and use directly the port mapping on the services an ingress definition.
I don't see any reason to have a nginx container for every other flask container. You can have one nginx container as API gateway to your entire set of apis

Can I run Nginx outside a kuberntes cluster and have it discover ingresses?

I have a bare metal cluster. Right now I have traefik running on a machine, outside the cluster, playing reverse proxy. Basically it just points its backend to the cluster api-server. It will discover ingresses and works fine. But it doesn't support TCP :-(
Can I do the same with Nginx? Is there a way to run it outside the cluster and point its backend to the kubernetes api-server to discover ingresses?
So what I did was run the ingress controller inside the cluster and tie it to a node which has the needed DNS entries pointing towards it with HostNetwork.

Traefik instance loadbalance to Kubernetes NodePort services

Intro:
On AWS, Loadbalancers are expensive ($20/month + usage), so I'm looking for a way to achieve flexible load-balancing between the k8s nodes, without having to pay that expense. The load is not that big, so I don't need the scalability of the AWS load balancer any time soon. I just need services to be HA. I can get a small EC2 instance for $3.5/month that can easily handle the current traffic, so I'm chasing that option now.
Current setup
Currently, I've set up a regular standalone Nginx instance (outside of k8s) that does load balancing between the nodes in my cluster, on which all services are set up to expose through NodePorts. This works really well, but whenever my cluster topology changes during restarts, adding, restarting or removing nodes, I have to manually update the upstream config on the Nginx instance, which is far from optimal, given that cluster nodes cannot be expected to stay around forever.
So the question is:
Can Trækfik be set up outside of Kubernetes to do simple load-balancing between the Kubernetes nodes, just like my Nginx setup, but keep the upstream/backend servers of the traefik config in sync with Kubernetes list of nodes, such that my Kubernetes services are still HA when I make changes to my node setup? All I really need is for Træfik to listen to the Kubernetes API and change the backend servers whenever the cluster changes.
Sounds simple, right? ;-)
When looking at the Træfik documentation, it seems to want an ingress resource to send its trafik to, and an ingress resource requires an ingress controller, which I guess, requires a load balancer to become accessible? Doesn't that defeat the purpose, or is there something I'm missing?
Here is something what would be useful in your case https://github.com/unibet/ext_nginx but I'm note sure if project is still in development and configuration is probably hard as you need to allow external ingress to access internal k8s network.
Maybe you can try to do that on AWS level? You can add cron job on Nginx EC2 instance where you query AWS using CLI for all EC2 instances tagged as "k8s" and make update in nginx configuration if something changed.

Resources