I understand there are various ways to get external traffic into the cluster - Ingress, cluster IP, node port and load balancer. I am particularly looking into the Ingress and k8s and from the documentation k8s supports AKS, EKS & Nginx controllers.
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
To implement Ingress, understand that we need to configure an Ingress Controller in the cluster. My query is whether Nginx ingress & proxy are an offering of core k8s (packaged / embedded)? Might have overlooked, did not find any documentation where it is mentioned. Any insight or pointer to documentation if stated above is true is highly appreciated.
Just reading the first rows of the page you linked, it states that no controller are started automatically with a cluster and that you must choose the one of your preference, depending on your requirements
Ingress controllers are not started automatically with a cluster. Use
this page to choose the ingress controller implementation that best
fits your cluster.
Kubernetes defines Ingress, IngressClass and other ingress-related resources but a fresh installation does not come with any default.
Some prepackaged installation of Kubernetes (like microk8s, minikube etc...) comes with ingress controller that, usually, needs to be enabled manually during the installation/configuration phase.
Related
So if I have 10 services that I need to expose to the outside world and use path-based routing to connect to different services, I can create an Nginx pod and service type LoadBalancer
I can then create Nginx configurations and can redirect to different services depending upon the URL path. After exploring more, I came to know about Nginx ingress which can also do the same. What is the difference between the two approaches and which approach is better?
In both cases, you are running an Nginx reverse proxy in a Kubernetes pod inside the cluster. There is not much technical difference between them.
If you run the proxy yourself, you have complete control over the Nginx version and the configuration, which could be desirable if you have very specific needs and you are already an Nginx expert. The most significant downside is that you need to manually reconfigure and redeploy the central proxy if you add or change a service.
If you use Kubernetes ingress, the cluster maintains the Nginx configuration for you, and it regenerates it based on Ingress objects that can be deployed per-service. This is easier to maintain if you are not an Nginx expert, and you can add and remove services without touching the centralized configuration.
The Kubernetes ingress system in principle can also plug in alternate proxies; it is not limited to Nginx. However, its available configuration is somewhat limited, and some of the other proxies I've looked at recommend using their own custom Kubernetes resources instead of the standard Ingress objects.
I am a kubernetes newbie, and no matter how much I read about it I cannot get my head around this issue.
I have a simple deployment , which is creating a pod with a not so complex app.
I know what an ingress and an ingress controller is doing ,but as far as I understand it is not required for me to expose my pod-app externally. Only a LoadBalancer service should be enough.
I do not have need of more than one rule for traffic routing.
Am I wrong about that?
Traditionally, you would create a LoadBalancer service for each service you want to expose externally. This can get rather expensive. Ingress gives you a way to route requests to services based on the request host or path, centralizing a number of services into a single entrypoint.
Also load balancer provisioning takes time and works only in supported cloud providers such as AWS, GCP etc.
One more thing to consider is the need of L4(TCP/UDP) layer routing because kubernetes Ingress API is primarily L7 layer but some of the ingress controllers such as traefik, nginx supports L4 layer(TCP/UDP) along with L7 layer(HTTP) routing.
So answer to your question is it depends based on your environment and use cases.
Ingress and IngressControllers are used for traffic routing at layer 7 i.e., if your backends talk L7 protocols like HTTP, GRPC etc. You can route requests based on request path to different backend services using Ingress.
If your app doesn't operate at Layer 7, you might not need an Ingress.
Another question you could ask yourself if your migrating your app from non-kubernetes environment to kuberneters is - are you using a reverse proxy like nginx already? If so, you might want to use Ingress. I say might because it is not necessary. You can achieve the same affect of using an Ingress by running the nginx container as a pod, writing the nginx.conf yourself and making it available externally(using a LoadBalancer service for example). Instead by using IngressController you need not maintain the nginx pod or write nginx.conf. You can instead express the same configuration as Ingress resource which is much simpler.
Ingress is needed if you want to expose your service to external. Especially for layer 7 in OSI model (HTTP transport). Ingress also provide the mechanism to enable TLS support in your load balancer. Traffic routing is controlled by rules defined on the Ingress resource. An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
By default Ingress controller depends on which kind of cloud provider you're using, or if you're using on premise you'll need to set it up based on what you need. In one cluster, you can create multiple Ingress controller as well. There's many kinds of Ingress controller, you can take a look on this article.
I think this article about Load Balancer and Ingress may also help.
I'm currently learning about Kubernetes. While the abstractions are great I'd like to understand what is actually happening under the hood with Ingress set up.
In a cloud context and using nginx ingress controller, following an external request from the load balancer to a pod, this is what I believe is happening:
The request arrives at the load balancer and, using some balancing algorithm, it selects an ingress controller. Many instances of the ingress controller will likely be running in a resilient production environment.
The ingress controller (nginx in this case) uses the rules exposed by the ingress service to select a node port, selecting a specific node to route to. Is there any load balancing happening by nginx here?
The kubelet on the node receives this request. Depending on the set up (iptables vs ipvs) this request will be further load balanced and using the clusterip a specific pod will be selected to route to. Can this pod could exist anywhere on the cluster, on a different node to the kubelet routing it?
The request is then forwarded to a specific pod and container.
Is this understanding correct?
You should check out this guide about Kubernetes Ingress with Nginx Example. Especially the Kubernetes Ingress vs LoadBalancer vs NodePort part. The examples there should describe how it goes. The clue here is how K8S SVC NodePort works. You should also check out this guide.
Also if you are new to Kubernetes I strongly recommend to get familiar with the official documentation
here
here
and here
Please let me know if that helped.
I finally found a 3 part article which went into enough detail to demystify how the networking works in Kubernetes.
https://medium.com/google-cloud/understanding-kubernetes-networking-pods-7117dd28727
Kubernetes installed on premise,
nginx-ingress
a service with multiple pods on multiple nodes
All this nodes are working as an nginx ingress.
The problem is when a request come from a load balancer can jump to another worker that have a pod, this cause unecesary trafic inside the workers network, I want to force when a request come from outside to the ingress,
the ingress always choice pods on the same node, in case no pods then
can forward to other nodes.
More or less this image represent my case.
example
I have the problem in the blue case, what I expect is the red case.
I saw exist the "externalTrafficPolicy: Local" but this only work for
serviceType nodePort/loadBalancer, nginx ingress try to connect using the "clusterIP" so it skips this functionality.
There are a way to have this feature working for clusterIP or something similar? I started to read about istio and linkerd, they seem so powerful but I don't see any parameter to configure this workflow.
You have to deploy an Ingress Controller using a NodeSelector to deploy it to specific nodes, named ingress or whatever you want: so you can proceed to create an LB on these node IPs using simple health-checking on port 80 and 443 (just to update the zone in case of node failure) or, even better, with a custom health-check endpoint.
As you said, the externalTrafficPolicy=Local works only for Load-Balancer services: dealing with on-prem clusters is tough :)
jwilder/nginx-proxy has 1.3K STARS and 10M+ PULLS on Docker Hub. And Watch 262, Star 7701, Fork 1546 on GitHub.
https://github.com/jwilder/nginx-proxy
kubernetes/ingress-nginx has 13 stars on kubeapps.com (one of the most starred charts) and Watch 137, Star 1596, Fork 918 on GitHub.
https://github.com/kubernetes/ingress-nginx
What's the difference between the two?
When would you use one over the other?
That is 2 different applications, but both are based on Nginx and have the similar function.
Nginx-proxy by jwilder is a proxy server for Docker containers which includes docker-gen to generate a configuration for Nginx automatically. You can use it for SSL termination, load balancing etc. But it will be hard to manage nginx-proxy in Kubernetes.
Ingress-nginx by Kubernetes is Ingress Controller which provides Ingress functional for your Kubernetes cluster. It also can do SSL termination and some other things, but it was created especially for use in Kubernetes, and it's abstractions. That means you can create the Ingress object which includes Services as backends and use selectors etc.
So, if you are using Kubernetes, Ingress-nginx is the best choice. If you are using just Docker containers without an orchestrator, use Nginx-proxy.
In Kubernetes, the user decides what set of features to make public and in what manner configuration should be implemented. Help may come from enterprise
vendors, like Ingress controller provided by GKE, and from community/private-held sides covering the uncommon approach to similar aspects of delivery services.
In this particular case, we have two nginx driven solutions.
An Ingress controller is full-featured and mostly recognized as default traffic controller to use with GKE.
Ingress can be configured to give services externally-reachable URLs, load balance traffic, terminate SSL, and offer name-based virtual hosting.
Users request ingress by POSTing the Ingress resource to the API server. An Ingress controller is responsible for fulfilling the Ingress, usually with a loadbalancer, though it may also configure your edge router or additional frontends to help handle the traffic in an HA manner. Nowadays Ingress is strictly cloud-oriented regarding configuration, it uses ConfigMap style and kubedns to register services.
If you know how old-fashioned virtual hosts work and you are not interested in every new cloud-oriented aspects of web services delivery, jwilder/nginx-proxy may be interesting for you. In this solution, nginx can act as a proxy to control internal hosting and world Web traffic with IPv6 ready endpoints. jwilder/nginx is not especially dedicated to clouds but also works fine there. If you are interested in having free Let's Encrypt certificates, there is an out-of-a-box support for it. Some users find it interesting that Basic Authentication is available, and the SSL is more flexible to configure for advanced purposed.