Most service meshes use Envoy as a sidecar proxy but that doesn't seem to be the default option for Linkerd. We are already using Envoy as ingress and don't really want to add a new proxy into the mix. Can we use Linkerd with Envoy?
I've heard this question asked a few times, and I think there are two questions here:
Can you use Linkerd with Envoy as the Ingress controller?
Does the Linkerd control plane support using Envoy as the data-plane proxy?
The answer to the first question is that you can definitely use Linkerd with Envoy as the Ingress controller. Ambassador and Contour are two examples of open source projects that use Envoy as the proxy for routing Ingress traffic.
Integrating Linkerd with any ingress controller requires some configuration that is specific to each of the ingress controllers. You also want to make sure that the Pod that runs the ingress controller container is injected with the Linkerd proxy so that the traffic between the ingress controller and the service workloads is part of the Linkerd service mesh.
The answer to the second question, which is the one that I think you're asking is that Linkerd provides its own proxy out of the box named linkerd2-proxy that is purpose built for managing microservice workloads within Kubernetes. The Linkerd control plane provides API endpoints and is designed to work with the linkerd2-proxy, so Envoy won't work out of the box.
That being said, Linkerd is an open source project and you could theoretically write an adapter between the Linkerd control plane components and the Envoy XSD API. Off the top of my head, I can't think of a common use case where the effort in building this adapter would bring additional value to Linkerd.
There's a great blog post that was written by William Morgan that describes why the Linkerd team decided to write their own proxy instead of using Envoy. The tl;dr is that Envoy is a general purpose proxy, whereas the linkerd2-proxy is purpose built for handling microservices traffic in the most efficient an unobtrusive way possible.
Related
I understand there are various ways to get external traffic into the cluster - Ingress, cluster IP, node port and load balancer. I am particularly looking into the Ingress and k8s and from the documentation k8s supports AKS, EKS & Nginx controllers.
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
To implement Ingress, understand that we need to configure an Ingress Controller in the cluster. My query is whether Nginx ingress & proxy are an offering of core k8s (packaged / embedded)? Might have overlooked, did not find any documentation where it is mentioned. Any insight or pointer to documentation if stated above is true is highly appreciated.
Just reading the first rows of the page you linked, it states that no controller are started automatically with a cluster and that you must choose the one of your preference, depending on your requirements
Ingress controllers are not started automatically with a cluster. Use
this page to choose the ingress controller implementation that best
fits your cluster.
Kubernetes defines Ingress, IngressClass and other ingress-related resources but a fresh installation does not come with any default.
Some prepackaged installation of Kubernetes (like microk8s, minikube etc...) comes with ingress controller that, usually, needs to be enabled manually during the installation/configuration phase.
All the tutorials I've found on the Internet demonstrate how to deploy a web site/app in an nginx container and set up an Ingress to proxy related requests to that container.
That seems kind of redundant to me. Why not to place the content on a volume, mount it to ingress-nginx-controller and serve static assets from there. It seems to be better from performance perspective: no additional proxy and no additional container (with nginx) that consumes compute resources.
An Ingress resource is an abstract concept that was designed to route HTTP/S requests to kubernetes services. It defines a limited subset of that functionality to remain generic and be implemented by many types of ingress controller.
ingress-nginx is only one implementation of an ingress controller, that happens to also be good at serving static content. HAProxy, Traefik or the AWS lb are at the other end of the scale.
An nginx based ingress controller could add custom annotations to support something custom like what you propose, but I imagine you would have a hard time arguing that into the project. Internally, they already add a second nginx instance for the default backend.
If one proxy hop is a critical differentiator in performance then you would probably get more benefit hosting the static content on an edge CDN/device. Another alternative is to not rely on an Ingress and publish the service directly to the outside world.
You need a container to run the application. Ingress service is just used to access that application outside the Kubernetes cluster. You cannot just keep the content in a volume and mount it to the ingress controller. One Ingress controller is used to route requests for multiple applications.
I am a kubernetes newbie, and no matter how much I read about it I cannot get my head around this issue.
I have a simple deployment , which is creating a pod with a not so complex app.
I know what an ingress and an ingress controller is doing ,but as far as I understand it is not required for me to expose my pod-app externally. Only a LoadBalancer service should be enough.
I do not have need of more than one rule for traffic routing.
Am I wrong about that?
Traditionally, you would create a LoadBalancer service for each service you want to expose externally. This can get rather expensive. Ingress gives you a way to route requests to services based on the request host or path, centralizing a number of services into a single entrypoint.
Also load balancer provisioning takes time and works only in supported cloud providers such as AWS, GCP etc.
One more thing to consider is the need of L4(TCP/UDP) layer routing because kubernetes Ingress API is primarily L7 layer but some of the ingress controllers such as traefik, nginx supports L4 layer(TCP/UDP) along with L7 layer(HTTP) routing.
So answer to your question is it depends based on your environment and use cases.
Ingress and IngressControllers are used for traffic routing at layer 7 i.e., if your backends talk L7 protocols like HTTP, GRPC etc. You can route requests based on request path to different backend services using Ingress.
If your app doesn't operate at Layer 7, you might not need an Ingress.
Another question you could ask yourself if your migrating your app from non-kubernetes environment to kuberneters is - are you using a reverse proxy like nginx already? If so, you might want to use Ingress. I say might because it is not necessary. You can achieve the same affect of using an Ingress by running the nginx container as a pod, writing the nginx.conf yourself and making it available externally(using a LoadBalancer service for example). Instead by using IngressController you need not maintain the nginx pod or write nginx.conf. You can instead express the same configuration as Ingress resource which is much simpler.
Ingress is needed if you want to expose your service to external. Especially for layer 7 in OSI model (HTTP transport). Ingress also provide the mechanism to enable TLS support in your load balancer. Traffic routing is controlled by rules defined on the Ingress resource. An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
By default Ingress controller depends on which kind of cloud provider you're using, or if you're using on premise you'll need to set it up based on what you need. In one cluster, you can create multiple Ingress controller as well. There's many kinds of Ingress controller, you can take a look on this article.
I think this article about Load Balancer and Ingress may also help.
The Problem
I need to expose a Kubernetes NodePort service externally over https.
The Setup
I've deployed Kubernetes on bare-metal and have deployed Polyaxon on the cluster via Helm
I need to access Polyaxon's dashboard via the browser, using a virtual machine that's external to the cluster
The dashboard is exposed as a NodePort service, and I'm able to connect to it over http. I am not able to connect over https, which is a hard requirement in my case.
Following an initial "buildout" period, both the cluster and the virtual machine will not have access to the broader internet. They will connect to one another and that's it.
Polyaxon supposedly supports SSL/TLS through its own configs, but there's very little documentation on this. I've made my best attempts to solve the issue that way and also bumped an issue on their github, but haven't had any luck so far.
So I'm now wondering if there might be a more general Kubernetes hack that could help me here.
The Solutions
I'm looking for the simplest solution, rather than the most elegant or scalable. There are also some things that might make my situation simpler than the average user who would want https, namely:
It would be OK to support https on just one node, rather than every node
I don't need (or really want) a domain name; connecting at https://<ip_address>:<port> is not just OK but preferred
A self-signed certificate is also OK
So I'm hoping there's some way to manipulate the NodePort service directly such that https will work on the virtual machine. If that's not possible, other solutions I've considered are using an Ingress Controller or some sort of proxy, but those solutions are both a little half-baked in my mind. I'm a novice with both Kubernetes and networking ideas in general, so if you're going to propose something more complex please speak very slowly :)
Thanks a ton for your help!
Ingress-controller it's a standard way to expose HTTP backend over TLS connection from cluster to client.
Existing NodePort service has ClusterIP which can be used as a backend for Ingress. ClusterIP type of service is enough, so you can change service type later to prevent HTTP access via nodeIP:nodePort.
Ingress-controller allows you to teminate TLS connection or pass-through TLS traffic to the backend.
You can use self-signed certificate or use cert-manager with Let's encrypt service.
Note, that starting from 0.22.0 version Nginx-ingress rewrite syntax has changed and some examples in the articles may be outdated.
Check the links:
TLS termination
TLS/HTTPS
How to get Kubernetes Ingress to terminate SSL and proxy to service?
Configure Nginx Ingress Controller for TLS termination on Kubernetes on Azure
jwilder/nginx-proxy has 1.3K STARS and 10M+ PULLS on Docker Hub. And Watch 262, Star 7701, Fork 1546 on GitHub.
https://github.com/jwilder/nginx-proxy
kubernetes/ingress-nginx has 13 stars on kubeapps.com (one of the most starred charts) and Watch 137, Star 1596, Fork 918 on GitHub.
https://github.com/kubernetes/ingress-nginx
What's the difference between the two?
When would you use one over the other?
That is 2 different applications, but both are based on Nginx and have the similar function.
Nginx-proxy by jwilder is a proxy server for Docker containers which includes docker-gen to generate a configuration for Nginx automatically. You can use it for SSL termination, load balancing etc. But it will be hard to manage nginx-proxy in Kubernetes.
Ingress-nginx by Kubernetes is Ingress Controller which provides Ingress functional for your Kubernetes cluster. It also can do SSL termination and some other things, but it was created especially for use in Kubernetes, and it's abstractions. That means you can create the Ingress object which includes Services as backends and use selectors etc.
So, if you are using Kubernetes, Ingress-nginx is the best choice. If you are using just Docker containers without an orchestrator, use Nginx-proxy.
In Kubernetes, the user decides what set of features to make public and in what manner configuration should be implemented. Help may come from enterprise
vendors, like Ingress controller provided by GKE, and from community/private-held sides covering the uncommon approach to similar aspects of delivery services.
In this particular case, we have two nginx driven solutions.
An Ingress controller is full-featured and mostly recognized as default traffic controller to use with GKE.
Ingress can be configured to give services externally-reachable URLs, load balance traffic, terminate SSL, and offer name-based virtual hosting.
Users request ingress by POSTing the Ingress resource to the API server. An Ingress controller is responsible for fulfilling the Ingress, usually with a loadbalancer, though it may also configure your edge router or additional frontends to help handle the traffic in an HA manner. Nowadays Ingress is strictly cloud-oriented regarding configuration, it uses ConfigMap style and kubedns to register services.
If you know how old-fashioned virtual hosts work and you are not interested in every new cloud-oriented aspects of web services delivery, jwilder/nginx-proxy may be interesting for you. In this solution, nginx can act as a proxy to control internal hosting and world Web traffic with IPv6 ready endpoints. jwilder/nginx is not especially dedicated to clouds but also works fine there. If you are interested in having free Let's Encrypt certificates, there is an out-of-a-box support for it. Some users find it interesting that Basic Authentication is available, and the SSL is more flexible to configure for advanced purposed.