How do I set a Kubernentes Ingress and Controller to essentially do what the following nginx.conf file does:
upstream backend {
server server1.example.com weight=5;
server server2.example.com:8080;
server backup1.example.com:8080 backup;
}
I want one http endpoint to map to multiple Kubernetes services with a preference for a primary one but also have a backup one. (For my particular project, I need to have multiple services instead of one service with multiple pods.)
Here's my attempted ingress.yaml file. I'm quite certain that the way I'm listing the multiple backends is incorrect. How would I do it? And how do I set the "backup" flag?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: fanout-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: "nginx"
# kubernetes.io/ingress.global-static-ip-name: "kubernetes-ingress"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: server1
servicePort:
- path: /
backend:
serviceName: server2
servicePort: 8080
- path: /
backend:
serviceName: backup1
servicePort: 8080
I'm running Kubernetes on GKE.
You can do simple fanout based on path or name based virtual hosting.
However, you'd need to distinguish based on something (other than port, since it's an Ingress), so your two options would be virtual host or path.
Paths will not work with some services that expect a standard path. Judging based on your example you'd most likely want to have something like a.example.com and b.example.com. Here's the example from the Kubernetes docs:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
Kubernetes Ingress is incapable of this.
You could create a new service that targets server1, server2 and backup1 and use that in the Ingress. But the backends will be used in a round robin fashion.
You can create a Deployment and a Service of (stateless) nginx reverse proxies with the config you wish and use that in Ingress.
Related
I have some simple deployments, pods, services and nginx ingress in Kubernetes. I want to use ingress to route to the services (cluster-ip).
However, there are 2 services for 2 pods with the same path (i.e /abc/def). After I applied the ingress.yaml file, I got an error message saying "nginx: [emerg] duplicate location "/abc/def/" in /tmp/nginx/nginx-cfg728288520:2893".
May I know how to let ingress accepts the same paths with different service and different port?
Here is the ingress.yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
name: ingress-nginx-default
namespace: default
spec:
rules:
- host:
http:
paths:
- path: /abc/def/
pathType: Prefix
backend:
service:
name: service1
port:
number: 8090
- path: /abc/def
backend:
service:
name: service2
port:
number: 8068
pathType: Prefix
ingressClassName: nginx
"nginx: [emerg] duplicate location "/abc/def/" in /tmp/nginx/nginx-cfg728288520:2893". This error indicates the same host with two same paths which is a duplicate location.
You can use simple fanout based on path or name based virtual hosting.
In order to do this you need to have two hosts that need to be mentioned in the ingress.
based on your example you'd most likely want to have something like foo.bar.com and bar.foo.com. Here's the example from the Kubernetes docs:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: foo.bar.com
http:
paths: #path_name
- backend:
serviceName: service1
servicePort: 80
- host: bar.foo.com
http:
paths: #path_name
- backend:
serviceName: service2
servicePort: 80
I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routingĀ¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.
I have a kubernetes cluster running in Azure Kubernetes Service (AKS). I've been following a series of workshops and I've set up an NGINX ingress controller. Right now I'm using nip.io in order to access my site. They had me create an ingress resource to expose the front end:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ratings-web-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: frontend.<redacted ip>.nip.io
http:
paths:
- backend:
serviceName: ratings-web
servicePort: 80
path: /
This works fine. I can brows the nip.io address and everything works right.
I wanted to extend this and create a separate site. I want it at the same address, just served at /foo
I tried to create an ingress resource:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: foo-web-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: frontend.<redacted ip>.nip.io
http:
paths:
- backend:
serviceName: foo-web
servicePort: 80
path: /foo
When I browse frontend.<redacted ip>.nip.io/foo I get a 404 not found from nginx.
I tried added the following annotation based on another SO post:
nginx.ingress.kubernetes.io/rewrite-target: /
Now when I browse /foo it responds, but all the content is blank. In the web app it's referencing everything at the root level like:
<link href="/assets/css/bootstrap.min.css" rel="stylesheet" type="text/css" />
When it sees this path, I think it's going to the original website so I get strict MIME type errors or 404 errors.
What do I need to do to host two websites on the same nginx ingress controller at different paths?
Ingress controller will be confused when merging these two ingresses, what the correct order should be because more than one ingress resource is defined for same host.
Referring from docs you could use a single ingress resource as fanout.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: <redacted ip>.nip.io
http:
paths:
- path: /foo
backend:
serviceName: foo-web
servicePort: 80
- path: /bar
backend:
serviceName: bar-web
servicePort: 80
I'm setting up a Kubernetes cluster which contains three applications, each running within their own respective pod and service. The web frontends of these applications should be accesible at ports 80, 9000 and 15672. There are also a number of Backend-APIs running in their own pods and services, which should be accesible at their respective ports. The cluster is accessed through a NGINX reverse proxy with the following ingress-definition:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
namespace: my-namespace
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
tls:
- hosts:
- myapp.com
secretName: my-certificat
rules:
- host: myapp.com
http:
paths:
- backend:
serviceName: my-service-1
servicePort: 8092
path: /api/someroute/(.*)
- backend:
serviceName: my-service-2
servicePort: 30003
path: /api/someotherroute/(.*)
- backend:
serviceName: my-other-frontend
servicePort: 9000
path: /other/(.*)
- backend:
serviceName: my-yetanother-frontend
servicePort: 15672
path: /yetanother/(.*)
- backend:
serviceName: my-main-frontend
servicePort: 80
path: /(.*)
This works for the api-services, but not for the frontends. Was I to enter a URI like myapp.com/other/ in my browser it would make a call to my-other-frontend:9000/other/, instead of my-other-frontend:9000/. This can of course be solved with a rewrite annotation like nginx.ingress.kubernetes.io/rewrite-target: /$1, the problem is, that this would also apply to the api-services, and those actually need the complete route in their call, so a general rewrite rule would break them.
So my question is: Is it possible to define rewrite rules that only apply to specific paths?
I would advise to split your ingress into 2 based on your different annotations.
1) First one for api-services, called my-api-ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-api-ingress
namespace: my-namespace
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
tls:
- hosts:
- myapp.com
secretName: my-certificat
rules:
- host: myapp.com
http:
paths:
- backend:
serviceName: my-service-1
servicePort: 8092
path: /api/someroute/(.*)
- backend:
serviceName: my-service-2
servicePort: 30003
path: /api/someotherroute/(.*)
2) And second one, lets say my-front-ingress with nginx.ingress.kubernetes.io/rewrite-target: /$1 annotation for rest of your frontends. Based on your question I believe you dont need the detailed instruction for ingress creation with rewrite annotation.
EDIT1:
Yes, its possible to create multiple ingresses with the same host. But please pay attention which exactly nginx ingress. Refer to #LazerBass answer with his investigations.
In short, based on nginx ingress comparation table,
The nginxinc controller does not support merging Ingress rules with the same host. You can use it only with Mergeable Ingresses
The easiest way is to use regular kubernetes/ingress-nginx
Hope that helps.
I'm currently using Kubernetes on GKE to serve the various parts of my product on different subdomains with the Ingress resource. For example: api.mydomain.com, console.mydomain.com, etc.
ingress.yml (current):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
spec:
rules:
- host: api.mydomain.com
http:
paths:
- backend:
serviceName: api-service
servicePort: 80
- host: console.mydomain.com
http:
paths:
- backend:
serviceName: console-service
servicePort: 80
That works wonderfully, with the L7 GCE load balancer routing to the appropriate places. What I would like to do, however, is deploy many feature-branch deployments as subdomains to test and demonstrate new features before pushing to production. These could be something like new-stylesheet.console.mydomain.com or upgraded-algorithm.api.mydomain.com, inspired by GitLab CI's environments.
Here's a potential workflow for each deployment:
Create feature-api-deployment.yml
Create feature-api-service.yml
Update ingress.yml with new subdomain rule: feature.api.mydomain.com specifying serviceName: feature-api-service
But enumerating and maintaining all subdomain->service mappings will get messy with tearing down deployments, and create a ton of GCE backends (default quota is 5...) so it's not ideal.
Is there anything built in to Kubernetes that I'm overlooking to handle this? Something like this would be ideal to pick a target service based on a matched subdomain:
ingress.yml (wanted)
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
spec:
rules:
- host: *.api.mydomain.com
http:
paths:
- backend:
serviceName: {value of *}-api-service
servicePort: 80
There certainly isn't anything like wildcard domains available in kubernetes, but you might be able to get want you want using Helm
With helm, you can template variables inside your manifests, so you can have the name of your branch be in the helm values file
From there, you can have gitlab-ci do the helm installation in a build pipeline and if you configure your chart correctly, you can specify a helm argument of the pipeline name.
There's a great blog post about this kind of workflow here - hopefully this'll get your where you need to go.
here is an example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-wildcard-host
spec:
rules:
- host: "foo.bar.com"
http:
paths:
- pathType: Prefix
path: "/bar"
backend:
service:
name: service1
port:
number: 80
- host: "*.foo.com"
http:
paths:
- pathType: Prefix
path: "/foo"
backend:
service:
name: service2
port:
number: 80
and the source.
Services are available locally as
my-svc.svc.cluster.local
You can write a simple NGINX proxy which can forward which can parse the subdomain and proxy pass it to http://$service as long as the subdomain and service name matches as in this case.