I'm currently using Kubernetes on GKE to serve the various parts of my product on different subdomains with the Ingress resource. For example: api.mydomain.com, console.mydomain.com, etc.
ingress.yml (current):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
spec:
rules:
- host: api.mydomain.com
http:
paths:
- backend:
serviceName: api-service
servicePort: 80
- host: console.mydomain.com
http:
paths:
- backend:
serviceName: console-service
servicePort: 80
That works wonderfully, with the L7 GCE load balancer routing to the appropriate places. What I would like to do, however, is deploy many feature-branch deployments as subdomains to test and demonstrate new features before pushing to production. These could be something like new-stylesheet.console.mydomain.com or upgraded-algorithm.api.mydomain.com, inspired by GitLab CI's environments.
Here's a potential workflow for each deployment:
Create feature-api-deployment.yml
Create feature-api-service.yml
Update ingress.yml with new subdomain rule: feature.api.mydomain.com specifying serviceName: feature-api-service
But enumerating and maintaining all subdomain->service mappings will get messy with tearing down deployments, and create a ton of GCE backends (default quota is 5...) so it's not ideal.
Is there anything built in to Kubernetes that I'm overlooking to handle this? Something like this would be ideal to pick a target service based on a matched subdomain:
ingress.yml (wanted)
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
spec:
rules:
- host: *.api.mydomain.com
http:
paths:
- backend:
serviceName: {value of *}-api-service
servicePort: 80
There certainly isn't anything like wildcard domains available in kubernetes, but you might be able to get want you want using Helm
With helm, you can template variables inside your manifests, so you can have the name of your branch be in the helm values file
From there, you can have gitlab-ci do the helm installation in a build pipeline and if you configure your chart correctly, you can specify a helm argument of the pipeline name.
There's a great blog post about this kind of workflow here - hopefully this'll get your where you need to go.
here is an example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-wildcard-host
spec:
rules:
- host: "foo.bar.com"
http:
paths:
- pathType: Prefix
path: "/bar"
backend:
service:
name: service1
port:
number: 80
- host: "*.foo.com"
http:
paths:
- pathType: Prefix
path: "/foo"
backend:
service:
name: service2
port:
number: 80
and the source.
Services are available locally as
my-svc.svc.cluster.local
You can write a simple NGINX proxy which can forward which can parse the subdomain and proxy pass it to http://$service as long as the subdomain and service name matches as in this case.
Related
I have a domain at Cloudflare and some wildcards for subdomains
which both point to the load balancer of an nginx ingress of a Kubernetes cluster (GKE) of the GCP. Now, we have two pods and services running each (echo1 and echo2, which are essentially identical) and when I apply an ingress
kind: Ingress
metadata:
name: echo-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: "echo1.eu3.example.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: echo1
port:
number: 80
- host: "echo2.example.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: echo2
port:
number: 80
I can reach echo2 under echo2.example.com, but not echo1.eu3.example.com. My question is how I can make the second one reachable as well.
I can advise you to make some check.
Just set the Proxy status for "echo1.eu3.example.com" as DNS only. Then check the access. If ok - install certificates in kubernetes via cert manager. We faced some times with this issue and resolved by using 3 deep domains. For instance "echo1-eu3.example.com". It seems cloudfront does not like such domains :) Of course if someone write a solution how to work with deep domains in cloudfront - it would be good practice for us :)
I have installed Ingress and linked my service to it (usign metallb).
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /api/tasks/*
# pathType: Exact
backend:
serviceName: tasks-service
servicePort: 5004
The thing is this, I set up the default prefix of the paths in the deployment to be
/api/tasks/
where /api/tasks/tasks shows the service is up while /api/tasks/tasks_count gives the total number. However in my k8s cluster, I cannot redirect to the different paths within the service. What could be the problem?
Since this is a result in Google for wildcards and prefixes, I'll answer this old question.
The functionality you're looking for comes from specifying the path type as pathType: Prefix
paths:
- path: /api/tasks
pathType: Prefix
backend:
serviceName: tasks-service
servicePort: 5004
Importantly, the path doesn't contain a wildcard character. In fact, cloud providers like AWS will throw errors if you're using their custom load balancer provisioners for Ingress resources:
prefix path shouldn't contain wildcards
I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routingĀ¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.
I have setup a name based ingress controller, but it doesn't seem to work for anything other than /.
So http://metabase.domain.com works but http://metabase.domain.com/style/app.css does not.
This is my config:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: domain
annotations:
kubernetes.io/ingress.global-static-ip-name: "domain"
name: domain-ingress
spec:
rules:
- host: metabase.domain.com
http:
paths:
- path: /
backend:
serviceName: metabase
servicePort: 80
- host: jenkins.domain.com
http:
paths:
- path: /
backend:
serviceName: jenkins
servicePort: 80
From the nginx.conf in the everything looks normal too. For some reason the nginx access and error logs are also empty so can't find anything from there too
As you mentioned, there is no error in the log files, and everything looks normal from
your perspective. I may suggest to tune up ingress using annotations tags.
I've checked documentation of ingress-nginx and found that below annotations may help a bit.
In some scenarios, the exposed URL in the backend service differs from the specified path
in the Ingress rule. Without a rewrite, any request will return 404.
Set the annotation
nginx.ingress.kubernetes.io/rewrite-target
to the path expected by the service.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
name: rewrite
namespace: default
spec:
rules:
- host: rewrite.bar.com
http:
paths:
- backend:
serviceName: http-svc
servicePort: 80
path: /something
If the Application Root is exposed in a different path and needs to be redirected, set the annotation
nginx.ingress.kubernetes.io/app-root
to redirect requests for /.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/app-root: /app1
name: approot
namespace: default
spec:
rules:
- host: approot.bar.com
http:
paths:
- backend:
serviceName: http-svc
servicePort: 80
path: /
If the application contains relative links, it is possible to add an additional annotation
nginx.ingress.kubernetes.io/add-base-url
that will prepend a base tag in the header of the returned HTML from the backend.
I got around to solving this, turns out somehow there was another ingress on my namespace that was a very old version of the nginx controller. I cleared my cluster and relaunched my ingress. Things worked out. Thanks for all the help everyone!
How do I set a Kubernentes Ingress and Controller to essentially do what the following nginx.conf file does:
upstream backend {
server server1.example.com weight=5;
server server2.example.com:8080;
server backup1.example.com:8080 backup;
}
I want one http endpoint to map to multiple Kubernetes services with a preference for a primary one but also have a backup one. (For my particular project, I need to have multiple services instead of one service with multiple pods.)
Here's my attempted ingress.yaml file. I'm quite certain that the way I'm listing the multiple backends is incorrect. How would I do it? And how do I set the "backup" flag?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: fanout-ingress
annotations:
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: "nginx"
# kubernetes.io/ingress.global-static-ip-name: "kubernetes-ingress"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: server1
servicePort:
- path: /
backend:
serviceName: server2
servicePort: 8080
- path: /
backend:
serviceName: backup1
servicePort: 8080
I'm running Kubernetes on GKE.
You can do simple fanout based on path or name based virtual hosting.
However, you'd need to distinguish based on something (other than port, since it's an Ingress), so your two options would be virtual host or path.
Paths will not work with some services that expect a standard path. Judging based on your example you'd most likely want to have something like a.example.com and b.example.com. Here's the example from the Kubernetes docs:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
Kubernetes Ingress is incapable of this.
You could create a new service that targets server1, server2 and backup1 and use that in the Ingress. But the backends will be used in a round robin fashion.
You can create a Deployment and a Service of (stateless) nginx reverse proxies with the config you wish and use that in Ingress.