I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routing¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.
Related
I have deployed Nginx in minikube in AWS EC2 Instance using the below commands:
kubectl create deployment --image=nginx nginx-app
kubectl expose deployment nginx-app --port=80 --name=nginx-http --type=NodePort
It is available on NodePort: 31568
I have added this port in Security Group and able to access it on browser in another laptop with http://EC2-PublicIP:31568
I have many microservices which are to be exposed to outside.
Is there any way to deploy an API Gateway (Nginx or Ingress) and expose on a port and should be able to access other microservices like
http://EC2-PublicIp:31568/helloworld
http://EC2-PublicIp:31568/mainpage
http://EC2-PublicIp:31568/editorspage
etc
I have tried adding metallb, accordingly, an IP is allocated to ingress (load balancer type). I was able to access all the microservices specified in ingress.yaml only inside the EC2 Instance.
This should be accessible from the outside. If yes, then how to configure it?
Any help is appreciated.
You should be able to do so since you are using Nginx.
the configuration should like something like:
Annotation comes to help
The "trick" is to set annotations to support regexp & rewrite-target with:
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
#
# the value can be set to `/` or `$1` or `$2` and so on
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
This is the "important" annotation - the rewrite one
# Without a rewrite any request will return 404
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
annotations:
#
# This is the expected line
#
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
spec:
rules:
- host: test.com # <- Set your host
http:
paths:
#
# List of desired paths
#
- path: /path1
backend:
serviceName: nginx-http
servicePort: 80
- path: /path2/[A-Z0-9]{3}
backend:
serviceName: nginx-http
servicePort: 80
I have REST API Web service on Internal GKE cluster which I would like to expose with internal HTTP load balancing.
Let's call this service "blue" service:
I would like to expose it in following mapping:
http://api.xxx.yyy.internal/blue/isalive -> http://blue-service/isalive
http://api.xxx.yyy.internal/blue/v1/get -> http://blue-service/v1/get
http://api.xxx.yyy.internal/blue/v1/create -> http://blue-service/v1/create
http://api.xxx.yyy.internal/ -> http://blue-service/ (expose Swagger)
I'm omitting deployment yaml, since it's less relevant to discussion.
But my service yaml looks like this:
apiVersion: v1
kind: Service
metadata:
name: blue-service
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
selector:
app: blue-service
My Ingress configuration is the following:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: blue-ingress
annotations:
kubernetes.io/ingress.class: "gce-internal"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: api.xxx.yyy.internal
http:
paths:
- path: /blue/*
backend:
serviceName: blue-service
servicePort: 80
However, I'm receiving 404 for all requests. /blue/v1/get, /blue/v1/create and /blue/isalive returns 404.
In my "blue" application I log all my notFound requests and I can clearly see that my URIs are not being rewritten, the requests hitting the application are /blue/v1/get, /blue/v1/create and /blue/isalive.
What am I missing in Ingress configuration? How can I fix those rewrites?
I solved the problem and writing it here to memo it and hopefully someone will find it as useful.
First problem is that I have mixed annotations types. one of GKE ingress controller and second for Nginx Server controller. Currently GKE ingress controller doesn't support URL rewrite feature, so I need to use nginx ingress controller.
so I need to install Nginx based ingress controller. It cloud be done easily using Helm chart or or deployment yaml. However, by default this controller will expose ingress using external load balancer and this not what I want. So we need to modify deployment charts or YAML file of this controller.
I'm not using Helm, so I downoaded yaml itself using wget command.
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/cloud/deploy.yaml
Open it in editor and find the definition of Service names ingress-nginx-controller in namespace ingress-nginx. Add the following annotation.
cloud.google.com/load-balancer-type: "Internal"
After it I can run kubectl apply -f deploy.yaml command which will create Ingress controller for me. It will take a few minutes to provision it.
In addition I need to open firewall rule which will allow master nodes access worker nodes on port 8443/tcp.
And the last item is an ingress yaml itself which should look like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
kubernetes.io/ingress.class: "nginx"
name: blue-ingress
namespace: default
spec:
rules:
- host: api.xxx.yyy.internal
http:
paths:
- backend:
serviceName: blue-service
servicePort: 80
path: /blue(/|$)(.*)
There is a kubernetes baremetal cluster which has 7 nodes.
Installed is helm and added a helm repo added
ingress-nginx https://kubernetes.github.io/ingress-nginx
In my lan i created a dns A record which is resolvable in my lan.
What works:
https://matthewpalmer.net/kubernetes-app-developer/articles/kubernetes-ingress-guide-nginx-example.html
I am able to access the apple and the banana location on the dns A record mentioned before.
When i deploy a default nginx image with a service with port 80 and of type cluster-ip the following nginx-ingress resource does not work
kubectl run nginx --image=nginx ; kubectl exposed pod nginx --port=80
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-test
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: kubernetes.somename.lan
http:
paths:
- path: /test
pathType: Prefix
backend:
serviceName: nginx
servicePort: 80
I am seeing in the logs that the file is being deployed in the nginx-ingress image, not seeing anything strange in comparison to the apple and banana ingress entry. Only difference i can come up with is that ingress controller and nginx image are bot using port 80
So what i need is that when i access http://somelan.lan/test i will end up at the placeholder of the nginx server (which is accessible by nginx service )
Question:
How can i access nginx on my A dns record under the path /test
Thank you in advancee
The provided apple / banana example was working because the pod is path insensitive. It does not react to path changes (ignores paths). While for nginx it is exactly opposite.
The blog has a mistake with annotation and its missing the nginx prefix. This is the one that you used:
ingress.kubernetes.io/rewrite-target
And this is the correct one as per docs:
nginx.ingress.kubernetes.io/rewrite-target
Ok, after research i found out how to solve this.
Hope this helps anyone
https://www.reddit.com/r/kubernetes/comments/j2neuf/nginxingress_path_routing/
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-precious
annotations:
nginx.org/rewrites: "serviceName=nginx-ingress rewrite=/;"
spec:
rules:
- host: kubernetes.somelan.lan
http:
paths:
- path: /test1
backend:
serviceName: nginx-ingress
servicePort: 80
I have one service called "workspace-service-b6" which is running on port 5000, See the below ingress file. Now I want to serve the static content on the same service (workspace-service-b6) by adding the path route.
Example:- Service is working on https://workspace-b6.dev.example.com
Now if the user adds "/workspace/v2/ "at the end of the URL.
Like this:- https://workspace-b6.dev.example.com/workspace/v2/ it will redirect to s3 bucket "https://s3.console/buckets/xyz/abc/build" where my static content is available.
My Ingress file :-
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: b6-ingress
namespace: b6
annotations:
kubernetes.io/ingress.class: "nginx"
kubernetes.io/tls-acme: "true"
spec:
tls:
- hosts:
- workspace-b6.dev.example.com
secretName: xyz-crt
rules:
- host: workspace-b6.dev.example.com
http:
paths:
- backend:
serviceName: workspace-service-b6
service port: 5000
While it’s kind of possible, the real answer is “don’t”. The ingress system is just a proxy, set up separate pods for content.
Problem
I would like to host multiple services on a single domain name under different paths. The problem is that I'm unable to get request path rewriting working using nginx-ingress.
What I've tried
I've installed nginx-ingress using these instructions:
helm install stable/nginx-ingress --name nginx-ingress --set controller.publishService.enabled=true
CHART APP VERSION
nginx-ingress-0.3.7 1.5.7
The example works great with hostname based backends:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: first.testdomain.com
http:
paths:
- backend:
serviceName: hello-kubernetes-first
servicePort: 80
However, I can't get path rewriting to work. This version redirects requests to the hello-kubernetes-first service, but doesn't do the path rewrite so I get a 404 error from that service because it's looking for the /foo directory within that service (which doesn't exist).
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: first.testdomain.com
http:
paths:
- backend:
serviceName: hello-kubernetes-first
servicePort: 80
path: /foo
I've also tried this example for paths / rewriting:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: first.testdomain.com
http:
paths:
- backend:
serviceName: hello-kubernetes-first
servicePort: 80
path: /foo(/|$)(.*)
But the requests aren't even directed to the hello-kubernetes-first service.
It appears that my rewrite configuration isn't making it to the /etc/nginx/nginx.conf file. When I run the following, I get no results:
kubectl exec nginx-ingress-nginx-ingress-XXXXXXXXX-XXXXX cat /etc/nginx/nginx.conf | grep rewrite
How do I get the path rewriting to work?
Additional information:
kubectl / kubernetes version: v1.14.8
Hosting on Azure Kubernetes Service (AKS)
This is not likely to be an issue with AKS, as the components you use are working on top of Kubernetes layer. However, if you want to be sure you can deploy this on top of minikube locally and see if the problem persists.
There are also few other things to consider:
There is a detailed guide about creating ingress controller on AKS. The guide is up to date and confirmed to be working fine.
This article shows you how to deploy the NGINX ingress controller in
an Azure Kubernetes Service (AKS) cluster. The cert-manager project is
used to automatically generate and configure Let's Encrypt
certificates. Finally, two applications are run in the AKS cluster,
each of which is accessible over a single IP address.
You may also want to use alternative like Traefik:
Traefik is a modern HTTP reverse proxy and load balancer made to
deploy microservices with ease.
Remember that:
Operators will typically wish to install this component into the
kube-system namespace where that namespace's default service account
will ensure adequate privileges to watch Ingress resources
cluster-wide.
Please let me know if that helped.