Move `X-Accel-Redirect` pattern for protected static in kubernetes setup - nginx

We are moving to kubernetes and we are totally new to it.
In our current mono service setup we have: Nginx -> Web application. These way we can protect some static assets via authentication in the web application and use internal and X-Accel-Redirect of Nginx to serve static files after authentication takes place.
Now in kubernetes we have Ingress and behind these services:
web app
private static service
Is there a way to tell in ingress from the web application to "redirect" the request as we kind of do with sendfile, so that the private static service will reply to it? Or somehow to achieve protecting our static while keeping the static service separate and independent in kubernetes setup?
We kind of made it work by chaining the private static service in front of the web application, but it feels there must be a better way to do it.

Here is how I managed to make it work.
I created two ingresses.
First:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
internal;
nginx.ingress.kubernetes.io/rewrite-target: /some/path/$1
name: static-service-internal
spec:
rules:
- http:
paths:
- backend:
serviceName: private-static-service
servicePort: 80
path: /protected/(.*)
and second service:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
name: web-app
spec:
rules:
- http:
paths:
- backend:
serviceName: web-app
servicePort: 80
path: /
What you see above is supposed to work as in this example from nginx documentation
When receiving X-Accel-Redirect: /protected/iso.img from web-app then it will request /some/path/iso.img from private static service.
Let me know if this solves you problem.

Related

Deploying Microservice in Kubernetes with swagger .net core using ingress controller

Hello I am using ingress controller for api gateway and used .net core miroservice along with swagger implementation, I referred most of the examples / questions related to it on stackover flow but when I try to hit API from swagger it gives 404, is there any way to solve this problem without specifying service name in a code?
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: rewrite
namespace: default
spec:
ingressClassName: nginx
rules:
- host: rewrite.bar.com
http:
paths:
- path: /Service1(/|$)(.*)
pathType: Prefix
backend:
service:
name: LoginService
port:
number: 80
when I try to access swagger then I am able to access it from url like {{domain}}/service1/swagger/.
but API end point generated by swagger doesn't contain service1 in the URL which causes 404.
I know there are solution to change Open API end point from the code but I don't want to do any changes in a code as microservice should not know about where it is going to host.
Any advice on the same?
You can use below code
app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint($"/Service1/swagger/v1/swagger.json", $"APP API - {version}");
});

GKE - expose service with Ingress and Internal Load Balancing

I have REST API Web service on Internal GKE cluster which I would like to expose with internal HTTP load balancing.
Let's call this service "blue" service:
I would like to expose it in following mapping:
http://api.xxx.yyy.internal/blue/isalive -> http://blue-service/isalive
http://api.xxx.yyy.internal/blue/v1/get -> http://blue-service/v1/get
http://api.xxx.yyy.internal/blue/v1/create -> http://blue-service/v1/create
http://api.xxx.yyy.internal/ -> http://blue-service/ (expose Swagger)
I'm omitting deployment yaml, since it's less relevant to discussion.
But my service yaml looks like this:
apiVersion: v1
kind: Service
metadata:
name: blue-service
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
selector:
app: blue-service
My Ingress configuration is the following:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: blue-ingress
annotations:
kubernetes.io/ingress.class: "gce-internal"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: api.xxx.yyy.internal
http:
paths:
- path: /blue/*
backend:
serviceName: blue-service
servicePort: 80
However, I'm receiving 404 for all requests. /blue/v1/get, /blue/v1/create and /blue/isalive returns 404.
In my "blue" application I log all my notFound requests and I can clearly see that my URIs are not being rewritten, the requests hitting the application are /blue/v1/get, /blue/v1/create and /blue/isalive.
What am I missing in Ingress configuration? How can I fix those rewrites?
I solved the problem and writing it here to memo it and hopefully someone will find it as useful.
First problem is that I have mixed annotations types. one of GKE ingress controller and second for Nginx Server controller. Currently GKE ingress controller doesn't support URL rewrite feature, so I need to use nginx ingress controller.
so I need to install Nginx based ingress controller. It cloud be done easily using Helm chart or or deployment yaml. However, by default this controller will expose ingress using external load balancer and this not what I want. So we need to modify deployment charts or YAML file of this controller.
I'm not using Helm, so I downoaded yaml itself using wget command.
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/cloud/deploy.yaml
Open it in editor and find the definition of Service names ingress-nginx-controller in namespace ingress-nginx. Add the following annotation.
cloud.google.com/load-balancer-type: "Internal"
After it I can run kubectl apply -f deploy.yaml command which will create Ingress controller for me. It will take a few minutes to provision it.
In addition I need to open firewall rule which will allow master nodes access worker nodes on port 8443/tcp.
And the last item is an ingress yaml itself which should look like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
kubernetes.io/ingress.class: "nginx"
name: blue-ingress
namespace: default
spec:
rules:
- host: api.xxx.yyy.internal
http:
paths:
- backend:
serviceName: blue-service
servicePort: 80
path: /blue(/|$)(.*)

How do I host multiple services using subdirectories with nginx-ingress?

Problem
I would like to host multiple services on a single domain name under different paths. The problem is that I'm unable to get request path rewriting working using nginx-ingress.
What I've tried
I've installed nginx-ingress using these instructions:
helm install stable/nginx-ingress --name nginx-ingress --set controller.publishService.enabled=true
CHART APP VERSION
nginx-ingress-0.3.7 1.5.7
The example works great with hostname based backends:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: first.testdomain.com
http:
paths:
- backend:
serviceName: hello-kubernetes-first
servicePort: 80
However, I can't get path rewriting to work. This version redirects requests to the hello-kubernetes-first service, but doesn't do the path rewrite so I get a 404 error from that service because it's looking for the /foo directory within that service (which doesn't exist).
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: first.testdomain.com
http:
paths:
- backend:
serviceName: hello-kubernetes-first
servicePort: 80
path: /foo
I've also tried this example for paths / rewriting:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: first.testdomain.com
http:
paths:
- backend:
serviceName: hello-kubernetes-first
servicePort: 80
path: /foo(/|$)(.*)
But the requests aren't even directed to the hello-kubernetes-first service.
It appears that my rewrite configuration isn't making it to the /etc/nginx/nginx.conf file. When I run the following, I get no results:
kubectl exec nginx-ingress-nginx-ingress-XXXXXXXXX-XXXXX cat /etc/nginx/nginx.conf | grep rewrite
How do I get the path rewriting to work?
Additional information:
kubectl / kubernetes version: v1.14.8
Hosting on Azure Kubernetes Service (AKS)
This is not likely to be an issue with AKS, as the components you use are working on top of Kubernetes layer. However, if you want to be sure you can deploy this on top of minikube locally and see if the problem persists.
There are also few other things to consider:
There is a detailed guide about creating ingress controller on AKS. The guide is up to date and confirmed to be working fine.
This article shows you how to deploy the NGINX ingress controller in
an Azure Kubernetes Service (AKS) cluster. The cert-manager project is
used to automatically generate and configure Let's Encrypt
certificates. Finally, two applications are run in the AKS cluster,
each of which is accessible over a single IP address.
You may also want to use alternative like Traefik:
Traefik is a modern HTTP reverse proxy and load balancer made to
deploy microservices with ease.
Remember that:
Operators will typically wish to install this component into the
kube-system namespace where that namespace's default service account
will ensure adequate privileges to watch Ingress resources
cluster-wide.
Please let me know if that helped.

Is it possible to rewrite HOST header in k8s Ingress Controller?

Due to some legacy application that relies on Host header to function correctly, I need to have an Ingress (proxy, etc) that capable of rewrite Host header and pass that to downstream (backend). Is there any Ingress Controller that supports this functionality?
Example:
End user access our website through foo.com/a for backend a and foo.com/b for backend b. But since a and b are legacy app, it only accept:
a accepts connection when Host: a.foo.com
b accepts connection when Host: b.foo.com
This can be done using this annotation: nginx.ingress.kubernetes.io/upstream-vhost: host.example.com
I'm not sure whether you can find appropriate annotation within NGINX Ingress Controller for Host header modification to match your requirement as well. However, you can consider using nginx.ingress.kubernetes.io/configuration-snippet annotation in order to append configuration snippet to the location block inside nginx.conf of the particular Nginx controller pod:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Host www.example-host.com;
name: my-app
spec:
rules:
- host: my-app.example.com
http:
paths:
- backend:
path: /app
serviceName: my-app
servicePort: http
We set here Host header www.example-host.com for target URL my-app.example.com.
I want to add my finding to this question of mine.
Although my solution is not using k8s Ingress Controller, our cluster is using Istio and Istio's VirtualService supports rewrite the uri and authority (Host header) as documented in this link: https://istio.io/docs/reference/config/istio.networking.v1alpha3/#HTTPRewrite
To know how I implement that in my case, you can take a look at this link: https://github.com/istio/istio/issues/11668
you can use ingress nginx controller on Kubernetes and set head and also transfer to backend and manage services connection from ingress objects.
here sharing link for rewrite target from header: https://kubernetes.github.io/ingress-nginx/examples/rewrite/
ingress nginx will be also good with SSL cert manager you can add it.
manage other thing using annotations of ingress.
check this out for ingress SSL setup you can modify it and per your need: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes
ingress will be like at last
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- myapp.abc.com
secretName: ingress-tls
rules:
- host: myapp.abc.com
http:
paths:
- path: /my-service
backend:
serviceName: my-backend
servicePort: 80

Kubernetes Ingress controllers for wildcard url mapping

I need for each of my users to access a service at a custom url eg. abccompany.mycloudapp.com , each service being a kubernetes service I'm looking at ingress controllers but I need a way to use a wildcard host field and somehow read the value into the path: and service: fields ; here's a sample ingress controller of what I have in mind:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
spec:
rules:
- host: *.bar.com
http:
paths:
- path: /{{ value of * in * .bar.com }}
backend:
serviceName: {{value of * in *.bar.com }}Svc
servicePort: 80
If you use the stock controllers you will be able to switch on hostname and go to different backends services. It sounds like you don't want to enumerate all the subdomains -> service mappings, in which case you probably need to write your own controller that writes out an nginx config that uses $http_host in the appropriate proxy_pass or redirect lines. Give it a shot (https://github.com/kubernetes/contrib/tree/master/ingress/controllers) and file bugs in that same repo if you need help.

Resources