Nginx Ingress resource annotation to setup proxy_pass directive - nginx

When I use nginx as ingress controller for my k8s cluster, specifying an ingress rule automatically creates corresponding configurations in the /etc/nginx/conf.d/ files of nginx pod.
So, it configures proxy_pass directive there to relevant upstream/backend service. And it is http.
For this ingress rule for my service:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: my-ingress
# annotations:
# nginx.org/redirect-to-https: "false"
# #nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- test.myapp.com
secretName: test-tls-secret
rules:
- host: test.myapp.com
http:
paths:
- backend:
serviceName: myui
servicePort: 80
- path: /api/
backend:
serviceName: myapp
servicePort: 88
By default, it automatically creates this directive for backend in nginx configuration:
proxy_pass http://default-my-ingress-test.myapp.com-myapp-88;
But instead, i need it to be https like this:
proxy_pass https://default-my-ingress-test.myapp.com-myapp-88;
Then only my application will work as that backend accepts https requests only.
Is there any way to modify that proxy_pass directive for a particular backend service for this purpose, using any annotations or something like that via Ingress resource?
EDIT:
Is there any annotation like that available?
Or Is there any option that I could handle it with a sidecar container in the same pod with the actual myapp container?

I had a similar requirement very recently where the backend pods expected the request on https.
What you'd need is ssl-passthrough feature of nginx controller. You need to start the nginx ingress controller with flag --enable-ssl-passthrough. This can be passed as a command line argument to the nginx deployment.
Thereafter, the ingress resource needs to be deployed with the following annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
to instruct the controller to send TLS connections directly to the backend.

Related

How to expose nginx in Minikube to outside

I have deployed Nginx in minikube in AWS EC2 Instance using the below commands:
kubectl create deployment --image=nginx nginx-app
kubectl expose deployment nginx-app --port=80 --name=nginx-http --type=NodePort
It is available on NodePort: 31568
I have added this port in Security Group and able to access it on browser in another laptop with http://EC2-PublicIP:31568
I have many microservices which are to be exposed to outside.
Is there any way to deploy an API Gateway (Nginx or Ingress) and expose on a port and should be able to access other microservices like
http://EC2-PublicIp:31568/helloworld
http://EC2-PublicIp:31568/mainpage
http://EC2-PublicIp:31568/editorspage
etc
I have tried adding metallb, accordingly, an IP is allocated to ingress (load balancer type). I was able to access all the microservices specified in ingress.yaml only inside the EC2 Instance.
This should be accessible from the outside. If yes, then how to configure it?
Any help is appreciated.
You should be able to do so since you are using Nginx.
the configuration should like something like:
Annotation comes to help
The "trick" is to set annotations to support regexp & rewrite-target with:
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
#
# the value can be set to `/` or `$1` or `$2` and so on
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
This is the "important" annotation - the rewrite one
# Without a rewrite any request will return 404
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
annotations:
#
# This is the expected line
#
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
spec:
rules:
- host: test.com # <- Set your host
http:
paths:
#
# List of desired paths
#
- path: /path1
backend:
serviceName: nginx-http
servicePort: 80
- path: /path2/[A-Z0-9]{3}
backend:
serviceName: nginx-http
servicePort: 80

GKE ingress pass cookie as header

I would like to use some kubernetes ingress (doesn't matter the ingress type) on google cloud GKE 1.19.7-gke.1302, and nginx ingress deployed by nginx-ingress-0.8.0 helm chart, and my need is to fetch a cookie and use it later to authenticate to our service. When I use nginx as deployment it works, but when I use nginx ingress it seems to ignore the annotation.
I tried using this configuration snippet for the nginx ingress, which works on the nginx deployment but not on the ingress:
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header "Authorization" "Basic $cookie_myToken";
And this is my entire ingress yaml:
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-nginx-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
# enable backend redirections
kubernetes.io/tls-acme: "true"
# enable stickiness
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-name: "route"
nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header "Authorization" "Basic $cookie_myToken";
spec:
tls:
- secretName: my-tls
hosts:
- "test.com"
defaultBackend:
service:
name: my-nodeport
port:
number: 80
rules:
- host: "test.com"
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: my-nodeport
port:
number: 80
Any ideas?
Posting this answer to make it more visible as issue was identified and solved in comment section.
Root cause of this issue was Nginx Ingress Controller misconfiguration.
In the OP's setup Nginx Ingress Controller, v.0.8.0 was used. After redeploying Nginx Ingress to latest version (v0.44.0) from Nginx Ingress Documentation it started working properly.
Yes, reinstalling nginx ingress to the latest version with helm solved the problem.

Kubernetes Nginx Ingress partial ssl termination

I'd like to split incoming traffic in Kubernetes Nginx in the following way:
Client --> Nginx --> {Service A, Service B}
The problem I am facing is Service A is an internal service and does not support HTTPS therefore SSL should be terminated for Service A. On the other hand, Service B is an external service (hosted on example.com) and only works over HTTPS.
I cannot manage to get this work easily with Kubernetes Nginx. Here is what I have come with:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-proxy
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: example.com
spec:
tls:
- hosts:
- proxy.com
secretName: secret
rules:
- host: proxy.com
http:
paths:
- path: /api/v1/endpoint
backend:
serviceName: service-a
servicePort: 8080
- path: /
backend:
serviceName: service-b
servicePort: 443
kind: Service
apiVersion: v1
metadata:
name: service-b
namespace: default
spec:
type: ExternalName
externalName: service-b.external
ports:
- port: 443
I have got a route for service-b.external:443 to point to example.com.
This solution only works if service-b is over HTTPS, but in my case, I cannot change to HTTPS for this service because of some other internal dependencies.
My problem is the backend-protocol annotation works for the whole kind and I cannot define it per path.
P.S: I am using AWS provider
Following the suggested solution and question from comments.
Yes, like mentioned below it is possible to have two ingress items. In your case
only one should have backend-protocol in it.
According to nginx ingress documentation:
Basic usage - host based routingĀ¶
ingress-nginx can be used for many use cases, inside various cloud provider and supports a lot of configurations. In this section you can find a common usage scenario where a single load balancer powered by ingress-nginx will route traffic to 2 different HTTP backend services based on the host name.
First of all follow the instructions to install ingress-nginx. Then imagine that you need to expose 2 HTTP services already installed: myServiceA, myServiceB. Let's say that you want to expose the first at myServiceA.foo.org and the second at myServiceB.foo.org. One possible solution is to create two ingress resources:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myservicea.foo.org
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-myserviceb
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myserviceb.foo.org
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
When you apply this yaml, 2 ingress resources will be created managed by the ingress-nginx instance. Nginx is configured to automatically discover all ingress with the kubernetes.io/ingress.class: "nginx" annotation. Please note that the ingress resource should be placed inside the same namespace of the backend resource.
On many cloud providers ingress-nginx will also create the corresponding Load Balancer resource. All you have to do is get the external IP and add a DNS A record inside your DNS provider that point myServiceA.foo.org and myServiceB.foo.org to the nginx external IP. Get the external IP by running:
kubectl get services -n ingress-nginx
It is also possible to have separate nginx classes as mentioned here.

Is it possible to rewrite HOST header in k8s Ingress Controller?

Due to some legacy application that relies on Host header to function correctly, I need to have an Ingress (proxy, etc) that capable of rewrite Host header and pass that to downstream (backend). Is there any Ingress Controller that supports this functionality?
Example:
End user access our website through foo.com/a for backend a and foo.com/b for backend b. But since a and b are legacy app, it only accept:
a accepts connection when Host: a.foo.com
b accepts connection when Host: b.foo.com
This can be done using this annotation: nginx.ingress.kubernetes.io/upstream-vhost: host.example.com
I'm not sure whether you can find appropriate annotation within NGINX Ingress Controller for Host header modification to match your requirement as well. However, you can consider using nginx.ingress.kubernetes.io/configuration-snippet annotation in order to append configuration snippet to the location block inside nginx.conf of the particular Nginx controller pod:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Host www.example-host.com;
name: my-app
spec:
rules:
- host: my-app.example.com
http:
paths:
- backend:
path: /app
serviceName: my-app
servicePort: http
We set here Host header www.example-host.com for target URL my-app.example.com.
I want to add my finding to this question of mine.
Although my solution is not using k8s Ingress Controller, our cluster is using Istio and Istio's VirtualService supports rewrite the uri and authority (Host header) as documented in this link: https://istio.io/docs/reference/config/istio.networking.v1alpha3/#HTTPRewrite
To know how I implement that in my case, you can take a look at this link: https://github.com/istio/istio/issues/11668
you can use ingress nginx controller on Kubernetes and set head and also transfer to backend and manage services connection from ingress objects.
here sharing link for rewrite target from header: https://kubernetes.github.io/ingress-nginx/examples/rewrite/
ingress nginx will be also good with SSL cert manager you can add it.
manage other thing using annotations of ingress.
check this out for ingress SSL setup you can modify it and per your need: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes
ingress will be like at last
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- myapp.abc.com
secretName: ingress-tls
rules:
- host: myapp.abc.com
http:
paths:
- path: /my-service
backend:
serviceName: my-backend
servicePort: 80

Kubernetes Nginx Reverse Proxy Return URL

I'm running k8s with nginx ingress. It's pointing https://www.example.com/app at some pod with rewrite "/app" to "/", app listen 80 and serves "/":
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
labels:
app: example
name: example
namespace: default
spec:
rules:
- host: www.example.com
http:
paths:
- backend:
serviceName: app
servicePort: 80
path: /app
- backend:
serviceName: app
servicePort: 80
path: /app/
tls:
- hosts:
- www.example.com
secretName: example-tls
If i'm going to https://www.example.com/app request passed to pod and pod responded with another URL (/content for example), and my browser shows me https://www.example.com/content - it's 404, because there is no ingress rule for "/content". AFAIK nginx can rewrite response with sub_filter. Another way is to develop app with some configurable path prefix. Also i can use base_url, but only with html. But is there simple way to pass prefix in return, so if pod return "/content" nginx rewrites it to https://www.example.com/app/content
Thank you.
But is there simple way to pass prefix in return, so if pod return "/content" nginx rewrites it to https://www.example.com/app/content
No, there is no simple way to make it by Ingress rules.
It is possible by Nginx itself, but not implemented in Nginx-Ingress.
I highly recommend you to return the right path from the application.

Resources