I have configured nginx-ingress-controller in kubernetes and I am trying to achieve method based routing from kubernetes.
This is my ingress.yaml file below:
kind: Ingress
metadata:
name: cafe-ingress-with-annotations
annotations:
kubernetes.io/ingress.class: "nginx"
#nginx.ingress.kubernetes.io/use-regex: "true"
#nginx.ingress.kubernetes.io/app-root: /
#nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/configuration-snippet: |
location /tea {
limit_except GET {
deny all;
}
}
spec:
rules:
- host: cafe.example.com
http:
paths:
- path: /tea
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee
backend:
serviceName: coffee-svc
servicePort: 80
according to the annotation defined it is supposed to block all other methods like POST/DELETE...etc except GET method. But it is not denying any of the methods. Please help me how can I achieve method based routing using limit_except in nginx.
Thanks in advance.
As you can read in here, the configuration-snippet annotation is used for adding an additional configuration to the NGINX location.
If you want to add custom locations in the server block, you need to use server-snippet annotation. As you can read here:
Using the annotation nginx.ingress.kubernetes.io/server-snippet it is
possible to add custom configuration in the server configuration
block.
The following Ingress manifest should work:
kind: Ingress
metadata:
name: cafe-ingress-with-annotations
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/server-snippet: |
location /tea {
limit_except GET {
deny all;
}
}
spec:
rules:
- host: cafe.example.com
http:
paths:
- path: /tea
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee
backend:
serviceName: coffee-svc
servicePort: 80
Related
I'm using nginx controller with Minikube. I can access rabbitmq management but when i access the queues i got this error:
Not found
The object you clicked on was not found; it may have been deleted on the server.
if i use port-forward it's working correctly
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rabbitmq-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
labels:
name: rabbitmq-ingress
spec:
rules:
- http:
paths:
- pathType: Prefix
path: /rabbit(/|$)(.*)
backend:
service:
name: rabbitmq-management
port:
number: 15672
Looks like you forget to mention host into ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-wildcard-host
spec:
rules:
- host: "foo.bar.com"
http:
paths:
- pathType: Prefix
path: "/bar"
backend:
service:
name: service1
port:
number: 80
host: "foo.bar.com"
Read more : https://kubernetes.io/docs/concepts/services-networking/ingress/#hostname-wildcards
I am adding the external authentication using auth-url annotation. How to set conditional request headers for the auth-url api which depends on incoming calls? Can I set the request headers in nginx controller according to incoming calls?
Edited:
Hi,
This is about adding a custom header(Id) which is expected into auth-url. I am setting the Id header which is required in authorize api of auth-url but not receiving in the api. Is this the right method to set? My next question is If it is set how can I set it conditionally depending on from which host server the request is coming through?
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-url: http://ca6dd3adc439.ngrok.io/authorize
nginx.ingress.kubernetes.io/auth-method: POST
nginx.ingress.kubernetes.io/auth-snippet: |
proxy_set_header Id "queryApps";
spec:
rules:
- host: "hw1.yourdomain"
http:
paths:
- pathType: Prefix
path: "/"
backend:
serviceName: hello-netcore-k8s
servicePort: 80
- host: "hw2.yourdomain"
http:
paths:
- pathType: Prefix
path: "/"
backend:
serviceName: hello-kubernetes-second
servicePort: 80
My next question is If it is set how can I set it conditionally depending on from which host server the request is coming through?
The best way would be to create two ingress objects with one where the external auth enabled for host hw1.yourdoman. For some reason while testing this the auth-snippet was not passing the header but with it works fine with configuration-snippet:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress-auth-on
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/auth-url: http://ca6dd3adc439.ngrok.io/authorize
nginx.ingress.kubernetes.io/auth-method: POST
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Id "queryApps";
spec:
rules:
- host: "hw1.yourdomain"
http:
paths:
- pathType: Prefix
path: "/"
backend:
serviceName: hello-netcore-k8s
servicePort: 80
As you can see here it passes the desired header:
"path": "/",
"headers": {
"host": "hw1.yourdomain",
"x-request-id": "5e91333bed960802a67958d71e787b75",
"x-real-ip": "192.168.49.1",
"x-forwarded-for": "192.168.49.1",
"x-forwarded-host": "hw1.yourdomain",
"x-forwarded-port": "80",
"x-forwarded-proto": "http",
"x-scheme": "http",
"id": "queryApps",
"user-agent": "curl/7.52.1",
"accept": "*/*"
},
"method": "GET",
"body": "",
"fresh": false,
Moving on, the second ingress object has to be configured the auth disabled for host hw2.yourdomain:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-kubernetes-ingress-auth-off
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: "hw2.yourdomain"
http:
paths:
- pathType: Prefix
path: "/"
backend:
serviceName: hello-kubernetes-second
servicePort: 80
You can then have a look at the nginx.conf to check how the those two ingresss objects are configure at controller level. This is the first ingress:
## start server hw1.yourdomain
server {
server_name hw1.yourdomain ;
listen 80 ;
listen 443 ssl http2 ;
set $proxy_upstream_name "-";
location = /_external-auth-Lw {
internal;
set $proxy_upstream_name "default-hello-netcore-k8s-80";
hello-netcore-k8s.default.svc.cluster.local;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
--------
--------
# Pass the extracted client certificate to the auth provider
set $target http://hello-netcore-k8s.default.svc.cluster.local;
proxy_pass $target;
location / {
set $namespace "default";
set $ingress_name "hello-kubernetes-ingress-auth-on";
set $service_name "hello-netcore-k8s";
set $service_port "80";
set $location_path "/";
set $balancer_ewma_score -1;
set $proxy_upstream_name "default-hello-netcore-k8s-80";
# this location requires authentication
auth_request /_external-auth-Lw;
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers to proxied server
--------
proxy_set_header Id "queryApps";
----
And this is the second one:
## start server hw2.yourdomain
server {
server_name hw2.yourdomain ;
listen 80 ;
listen 443 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "default";
set $ingress_name "hello-kubernetes-ingress-auth-off";
set $service_name "hello-kubernetes-second";
set $service_port "80";
set $location_path "/";
Your question is not pretty clear so I assume that it was something related to authentication and header injection. For NGINX ingress, there are a couple ways for you to setup the authentication. The second ways in the following will talk about the header injection.
The first method will be the easiest one. You simply setup the secret and the annotation on the ingress.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: auth-ingress
annotations:
nginx.ingress.kubernetes.io/auth-secret: my-secret
nginx.ingress.kubernetes.io/auth-type: basic
spec:
rules:
- http:
paths:
- path: /auth-url
backend:
service:
name: test
port:
number: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: normal-ingress
spec:
rules:
- http:
paths:
- path: /
backend:
service:
name: test
port:
number: 80
The second one will be more complicated but it will be useful if you do your authentication with a particular header. You can inject the snippet of NGINX configuration to the ingress. Of course, if you want to do more manipulation like header adding, you can do it in this way as well.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: auth-ingress
annotations:
nginx.ingress.kubernetes.io/server-snippet: |
if ( $some_condtion ) {
return 403;
}
spec:
rules:
- http:
paths:
- path: /auth-url
backend:
service:
name: test
port:80
Following this guide, I created an ingress controller on my local kubernetes server, the only difference is that it is created as a NodePort.
I have done some test deployments, with respective services and everything works, here the file
Deploy1:
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: helloworld1
spec:
selector:
matchLabels:
app: helloworld1
replicas: 1
template:
metadata:
labels:
app: helloworld1
spec:
containers:
- name: hello
image: gcr.io/google-samples/hello-app:1.0
ports:
- containerPort: 8080
Deploy2:
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: helloworld2
spec:
selector:
matchLabels:
app: helloworld2
replicas: 1
template:
metadata:
labels:
app: helloworld2
spec:
containers:
- name: hello
image: gcr.io/google-samples/hello-app:2.0
ports:
- containerPort: 8080
Deploy3:
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: geojson-example
spec:
selector:
matchLabels:
app: geojson-example
replicas: 1
template:
metadata:
labels:
app: geojson-example
spec:
containers:
- name: geojson-container
image: "nmex87/geojsonexample:latest"
ports:
- containerPort: 8080
Service1:
apiVersion: v1
kind: Service
metadata:
name: helloworld1
spec:
# type: NodePort
ports:
- port: 8080
selector:
app: helloworld1
Service2:
apiVersion: v1
kind: Service
metadata:
name: helloworld2
spec:
# type: NodePort
ports:
- port: 8080
selector:
app: helloworld2
Service3:
apiVersion: v1
kind: Service
metadata:
name: geojson-example
spec:
ports:
- port: 8080
selector:
app: geojson-example
This is the ingress controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/default-backend: geojson-example
spec:
rules:
- http:
paths:
- path: /geo
pathType: Prefix
backend:
service:
name: geojson-example
port:
number: 8080
- path: /test1
pathType: Prefix
backend:
service:
name: helloworld1
port:
number: 8080
- path: /test2
pathType: Prefix
backend:
service:
name: helloworld2
port:
number: 8080
When I do a GET on myServer:myPort/test1 or /test2 everything works, on /geo i get the following answer
{
"timestamp": "2021-03-09T17:02:36.606+00:00",
"status": 404,
"error": "Not Found",
"message": "",
"path": "/geo"
}
Why??
if I create a pod, and from inside the pod, i do a curl on geojson-example it works, but from the external, i obtain a 404 (i think by nginx ingress controller)
This is the log of nginx pod:
x.x.x.x - - [09/Mar/2021:17:02:21 +0000] "GET /test1 HTTP/1.1" 200 68 "-" "PostmanRuntime/7.26.8" 234 0.006 [default-helloworld1-8080] [] 192.168.168.92:8080 68 0.008 200
x.x.x.x - - [09/Mar/2021:17:02:36 +0000] "GET /geo HTTP/1.1" 404 116 "-" "PostmanRuntime/7.26.8" 232 0.013 [default-geojson-example-8080] [] 192.168.168.109:8080 116 0.012 404
What can I do?
As far the doc: This annotation is of the form nginx.ingress.kubernetes.io/default-backend: <svc name> to specify a custom default backend. This <svc name> is a reference to a service inside of the same namespace in which you are applying this annotation. This annotation overrides the global default backend.
This service will be handle the response when the service in the Ingress rule does not have active endpoints.
You cannot use same service as default backend and also for a path. When you do this the path /geo became invalid. As we know default backend serves only the inactive endpoints. Now If you tell that you want geojson-example as default backend(for inactive endpoints) again in the paths if you tell that use geojson-example for a valid path /geo then it became invalid as you are creating a deadlock type situation here.
You actually do not need to give this nginx.ingress.kubernetes.io/default-backend annotation.
Your ingress should be like below without the default annotation, or you can use the annotation but in that case you need to remove geojson-example from using for any valid path in the paths, or need to use another service for the path /geo. Options that you can use are given below:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /geo
pathType: Prefix
backend:
service:
name: geojson-example
port:
number: 8080
- path: /test1
pathType: Prefix
backend:
service:
name: helloworld1
port:
number: 8080
- path: /test2
pathType: Prefix
backend:
service:
name: helloworld2
port:
number: 8080
Or:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/default-backend: geojson-example
spec:
rules:
- http:
paths:
- path: /geo
pathType: Prefix
backend:
service:
name: <any_other_service> # here use another service except `geojson-example`
port:
number: 8080
- path: /test1
pathType: Prefix
backend:
service:
name: helloworld1
port:
number: 8080
- path: /test2
pathType: Prefix
backend:
service:
name: helloworld2
port:
number: 8080
Or:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/default-backend: geojson-example
spec:
rules:
- http:
paths:
- path: /test1
pathType: Prefix
backend:
service:
name: helloworld1
port:
number: 8080
- path: /test2
pathType: Prefix
backend:
service:
name: helloworld2
port:
number: 8080
This is for your default backend. You set the geojson-example service as a default backend.
The default backend is a service which handles all URL paths and hosts the nginx controller doesn't understand (i.e., all the requests that are not mapped with an Ingress).
Basically a default backend exposes two URLs:
/healthz that returns 200
/ that returns 404
So , if you want geojson-example service as a default backend then you don't need /geo path specification. Then your manifest file will be:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/default-backend: geojson-example
spec:
rules:
- http:
paths:
- path: /test1
pathType: Prefix
backend:
service:
name: helloworld1
port:
number: 8080
- path: /test2
pathType: Prefix
backend:
service:
name: helloworld2
port:
number: 8080
Or if you want geojson-example as a ingress valid path then you have to remove default backend annotation. Then your manifest file will be:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /geo
pathType: Prefix
backend:
service:
name: geojson-example
port:
number: 8080
- path: /test1
pathType: Prefix
backend:
service:
name: helloworld1
port:
number: 8080
- path: /test2
pathType: Prefix
backend:
service:
name: helloworld2
port:
number: 8080
I have an application running inside EKS. Istio is used as a ServiceMesh. I am having some problem with https redirect to http and then to https. It looks problem is at istio virtual service, it momentarily switches to http which I want to prevent.
This is how we installed istio [Installed version is 1.5.1]
istioctl -n infrastructure manifest apply \
--set profile=default --set values.kiali.enabled=true \
--set values.gateways.istio-ingressgateway.enabled=true \
--set values.gateways.enabled=true \
--set values.gateways.istio-ingressgateway.type=NodePort \
--set values.global.k8sIngress.enabled=false \
--set values.global.k8sIngress.gatewayName=ingressgateway \
--set values.global.proxy.accessLogFile="/dev/stdout"
This is our virtual service. Cluster contains two deployments:
myapps-front
myapps-api
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: dev-sanojapps-virtual-service
namespace: istio-system
spec:
hosts:
- "dev-mydomain.com"
gateways:
- ingressgateway
http:
- match:
- uri:
prefix: /myapp/
- uri:
prefix: /myapp
rewrite:
uri: /
route:
- destination:
host: myapp-front.sanojapps-dev.svc.cluster.local
headers:
request:
set:
"X-Forwarded-Proto": "https"
"X-Forwarded-Port": "443"
response:
set:
Strict-Transport-Security: max-age=31536000; includeSubDomains
- match:
- uri:
prefix: /v1/myapp-api/
- uri:
prefix: /v1/myapp-api
rewrite:
uri: /
route:
- destination:
host: myapp-api.sanojapps-dev.svc.cluster.local
port:
number: 8080
- match:
- uri:
prefix: /
redirect:
uri: /myapp/
https_redirect: true
headers:
request:
set:
"X-Forwarded-Proto": "https"
"X-Forwarded-Port": "443"
response:
set:
Strict-Transport-Security: max-age=31536000; includeSubDomains
Below is front-end apps yaml deployment.
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-front
namespace: sanojapps-dev
labels:
app: myapp-front
spec:
selector:
matchLabels:
app: myapp-front
template:
metadata:
labels:
app: myapp-front
spec:
containers:
- name: myapp-front
image: <ECR_REPO:TAG>
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
name: http
protocol: TCP
resources:
limits:
cpu: 500m
memory: 1024Mi
requests:
cpu: 50m
memory: 256Mi
Our gateway is configured like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: istio-system
name: sanojapps-ingress
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/subnets: ""
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:region:account:certificate/<ACM_ID>
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-2019-08
alb.ingress.kubernetes.io/wafv2-acl-arn: arn:aws:wafv2:region:account:regional/webacl/sanojapps-acl/<ACM_ID>
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: ssl-redirect
servicePort: use-annotation
- path: /*
backend:
serviceName: istio-ingressgateway
servicePort: 80
I followed the official document to install the NGINX ingress and tried to test with the Kubenretes /foo/bar/ example for regular expression.
Example:
kind: Ingress
metadata:
name: test-ingress-3
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: test.com
http:
paths:
- path: /foo/bar/bar
backend:
serviceName: test
servicePort: 80
- path: /foo/bar/[A-Z0-9]{3}
backend:
serviceName: test
servicePort: 80
NGINX config file expected content:
location ~* "^/foo/bar/[A-Z0-9]{3}" {
...
}
location ~* "^/foo/bar/bar" {
...
}
Actual content
location /foo/bar/[A-Z0-9]{3} {
...
}
location /foo/bar/bar {
...
}
Issue:
I am not able to get the same NGINX file content in ingress-controller.
How to resolve this?