I have many domain pointing to Ingress Controller IP. I want to block /particular-path for all the domains/sites. Is there a way to do this.
I can use nginx.ingress.kubernetes.io/configuration-snippet: | for each site. But looking for way to do for all sites/domains/Ingress resource at once.
Controller used: https://kubernetes.github.io/ingress-nginx/
There are two ways to achieve this:
1. First one is with using server-snippet annotation:
Using the annotation nginx.ingress.kubernetes.io/server-snippet it
is possible to add custom configuration in the server configuration
block.
Here is my manifest for the ingress object:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
nginx.ingress.kubernetes.io/server-snippet: |
location ~* /admin-access {
deny all;
return 403;
}
spec:
rules:
- host: domain.com
http:
paths:
- path: /
backend:
serviceName: web
servicePort: 80
Please note that using this approach :
This annotation can be used only once per host.
2. Second one is with usage of ConfigMaps and Server-snippet:
What you have to do is to locate your configMap:
kubectl get pod <nginx-ingress-controller> -o yaml
This is located the container args:
spec:
containers:
- args:
- /nginx-ingress-controller
- configmap=$(POD_NAMESPACE)/nginx-loadbalancer-conf
And then just edit it and place add the server-snippet part:
apiVersion: v1
data:
server-snippet: |
location /admin-access {
deny all;
}
This approach allows you to define restricted location globally for all host defined in Ingress resource.
Please note that with usage of server-snippet the path that you are blocking cannot be defined in ingress resource object. There is however another way with location-snippet via ConfigMap:
location ~* "^/web/admin {
deny all;
}
With this for every existing path in ingress object there will be ingress rule but it will be blocked for specific uri (In the example above it be be blocked when admin will appear after web). All of the other uri will be passed through.
3. Here`s a test:
➜ curl -H "Host: domain.com" 172.17.0.4/test
...
"path": "/test",
"headers": {
...
},
"method": "GET",
"body": "",
"fresh": false,
"hostname": "domain.com",
"ip": "172.17.0.1",
"ips": [
"172.17.0.1"
],
"protocol": "http",
"query": {},
"subdomains": [],
"xhr": false,
"os": {
"hostname": "web-6b686fdc7d-4pxt9"
...
And here is a test with a path that has been denied:
➜ curl -H "Host: domain.com" 172.17.0.4/admin-access
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.19.0</center>
</body>
</html>
➜ curl -H "Host: domain.com" 172.17.0.4/admin-access/test
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.19.0</center>
</body>
</html>
Additional information: Deprecated APIs Removed In 1.16. Here’s What You Need To Know:
The v1.22 release will stop serving the following deprecated API
versions in favor of newer and more stable API versions:
Ingress in the extensions/v1beta1 API version will no longer be
served
You cannot block specific paths. What you can do is point the path of the host inside your ingress to a default backedn application that says 404 default backedn for example.
you can apply it using the ingress annotation
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: channel-dev
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/whitelist-source-range: "27.110.30.45, 68.50.85.421"
name: dev-ingress
namespace: development
spec:
rules:
- host: hooks.dev.example.com
http:
paths:
- backend:
serviceName: hello-service
servicePort: 80
path: /incoming/message/
tls:
- hosts:
- hooks.dev.example.com
secretName: channel-dev
path https://hooks.dev.example.com/incoming/message/ will be only accessible from mentioned IPs other users will get 403 error and wont be able to access the URL.
just add this annotation in ingress
nginx.ingress.kubernetes.io/whitelist-source-range
Related
We would like to use annotation for redirecting requests to a different backend service based on url args (query)
Example:
https://example.com/foo?differentQueryString=0 -> service-a
https://example.com/foo/bar?queryString=0 - service-b
Notes: path does not matter, this can be either /foo/bar or /foo or /bar/foo
We followed up on this
Kubernetes NGINX Ingress controller - different route if query string exists
and this
Kubernetes ingress routes with url parameter
But we don't want to setup ConfigMap just for this and also we don't want to duplicate requests to the ingress but rewriting
This is what we tried
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
backend.service.name = service-b
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
We were expecting to get the response but we got 502 from the Ingress Nginx
We managed to find a nice solution without rewriting and ConfigMap
Works great and also include Nginx Ingress metrics so we can do HPA accordingly
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
set $proxy_upstream_name "default-service-b-80";
set $proxy_host $proxy_upstream_name;
set $service_name "service-b";
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
$proxy_upsteam_name convention is NAMESPACE-SERVICE_NAME-PORT
We are trying to edit our ingress-nginx.yml to make ingress-controllers pods debug traffic coming from a specific source IP.
Our setup is:
Kubernetes v1.13
Ingress-Controller v0.24.1
From NGINX and Kubernetes DOCs it appears there is no very easy way to debug traffic from a single ip (you cannot edit the nginx config directly). So, we would like to add the debug_connection directive to appear like this:
error_log /path/to/log;
...
events {
debug_connection 192.168.1.1;
}
The correct way to do it shall be through CustomAnnotations in a ConfigMap + a new ingress to enable the CustomAnnotation, so we tried this:
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app: ingress-nginx
data:
ingress-template: |
#Creating the custom annotation to make debug_connection on/off
{if index $.Ingress.Annotations "custom.nginx.org/debug_connection"}
{$ip := index $.Ingress.Annotations "custom.nginx.org/ip"}
{end}
{range $events := .Events}
events {
# handling custom.nginx.org/debug_connection
{if index $.Ingress.Annotations "custom.nginx.org/debug_connection"}
{end}
And:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: debugenabler
annotations:
kubernetes.io/ingress.class: "nginx"
custom.nginx.org/debug_connection: "on"
custom.nginx.org/ip: "192.168.1.1"
spec:
rules:
- host: "ourhostname"
http:
paths:
- path: /tea
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee
backend:
serviceName: coffee-svc
servicePort: 80
We applied ingress-nginx.yml with no errors. We see new lines in the nginx conf:
location /coffee {
set $namespace "test";
set $ingress_name "debugenabler";
set $service_name "coffee-svc";
set $service_port "80";
set $location_path "/coffee";
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = true,
use_port_in_redirects = false,
})
balancer.rewrite()
But still nothing as regard the debug_connection in the events block:
events {
multi_accept on;
worker_connections 16384;
use epoll;
}
How to insert debug_connection in the events context ?
For those who may face similar challenges, I actually managed to do it by:
Creating a ConfigMap with a new ingress-controller template file (nginx.tmpl) containing the debug_connection line (double check your ingress-controller version here, the file changes dramatically)
Creating a Volume which links at the Configmap (specifying Volume and Volumemount)
Creating a InitContainer which copy the content of the volume inside the /etc/nginx/template (this was needed to overcome probably permission-related issues) before the container start.
For point 2 and 3 you can add the relevant code at the end of a deployment or a pod code, I share an example:
volumes:
- name: nginxconf2
configMap:
name: nginxconf2
items:
- key: nginx.tmpl
path: nginx.tmpl
initContainers:
- name: copy-configs
image: {{ kubernetes.ingress_nginx.image }}
volumeMounts:
- mountPath: /nginx
name: nginxconf2
command: ['sh', '-c', 'cp -R /nginx/ /etc/nginx/template/']
I can create ingress with basic auth. I followed the template from kubernetes/ingress-nginx:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-with-auth
annotations:
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /
backend:
serviceName: http-svc
servicePort: 80
It works fine, but I need to allow 'OPTIONS' method without basic auth for pre-flight requests. Any pointers on how to do it will be very helpful.
I just encountered the same problem. I solved it by using a configuration-snippet.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-cors-auth-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
# fix cors issues of ingress when using external auth service
if ($request_method = OPTIONS) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 204;
}
more_set_headers "Access-Control-Allow-Credentials: true";
more_set_headers "Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS";
more_set_headers "Access-Control-Allow-Headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization";
more_set_headers "Access-Control-Allow-Origin: $http_origin";
more_set_headers "Access-Control-Max-Age: 600";
nginx.ingress.kubernetes.io/auth-url: "http://auth-service.default.svc.cluster.local:80"
I am setting up my first ingress in kubernetes using nginx-ingress. I set up the ingress-nginx load balancer service like so:
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "ingress-nginx",
"namespace": "...",
"labels": {
"k8s-addon": "ingress-nginx.addons.k8s.io"
},
"annotations": {
"service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "tcp",
"service.beta.kubernetes.io/aws-load-balancer-proxy-protocol": "*",
"service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn....",
"service.beta.kubernetes.io/aws-load-balancer-ssl-ports": "443"
}
},
"spec": {
"ports": [
{
"name": "http",
"protocol": "TCP",
"port": 80,
"targetPort": "http",
"nodePort": 30591
},
{
"name": "https",
"protocol": "TCP",
"port": 443,
"targetPort": "http",
"nodePort": 32564
}
],
"selector": {
"app": "ingress-nginx"
},
"clusterIP": "...",
"type": "LoadBalancer",
"sessionAffinity": "None",
"externalTrafficPolicy": "Cluster"
},
"status": {
"loadBalancer": {
"ingress": [
{
"hostname": "blablala.elb.amazonaws.com"
}
]
}
}
}
Notice how the https port has its targetPort property pointing to port 80 (http) in order to terminate ssl at the load balancer.
My ingress looks something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: something
namespace: ...
annotations:
ingress.kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
rules:
- host: www.exapmle.com
http:
paths:
- path: /
backend:
serviceName: some-service
servicePort: 2100
Now when I navigate to the url I get a Too many redirects error. Something that is confusing me is that when I add the following header "X-Forwarded-Proto: https" I get the expected response (curl https://www.example.com -v -H "X-Forwarded-Proto: https").
Any ideas how I can resolve the issue?
P.S. this works just fine with ingress.kubernetes.io/force-ssl-redirect: "false" and it doesn't seem that there are any extraneous redirects.
That is a known issue with the annotation for SSL-redirection in combination with proxy-protocol and termination of SSL connections on ELB.
Question about it was published on GitHub and here is a fix from that thread:
You should create a custom ConfigMap for an Nginx-Ingress instead of using force-ssl-redirect annotation like the following:
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app: ingress-nginx
name: nginx-ingress-configuration
namespace: <ingress-namespace>
data:
ssl-redirect: "false"
hsts: "true"
server-tokens: "false"
http-snippet: |
server {
listen 8080 proxy_protocol;
server_tokens off;
return 301 https://$host$request_uri;
}
That configuration will create an additional listener with a simple redirection to https.
Then, apply that ConfigMap to your ingress-controller, add NodePort 8080 to its container definition and to the Service.
Now, you can point the port 80 of your ELB to port 8080 of the Service.
With that additional listener, it will work.
Adding another cause for the Too many redirects error.
While working with ingress-nginx as an ingress controller in front of some k8s services.
One of the services (ArgoCD in my case) handled TLS termination by itself and always redirects HTTP requests to HTTPS.
The problem is that the nginx ingress controller also handled TLS termination and communicates with the backend service with HTTP then the result is that the ArgoCD's server always responding with a redirects to HTTPS which is the cause for the multiple redirects.
Any attempts to pass relevant values to the ingress annotations below will not help:
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: false/true
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"/"HTTPS"
The solution was to ensure that the service doesn't handle TLS by passing --insecure flag to the argocd-server deployment:
spec:
template:
spec:
containers:
- name: argocd-server
command:
- argocd-server
- --repo-server
- argocd-repo-server:8081
- --insecure # <-- Here
I had to add these annotations to make it work without changing the ingress-controller:
annotations:
kubernetes.io/ingress.class: nginx-ingress-internal # <- AWS NLB
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($http_x_forwarded_proto = 'http') {
return 301 https://$host$request_uri;
}
Another approach that worked for my environment (k8s v1.16.15, rancher/nginx-ingress-controller:nginx-0.32.0-rancher1):
apiVersion: v1
data:
compute-full-forwarded-for: "true"
use-forwarded-headers: "true"
kind: ConfigMap
metadata:
labels:
app: ingress-nginx
name: nginx-configuration
namespace: ingress-nginx
This worked with the force-ssl-redirect on the ingress of the application. It seems that the ingress-controller does not use the X-Forwarded-Proto header from the ELB out of the box.
I had this issues in Keycloak setup via helm chart as well. The SSL termination is done on ELB so to fix it . I made the following changes in helm values.
ingress:
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
This helped me fixed it.
I have a pod that responds to requests to /api/
I want to do a rewrite where requests to /auth/api/ go to /api/.
Using an Ingress (nginx), I thought that with the ingress.kubernetes.io/rewrite-target: annotation I could do it something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapi-ing
annotations:
ingress.kubernetes.io/rewrite-target: /api
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: api.myapp.com
http:
paths:
- path: /auth/api
backend:
serviceName: myapi
servicePort: myapi-port
What's happening however is that /auth/ is being passed to the service/pod and a 404 is rightfully being thrown. I must be misunderstanding the rewrite annotation.
Is there a way to do this via k8s & ingresses?
I don't know if this is still an issue, but since version 0.22 it seems you need to use capture groups to pass values to the rewrite-target value
From the nginx example available here
Starting in Version 0.22.0, ingress definitions using the annotation nginx.ingress.kubernetes.io/rewrite-target are not backwards compatible with previous versions. In Version 0.22.0 and beyond, any substrings within the request URI that need to be passed to the rewritten path must explicitly be defined in a capture group.
For your specific needs, something like this should do the trick
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapi-ing
annotations:
ingress.kubernetes.io/rewrite-target: /api/$2
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: api.myapp.com
http:
paths:
- path: /auth/api(/|$)(.*)
backend:
serviceName: myapi
servicePort: myapi-port
I have created the following example that works and which I will explain. To run this minimal example, run these commands:
$ minikube start
$ minikube addons enable ingress # might take a while for ingress pod to bootstrap
$ kubectl apply -f kubernetes.yaml
$ curl https://$(minikube ip)/auth/api/ --insecure
success - path: /api/
$ curl https://$(minikube ip)/auth/api --insecure
failure - path: /auth/api
$ curl https://$(minikube ip)/auth/api/blah/whatever --insecure
success - path: /api/blah/whatever
As you'll notice, the ingress rewrite annotation appears to be very particular about trailing slashes. If a trailing slash is not present, the request will not be rewritten. However, if a trailing slash is provided, the request uri will be rewritten and your proxy will function as expected.
After inspecting the generated nginx.conf file from inside the ingress controller, the line of code responsible for this behavior is:
rewrite /auth/api/(.*) api/$1 break;
This line tells us that only requests matching the first argument will be rewritten with the path specified by the second argument.
I believe this is bug worthy.
kubernetes.yaml
---
apiVersion: v1
kind: Service
metadata:
name: ingress-rewite-example
spec:
selector:
app: ingress-rewite-example
ports:
- name: nginx
port: 80
protocol: TCP
targetPort: 80
type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ingress-rewite-example
spec:
template:
metadata:
labels:
app: ingress-rewite-example
spec:
containers:
- name: ingress-rewite-example
image: fbgrecojr/office-hours:so-47837087
imagePullPolicy: Always
ports:
- containerPort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rewite-example
annotations:
ingress.kubernetes.io/rewrite-target: /api
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /auth/api
backend:
serviceName: ingress-rewite-example
servicePort: 80
main.go
package main
import (
"fmt"
"strings"
"net/http"
)
func httpHandler(w http.ResponseWriter, r *http.Request) {
var response string
if strings.HasPrefix(r.URL.Path, "/api") {
response = "success"
} else {
response = "failure"
}
fmt.Fprintf(w, response + " - path: " + r.URL.Path + "\n")
}
func main() {
http.HandleFunc("/", httpHandler)
panic(http.ListenAndServe(":80", nil))
}