How to debug ingress-controller connections with a single IP by ConfigMap - nginx

Question

We are trying to edit our ingress-nginx.yml to make ingress-controllers pods debug traffic coming from a specific source IP.
Our setup is:
Kubernetes v1.13
Ingress-Controller v0.24.1
From NGINX and Kubernetes DOCs it appears there is no very easy way to debug traffic from a single ip (you cannot edit the nginx config directly). So, we would like to add the debug_connection directive to appear like this:
error_log /path/to/log;
...
events {
debug_connection 192.168.1.1;
}
The correct way to do it shall be through CustomAnnotations in a ConfigMap + a new ingress to enable the CustomAnnotation, so we tried this:
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app: ingress-nginx
data:
ingress-template: |
#Creating the custom annotation to make debug_connection on/off
{if index $.Ingress.Annotations "custom.nginx.org/debug_connection"}
{$ip := index $.Ingress.Annotations "custom.nginx.org/ip"}
{end}
{range $events := .Events}
events {
# handling custom.nginx.org/debug_connection
{if index $.Ingress.Annotations "custom.nginx.org/debug_connection"}
{end}
And:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: debugenabler
annotations:
kubernetes.io/ingress.class: "nginx"
custom.nginx.org/debug_connection: "on"
custom.nginx.org/ip: "192.168.1.1"
spec:
rules:
- host: "ourhostname"
http:
paths:
- path: /tea
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee
backend:
serviceName: coffee-svc
servicePort: 80
We applied ingress-nginx.yml with no errors. We see new lines in the nginx conf:
location /coffee {
set $namespace "test";
set $ingress_name "debugenabler";
set $service_name "coffee-svc";
set $service_port "80";
set $location_path "/coffee";
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = true,
use_port_in_redirects = false,
})
balancer.rewrite()
But still nothing as regard the debug_connection in the events block:
events {
multi_accept on;
worker_connections 16384;
use epoll;
}
How to insert debug_connection in the events context ?

Answer 1

For those who may face similar challenges, I actually managed to do it by:
Creating a ConfigMap with a new ingress-controller template file (nginx.tmpl) containing the debug_connection line (double check your ingress-controller version here, the file changes dramatically)
Creating a Volume which links at the Configmap (specifying Volume and Volumemount)
Creating a InitContainer which copy the content of the volume inside the /etc/nginx/template (this was needed to overcome probably permission-related issues) before the container start.
For point 2 and 3 you can add the relevant code at the end of a deployment or a pod code, I share an example:
volumes:
- name: nginxconf2
configMap:
name: nginxconf2
items:
- key: nginx.tmpl
path: nginx.tmpl
initContainers:
- name: copy-configs
image: {{ kubernetes.ingress_nginx.image }}
volumeMounts:
- mountPath: /nginx
name: nginxconf2
command: ['sh', '-c', 'cp -R /nginx/ /etc/nginx/template/']