Kubernetes (AKS) Multiple Nginx Ingress Controllers - nginx

Question

I have a cluster set up in AKS, and need to install a second Nginx Ingress controller so we can manage the source IP using Local instead of Cluster IP.
Reference:
https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-typeloadbalancer
I know how to do the above, but I am confused on how I can deploy my Services that need to be running on that Ingress Controller from my manifests. I use helm to schedule my deployments. Below is an example of how I am installing the ingress controller currently with Ansible in our pipeline:
- name: Create NGINX Load Balancer (Internal) w/ IP {{ aks_load_balancer_ip }}
command: >
helm upgrade platform ingress-nginx/ingress-nginx -n kube-system
--set controller.service.loadBalancerIP="{{ aks_load_balancer_ip }}"
--set controller.service.annotations."service\.beta\.kubernetes\.io\/azure-load-balancer-internal"="true"
--set controller.replicaCount="{{ nginx_replica_count }}"
--version {{ nginx_chart_ver }}
--install
I am aware here, that I will need to set some values to change the metadata name etc since we are using charts from bitnami.
Below is an example of my Ingress charts:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ template "service.fullname" . }}
labels:
app: {{ template "service.name" . }}
chart: {{ template "service.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/proxy-body-size: {{ .Values.ingress.proxy_body_size }}
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/add-base-url: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-read-timeout: "{{ .Values.ingress.proxy_read_timeout }}"
nginx.ingress.kubernetes.io/proxy-send-timeout: "{{ .Values.ingress.proxy_send_timeout }}"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "{{ .Values.ingress.proxy_connect_timeout }}"
spec:
tls:
- hosts:
- "{{ .Values.service_url }}"
secretName: tls-secret
rules:
- host: "{{ .Values.service_url }}"
http:
paths:
- pathType: Prefix
path: "/?(.*)"
backend:
service:
name: {{ template "service.name" . }}
port:
number: {{ .Values.service.port }}
I am unsure if there is an annotation that should be applied in the ingress to define which ingress controller this gets assigned to.
I was reading this: https://kubernetes.io/docs/reference/kubernetes-api/service-resources/ingress-v1/#IngressSpec
And feel like I just need to add that underneath spec and define the name of the load balancer? Any assistance would be appreciated.

Answer 1

Solution:
I was able to figure this out myself, I created both my ingresses using the following logic:
helm install nginx-two ingress-nginx/ingress-nginx -n kube-system \
--set controller.replicaCount=2 \
--set controller.service.loadBalancerIP="10.24.1.253" \
--set controller.ingressClassResource.name=nginx-two \
--set controller.ingressClass="nginx-two" \
--set controller.service.annotations."service\.beta\.kubernetes\.io\/azure-load-balancer-internal"="true" \
--version {{ nginx_chart_ver }}
--install
helm install nginx-oneingress-nginx/ingress-nginx -n kube-system \
--set controller.replicaCount=2 \
--set controller.service.loadBalancerIP="10.24.1.254" \
--set controller.ingressClassResource.name=nginx-one \
--set controller.ingressClass="nginx-one" \
--set controller.service.annotations."service\.beta\.kubernetes\.io\/azure-load-balancer-internal"="true" \
--version {{ nginx_chart_ver }}
--install
Then I was able to reference it in my ingress chart by updating the annotation to the name. I am sure this can be smoothed out further by defining the namespace if I wanted the ingress controller there, but I would need to update my PSP first.
Working Ingress example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ template "service.fullname" . }}
labels:
app: {{ template "service.name" . }}
chart: {{ template "service.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
annotations:
kubernetes.io/ingress.class: "nginx-two"