Installed: https://helm.nginx.com/ stable version
[root#node1 ~]# helm search repo nginx-stable
NAME CHART VERSION APP VERSION DESCRIPTION
nginx-stable/nginx-ingress 0.6.1 1.8.1 NGINX Ingress Controller
kubectl run nginx --image=nginx
kubectl run apache --image=httpd
kubectl expose pod nginx --port=80 --name=nginx-svc
kubectl expose pod apache --port=80
As a method of testing both pods were accessible by nodeport succesfully
I created two yaml files to get ingress working
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ing-rule
annotations:
nginx.org/rewrites: "serviceName=nginx-svc rewrite=/"
spec:
rules:
- host: kubernetes.somelan.lan
http:
paths:
- path: /nginx
backend:
serviceName: nginx-svc
servicePort: 80
and for the apache
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: apache-ingress
annotations:
nginx.org/rewrites: "serviceName=apache rewrite=/"
spec:
rules:
- host: kubernetes.somelan.lan
http:
paths:
- path: /apache
backend:
serviceName: apache
servicePort: 80
When i apply both yaml seperatly, they are being accepted by the nginx-controller be it that it logs
conflicting server name "kubernetes.somelan.lan" on 0.0.0.0:80, ignored
result:
I can succesfully access http://kubernetes.somelan.lan/apache, but i am unable to access http://kubernetes.somelan.lan/nginx. Applying the yaml in different order renders the same result
If i remove the apache yaml, the nginx ingress start to work; i can access http://kubernetes.somelan.lan/nginx
The error code and logs are:
020/10/27 20:34:28 [error] 62#62: *20 open() "/etc/nginx/html/nginx" failed (2: No such file or directory), client: 10.10.70.22, server: kubernetes.somelan.lan, request: "GET /nginx HTTP/1.1", host: "kubernetes.somelan.lan"
10.10.70.22 - - [27/Oct/2020:20:34:28 +0000] "GET /nginx HTTP/1.1" 404 153 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0" "-"
2020/10/27 20:34:28 [error] 62#62: *20 open() "/etc/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.10.70.22, server: kubernetes.somelan.lan, request: "GET /favicon.ico HTTP/1.1", host: "kubernetes.somelan.lan", referrer: "kubernetes.somelan.lan/nginx"
10.10.70.22 - - [27/Oct/2020:20:34:28 +0000] "GET /favicon.ico HTTP/1.1" 404 153 "kubernetes.somelan.lan/nginx" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0" "-"
However if i create one yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ing-rule
annotations:
nginx.org/rewrites: "serviceName=nginx-svc rewrite=/; serviceName=apache rewrite=/"
spec:
rules:
- host: kubernetes.somelan.lan
http:
paths:
- path: /nginx
backend:
serviceName: nginx-svc
servicePort: 80
- path: /apache
backend:
serviceName: apache
servicePort: 80
All works as expected, i can access both /nginx and /apache
On the nginx controller the usage of a single yaml gives the following config on the controller:
# configuration for default/nginx-ing-rule
upstream default-nginx-ing-rule-kubernetes.somelan.lan-apache-80 {
zone default-nginx-ing-rule-kubernetes.somelan.lan-apache-80 256k;
random two least_conn;
server 10.233.112.2:80 max_fails=1 fail_timeout=10s max_conns=0;
}
upstream default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 {
zone default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 256k;
random two least_conn;
server 10.233.64.2:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
listen 80;
server_tokens on;
server_name kubernetes.somelan.lan;
location /nginx {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80/
}
location /apache {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-nginx-ing-rule-kubernetes.somelan.lan-apache-80/;
}
If i apply 2 yamls that i specified earlier, nginx controller has 2 configs
Config for the apache ingress
# configuration for default/apache-ingress
upstream default-apache-ingress-kubernetes.somelan.lan-apache-80 {
zone default-apache-ingress-kubernetes.somelan.lan-apache-80 256k;
random two least_conn;
server 10.233.112.2:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
listen 80;
server_tokens on;
server_name kubernetes.somelan.lan;
location /apache {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-apache-ingress-kubernetes.somelan.lan-apache-80/;
}
}
}
config for the nginx ingress
# configuration for default/nginx-ing-rule
upstream default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 {
zone default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80 256k;
random two least_conn;
server 10.233.64.2:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
listen 80;
server_tokens on;
server_name kubernetes.somelan.lan;
location /nginx {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-nginx-ing-rule-kubernetes.somelan.lan-nginx-svc-80/;
}
}
Am i wrong in expecting the method of using two yaml should render the same result as using one yaml file?
There are few things to explain here.
The error you see:
conflicting server name "kubernetes.somelan.lan" on 0.0.0.0:80, ignored
is because you cannot have two server blocks with the same server_name and listening at the same port.
It is important to understand how the operations in nginx model work. These are as follows:
Order Ingress rules by CreationTimestamp field, i.e., old rules first.
If the same path for the same host is defined in more than one Ingress, the oldest rule wins.
If more than one Ingress contains a TLS section for the same host, the oldest rule wins.
If multiple Ingresses define an annotation that affects the configuration of the Server block, the oldest rule wins.
Create a list of NGINX Servers (per hostname)
Create a list of NGINX Upstreams
If multiple Ingresses define different paths for the same host, the ingress controller will merge the definitions.
Annotations are applied to all the paths in the Ingress.
Multiple Ingresses can define different annotations. These definitions are not shared between Ingresses.
Also, bear in mind When a reload is required:
New Ingress Resource Created.
TLS section is added to existing Ingress.
Change in Ingress annotations that impacts more than just upstream configuration. For instance load-balance annotation does not require a
reload.
A path is added/removed from an Ingress.
An Ingress, Service, Secret is removed.
Some missing referenced object from the Ingress is available, like a Service or Secret.
A Secret is updated.
Now, according to the info above, in your use case the below scenario should take place if you have created two separate Ingresses:
If multiple Ingresses define different paths for the same host, the
ingress controller will merge the definitions.
And if this has not happened, you need to check your Ingress Controller and it's version to make sure that it support the merging operation. You can see more details about that here.
Finally to answer your main question:
Am i wrong in expecting the method of using two yaml should render the
same result as using one yaml file?
It depends on your configuration and the Ingress Controller that you use.
In your particular use case using two yamls will not work the same as using a single one due to the server_name conflict. You you find it difficult to enable the merging option, consider using the Merge Ingress Controller:
Merge Ingress Controller combines multiple ingress resources into a
new one.
I'm trying to setup basic auth for my test ingress rule and I couldn't figure out why it doesn't work. I can still access the site without a password prompt.
Versions:
EKS 1.16
Helm chart nginx-ingress-0.5.2
Nginx version 1.7.2(also tried with 1.7.0 and latest)
basic-auth secret content:
kubectl get secret basic-auth -o yaml
apiVersion: v1
data:
auth: Zm9vOiRhcHIxJHZ4RzVoc1VQJE1KZmpNcEQ2WHdPV1RaaTFDQUdlYTEK
kind: Secret
metadata:
creationTimestamp: "2020-07-02T04:46:58Z"
name: basic-auth
namespace: default
resourceVersion: "8252"
selfLink: /api/v1/namespaces/default/secrets/basic-auth
uid: e3b8a6d3-009b-4a4c-ad8b-b460381933d8
type: Opaque
Ingress rule:
Ingress rule:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ing
annotations:
kubernetes.io/ingress.class: "nginx"
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
rules:
- host: test.*****.com
http:
paths:
- backend:
serviceName: docker-hello-world-svc
servicePort: 8088
Also I haven't found basic-auth section within nginx controller configuration file for hello-world-ing service:
kubectl -n nginx-ingress exec -it dev-nginx-ingress-6d5f459bf5-s4qqg -- cat /etc/nginx/conf.d/default-hello-world-ing.conf
***
location / {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-hello-world-ing-***-docker-hello-world-svc-8088;
}
***
I haven't found anything suspicious in controller logs.
Basic auth works fine with another helm repo stable/nginx-ingress instead of nginx-stable/nginx-ingress.
the nginx-stable repository is for the commercial Nginx/NginxPlus that uses different configurations while the official Helm stable/nginx-ingress uses the open source nginx ingress.
I am running kubernetes v1.16 under docker desktop for windows. I have installed the nginx-ingress controller v1.7.9 using helm. I have update my hosts file to have the following entry
127.0.0.1 application.local
I have a backend service named hedgehog-service.
The following ingress definition correctly forwards request to the backend.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ml-zoo-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: application.local
http:
paths:
- path: /hedgehog/
backend:
serviceName: hedgehog-service
servicePort: 80
curl application.local/hedgehog works as expected and hits the backend service.
However in order to correctly use the backend service I need to rewrite the target removing the url prefix /hedgehog. Hence I have the following ingress definition
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ml-zoo-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: application.local
http:
paths:
- path: /hedgehog(/|$)(.*)
backend:
serviceName: hedgehog-service
servicePort: 80
As indicated here: https://kubernetes.github.io/ingress-nginx/examples/rewrite/#rewrite-target
Now when I curl application.local/hedgehog/test the ingress controller does not communicate with the backend but according to the logs attempts to open a file
2020/06/23 12:46:48 [error] 708#708: *792 open() "/etc/nginx/html/hedgehog/test" failed (2: No such file or directory), client: 192.168.65.3, server: application.local, request: "GET /hedgehog/test HTTP/1.1", host: "application.local"
192.168.65.3 - - [23/Jun/2020:12:46:48 +0000] "GET /hedgehog/test HTTP/1.1" 404 153 "-" "curl/7.65.3" "-"
Here is the content of etc/nginx/conf.d/default-ml-zoo-ingress
# configuration for default/ml-zoo-ingress
upstream default-ml-zoo-ingress-application.local-hedgehog-service-80 {
zone default-ml-zoo-ingress-application.local-hedgehog-service-80 256k;
random two least_conn;
server 10.1.0.48:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
listen 80;
server_tokens on;
server_name application.local;
location /hedgehog(/|$)(.*) {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://default-ml-zoo-ingress-application.local-hedgehog-service-80;
}
}
Does anyone know why my URLs are not getting rewritten and the requsts delivered to the backend service?
Thanks in advance!
OK having played around with this for hours, once I had written the question my next google turned up an answer.
I installed nginx using helm with the following repo stable/nginx-ingress. However according to this issue https://github.com/kubernetes/ingress-nginx/issues/5756 that is in fact a legacy repository. I uninstalled my controller and changed the repository to ingress-nginx
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
and everything appears to be working as expected. Still not sure why the previous controller installation failed but I can get back to work :)
EDIT: For the aid of others that might end up here - in hindsight I am wondering if the reinstallation simply meant that I deleted and recreated my ingress which might have solved the original problem. In other words make sure you try recreating the ingress before reinstalling the ingress controller with helm.
Hi nginx extraordinaire's
I am using nginx as a load balancer and reverse proxy to play the role of an external-facing API gateway to interface applications and api's hosted in kubernetes. these are all exposed via ingress
The issue I am facing is that nginx is giving me 404s when I try to access a standard angular web app via the /test URL
See error:
Nginx was set up using the following config - nginx.conf
events {
worker_connections 1024;
}
http {
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
ssl on;
ssl_certificate /etc/letsencrypt/live/myhostname/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myhostname/privkey.pem;
client_max_body_size 1G;
upstream k8snodes {
server 192.168.2.10;
server 192.168.2.11;
}
server {
listen 443 ssl;
location / {
proxy_pass http://k8snodes/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Forwarded-Proto $scheme;
}
include /etc/nginx/conf.d/*.conf;
}
}
Sitting behind /test is a Kubernetes ingress controller that serves the angular application. I can confirm that application can be accessed fine when directly through ingress so there is something that nginx is not happy with.
Ingress config
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
creationTimestamp: "2019-09-14T07:43:49Z"
generation: 2
name: test-ingress
namespace: default
resourceVersion: "12193067"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/test-
ingress
uid: f8c5ea11-9caf-431e-9a18-19bd3981eece
spec:
rules:
- host: myhostname
http:
paths:
- backend:
serviceName: test-svc
servicePort: 80
path: /test
status:
loadBalancer:
ingress:
- {}
Is there something I have done wrong on the Nginx config? I have other API's that are reverse proxied fine it seems to be the web applications that try to serve static files that are giving issues.
When I add proxy_set_header Host $host; to my ingress file, it stops working. I think I need to set that as the value as I'm trying to expose SSL over Keycloak where they state
Configure your reverse proxy or loadbalancer to properly
set X-Forwarded-For and X-Forwarded-Proto HTTP headers.
Configure your reverse proxy or loadbalancer to preserve
the original 'Host' HTTP header.
This is my ingress file:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-keycloak
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
kubernetes.io/ingress.class: "nginx"
certmanager.k8s.io/issuer: "letsencrypt-prod"
certmanager.k8s.io/acme-challenge-type: http01
spec:
tls:
- hosts:
- mysite.com
secretName: staging-iam-tls
rules:
- host: mysite.com
http:
paths:
- path: /auth
backend:
serviceName: keycloak-http
servicePort: 80
How do I set it up that I can set the host?
try this:
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "HOST $host";
more_set_headers "X-Real-IP $remote_addr";
more_set_headers "X-Forwarded-For $proxy_add_x_forwarded_for";
more_set_headers "X-Forwarded-Proto $scheme";
basically keep your configuration snippet and change the proxy_set_headers to more_set_headers
see: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#configuration-snippet