K8S Nginx Ingress issue - invalid condition - nginx

I have trouble about nginx ingress using condition
Install Nginx Ingress via helm instal (tested in both nginx-ingress-1.16 and nginx-ingress-1.36
I am try to follow
ingress nginx redirect from www to https
Setup some condition
like
nginx.ingress.kubernetes.io/configuration-snippet: |
if ( $host = "mydomain.co" ) {
rewrite ^ https://www.mydomain.co$uri permanent;
}
When apply the ingress rule, nginx ingress start reload in fail status
-------------------------------------------------------------------------------
W0602 07:35:36.244415 6 queue.go:130] requeuing vincent/demoheader-ingress, err
-------------------------------------------------------------------------------
Error: exit status 1
2020/06/02 07:35:36 [notice] 982#982: ModSecurity-nginx v1.0.0
2020/06/02 07:35:36 [emerg] 982#982: invalid condition "~" in /tmp/nginx-cfg971999838:530
nginx: [emerg] invalid condition "~" in /tmp/nginx-cfg971999838:530
nginx: configuration file /tmp/nginx-cfg971999838 test failed
My Full ingress rule
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: demoheader-ingress
namespace: namespace
annotations:
kubernetes.io/ingress.class: nginx-temp
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
nginx.ingress.kubernetes.io/configuration-snippet: |
if ( $uri ~* ^/xx/(.*) ) {
rewrite ^ https://www.xxx.co permanent;
}
spec:
rules:
- host: mydomain
http:
paths:
- backend:
serviceName: header-headerv1
servicePort: 80
path: /
EOF
Any idea ?

OK i know what happen in here
I encount a wired issue for k8s apply stuff ….
While official document told you , you can apply object by this method
cat <<EOF | kubectl apply -f -
xxx
yyy
eee
EOF
https://kubernetes.io/docs/reference/kubectl/cheatsheet/
However for ingress rule , if you under a specific condition such like this
nginx.ingress.kubernetes.io/configuration-snippet: |
if ( $host = ^mydomain ) {
rewrite ^ https://www.mydomain$uri permanent;
}
you are unable to make the nginx working again (never config reload success, it will affect the following config change)
Until you delete the ingress rule and re-apply by
kubectl apply -f the-ingress-file

Related

SSL/TLS passthrough NGINX-Ingress-Controller on Openshift Not Working

I have deployed NGINX-Operator and NGINX-Ingress-Controller per the following github and the secrets from devopscube.
The current setup is:
AWS Classic LB -> ROSA Cluster [Helm NGINX-Ingress-Controller -> NGINX-Ingress -> Service -> Pod]
Here is the YAML file I used to create the NGINX-Ingress-Controller Resource. You will see that enableTLSPassthrough is set to true. However, I am unsure this is taking effect. My goal here is end to end TLS encryption from client to the NGINX service/pod. Right now I am met with error code 400 when accessing in browser through http (http works perfect fine in hello-world set up).
"400 Bad Request The plain HTTP request was sent to HTTPS port"
kind: NginxIngress
apiVersion: charts.nginx.org/v1alpha1
metadata:
name: nginxingress
namespace: nginx-ingress
spec:
controller:
affinity: {}
appprotect:
enable: false
appprotectdos:
debug: false
enable: false
maxDaemons: 0
maxWorkers: 0
memory: 0
config:
annotations: {}
entries: {}
customPorts: []
defaultTLS:
secret: nginx-ingress/default-server-secret
enableCertManager: false
enableCustomResources: true
enableExternalDNS: false
enableLatencyMetrics: false
enableOIDC: false
enablePreviewPolicies: false
enableSnippets: false
enableTLSPassthrough: true
extraContainers: []
globalConfiguration:
create: false
spec: {}
healthStatus: false
healthStatusURI: /nginx-health
hostNetwork: false
image:
pullPolicy: IfNotPresent
repository: nginx/nginx-ingress
tag: 2.3.0-ubi
ingressClass: nginx
initContainers: []
kind: deployment
logLevel: 1
nginxDebug: false
nginxReloadTimeout: 60000
nginxStatus:
allowCidrs: 127.0.0.1
enable: true
port: 8080
nginxplus: false
nodeSelector: {}
pod:
annotations: {}
extraLabels: {}
priorityClassName: null
readyStatus:
enable: true
port: 8081
replicaCount: 1
reportIngressStatus:
annotations: {}
enable: true
enableLeaderElection: true
ingressLink: ''
resources:
requests:
cpu: 100m
memory: 128Mi
service:
annotations: {}
create: true
customPorts: []
externalIPs: []
externalTrafficPolicy: Local
extraLabels: {}
httpPort:
enable: true
nodePort: ''
port: 80
targetPort: 80
httpsPort:
enable: true
nodePort: ''
port: 443
targetPort: 443
loadBalancerIP: ''
loadBalancerSourceRanges: []
type: LoadBalancer
serviceAccount:
imagePullSecretName: ''
setAsDefaultIngress: true
terminationGracePeriodSeconds: 30
tolerations: []
volumeMounts: []
volumes: []
watchNamespace: ''
wildcardTLS:
secret: null
nginxServiceMesh:
enable: false
enableEgress: false
prometheus:
create: true
port: 9113
scheme: http
secret: ''
rbac:
create: true
Taking a look at the NGINX-Ingress-Controller pod logs on creation I can see nothing about TLS being enabled. A flag does get set in the args section once the pod deploys but I am still unsure this is working.
W0802 20:33:26.594545 1 flags.go:273] Ignoring unhandled arguments: []
I0802 20:33:26.594683 1 flags.go:190] Starting NGINX Ingress Controller Version=2.3.0 PlusFlag=false
I0802 20:33:26.594689 1 flags.go:191] Commit=979db22d8065b22fedb410c9b9c5875cf0a6dc66 Date=2022-07-12T08:51:24Z DirtyState=false Arch=linux/amd64 Go=go1.18.3
I0802 20:33:26.601340 1 main.go:210] Kubernetes version: 1.22.0
I0802 20:33:26.606551 1 main.go:326] Using nginx version: nginx/1.23.0
2022/08/02 20:33:26 [notice] 13#13: using the "epoll" event method
2022/08/02 20:33:26 [notice] 13#13: nginx/1.23.0
2022/08/02 20:33:26 [notice] 13#13: built by gcc 8.5.0 20210514 (Red Hat 8.5.0-4) (GCC)
2022/08/02 20:33:26 [notice] 13#13: OS: Linux 4.18.0-305.19.1.el8_4.x86_64
2022/08/02 20:33:26 [notice] 13#13: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2022/08/02 20:33:26 [notice] 13#13: start worker processes
2022/08/02 20:33:26 [notice] 13#13: start worker process 15
2022/08/02 20:33:26 [notice] 13#13: start worker process 16
2022/08/02 20:33:26 [notice] 13#13: start worker process 17
2022/08/02 20:33:26 [notice] 13#13: start worker process 18
I0802 20:33:26.630298 1 listener.go:54] Starting Prometheus listener on: :9113/metrics
I0802 20:33:26.630860 1 leaderelection.go:248] attempting to acquire leader lease nginx-ingress/nginxingress-nginx-ingress-leader-election...
I0802 20:33:26.639466 1 leaderelection.go:258] successfully acquired lease nginx-ingress/nginxingress-nginx-ingress-leader-election
Here is the Ingress Resource YAML
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
# kubernetes.io/ingress.class: addon-http-application-routing
# nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
# nginx.ingress.kubernetes.io/ssl-redirect: "true"
# nginx.ingress.kubernetes.io/proxy-redirect-from: https
# nginx.ingress.kubernetes.io/proxy-redirect-to: https
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
# nginx.ingress.kubernetes.io/proxy-ssl-protocols: "HTTPS"
# nginx.ingress.kubernetes.io/secure-backends: "true"
spec:
defaultBackend:
service:
name: nginx
port:
number: 443
ingressClassName: nginx
tls:
- hosts:
- nginx-tlssni.apps.clustername.openshiftapps.com
secretName: nginx-tls
rules:
- host: "nginx-tlssni.apps.clustername.openshiftapps.com"
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: nginx
port:
number: 443
Thank you for your insight :)
there are many kinds of NGINX based ingress controllers. The two that are most easily confused are the NGINX INC ingress controller, and the CNCF Kubernetes Ingress Controller.
My understanding is that nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" is for the CNCF Kubernetes Ingress Controller.
Now to your question - based on this example, try changing your annotations to the following:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
nginx.org/ssl-services: "nginx" # Name of your k8s service with TLS
...

nginx-ingress rewrite-target to /api not working

i am trying to get api as location (/api) to work with intress settings
this is my ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-dev
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
#nginx.ingress.kubernetes.io/rewrite-target: /
#nginx.ingress.kubernetes.io/app-root: /
spec:
rules:
- host: 'dev.example.com'
http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: my-ruby
port: 3000
when i curl dev.example.com/api/check_version
i get error
I, [2021-12-01T16:43:35.776502 #13] INFO -- : [7253cca0b88503d625af527db32eb92e] Started GET "/api/check_serverr" for 10.42.1.228 at 2021-12-01 16:43:35 +0300
F, [2021-12-01T16:43:35.779603 #13] FATAL -- : [7253cca0b88503d625af527db32eb92e]
[7253cca0b88503d625af527db32eb92e] ActionController::RoutingError (No route matches [GET] "/api/check_version"):
if i add annotation nginx.ingress.kubernetes.io/rewrite-target: /
get error
I, [2021-12-01T16:49:11.153280 #13] INFO -- : [7832de5c07e3a173ddc86ebab5735cec] Started GET "/" for 10.42.1.228 at 2021-12-01 16:49:11 +0300
F, [2021-12-01T16:49:11.154435 #13] FATAL -- : [7832de5c07e3a173ddc86ebab5735cec]
[7832de5c07e3a173ddc86ebab5735cec] ActionController::RoutingError (No route matches [GET] "/"):
how to make a rewrite correctly in this case?
Based on the comments, the following seems to have fixed the issue.
You are looking to strip off the /api part off before the request is sent to the backend, thus requesting /api/check_version will become /check_version before it hits the backend:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
...<omitted>...
- path: /api(/|$)(.*)

Kubernetes Ingress path rewrite dosen't work as expected

I have an ingress, defined as the following:
Name: online-ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
jj.cloud.com
/online/(.*) online:80 (172.16.1.66:5001)
/userOnline online:80 (172.16.1.66:5001)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /$1
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal AddedOrUpdated 29m (x4 over 74m) nginx-ingress-controller Configuration for default/online-ingress was added or updated
If I test it with no rewrite, it's ok.
curl -X POST jj.cloud.com:31235/userOnline -H 'Content-Type: application/json' -d '{"url":"baidu.com","users":["ua"]}'
OK
However, if I try to use rewrite, it will fail.
curl -X POST jj.cloud.com:31235/online/userOnline -H 'Content-Type: application/json' -d '{"url":"baidu.com","users":["ua"]}'
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.21.3</center>
</body>
</html>
And it will produce the following error logs:
2021/11/03 10:21:25 [error] 134#134: *63 open() "/etc/nginx/html/online/userOnline" failed (2: No such file or directory), client: 172.16.0.0, server: jj.cloud.com, request: "POST /online/userOnline HTTP/1.1", host: "jj.cloud.com:31235"
172.16.0.0 - - [03/Nov/2021:10:21:25 +0000] "POST /online/userOnline HTTP/1.1" 404 153 "-" "curl/7.29.0" "-"
Why the path /online/userOnline doesn't match /online/(.*) and rewrite it to /userOnline? Or are there some other errors?
Here is the yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: online-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: jj.cloud.com
http:
paths:
- path: /online(/|$)/(.*)
pathType: Prefix
backend:
service:
name: online
port:
number: 80
- path: /userOnline
pathType: Prefix
backend:
service:
name: online
port:
number: 80
ingressClassName: nginx
When I checked the generated nginx config, I found (default-online-ingress.conf):
location /online(/|$)/(.*) {
It seems lost of the modifier for regex match, like this:
location ~* "^/online(/|$)/(.*)" {
If it's true, how to make rewrite take effect and generate correct nginx config?
If I understood your issue correctly, you have a problem with captured groups.
According to Nginx Rewrite targets it seems to me,
your patch should be path: /online(/|$)(.*)
your rewrite-target: should be rewrite-target: /$2
in addition, if you use nginx ingress, I believe you should specify that in annotation section as kubernetes.io/ingress.class: nginx
yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: online-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: jj.cloud.com
http:
paths:
- path: /online(/|$)(.*)
pathType: Prefix
backend:
service:
name: online
port:
number: 80
- path: /userOnline
pathType: Prefix
backend:
service:
name: online
port:
number: 80
ingressClassName: inner-nginx

Error when exposing Kubernetes service through NGINX INGRESS and file type: LoadBalancer

I have a Vuejs application built with Docker in 2 stages with Nginx:
# estágio de compilação
FROM node:10-alpine as build-stage
WORKDIR /app
ARG VUE_APP_BASE_URL="minha-base-url"
ENV VUE_APP_BASE_URL=${VUE_APP_BASE_URL}
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
# estágio de produção
FROM nginx:stable-alpine as production-stage
COPY --from=build-stage /app/dist /usr/share/nginx/html
COPY --from=build-stage /app/nginx.conf /etc/nginx/conf.d/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
Nginx.conf:
server {
listen 80;
location / {
alias /app/dist;
index index.html index.htm;
try_files $uri $uri/ #rewrites;
}
location #rewrites {
rewrite ^.$ /index.html last;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
Everything works perfectly by triggering the docker run command. When going up to kubernetes everything works perfectly using port-forward:
kubectl port-forward deployment/app-front 8082:80
The big problem is when I create the Service with 1- type: LoadBalancer I can't access the application via the external IP generated, 2- also when I try through NGINX INGRESS it gives error 503. Note: I have other services that work perfectly in these 2 situations mentioned above, but these other applications are not built on top of the NGINX image on the docker, so I suspect that the error may come from there due to some conflict but I cannot reach a concrete conclusion. Here are the files:
App-front-deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
kompose.cmd: kompose convert
kompose.version: 1.22.0 (955b78124)
creationTimestamp: null
labels:
io.kompose.service: app-front
name: app-front
spec:
replicas: 1
selector:
matchLabels:
io.kompose.service: app-front
app: app-front
strategy: {}
template:
metadata:
annotations:
kompose.cmd: kompose convert
kompose.version: 1.22.0 (955b78124)
creationTimestamp: null
labels:
io.kompose.service: app-front
app: app-front
spec:
containers:
- image: fabiomdsdj/agendime-front:0.0.3
name: app-front
ports:
- containerPort: 8082
name: http
env:
- name: VUE_APP_BASE_URL
value: "app"
resources: {}
restartPolicy: Always
status: {}
App-front-service.yaml:
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.cmd: kompose convert
kompose.version: 1.22.0 (955b78124)
creationTimestamp: null
labels:
io.kompose.service: app-front
name: app-front
spec:
type: ClusterIP
ports:
- name: "8082"
port: 8082
targetPort: 8082
selector:
io.kompose.service: app-front
app: app-front
tier: app-front
App-front-ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: "app.agendi.me"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: app-front
port:
number: 8082
I've been breaking my head for several days and I can't find the solution, could someone help me?
Note: With these same configurations I can expose other services that I have.

How do I mount file as ConfigMap inside DaemonSet?

I have following nginx config file (named nginx-daemonset.conf) that I want to use inside my Daemonset:
events {
worker_connections 1024;
}
http {
server {
listen 80;
location / {
proxy_pass http://my-nginx;
}
}
}
I created a ConfigMap using following command: kubectl create configmap nginx2.conf --from-file=nginx-daemonset.conf
I have following DaemonSet (nginx-daemonset-deployment.yml) inside which I am trying to mount this ConfigMap - so the previous nginx config file is used:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx-daemonset
namespace: kube-system
labels:
k8s-app: nginx-daemonset
spec:
selector:
matchLabels:
name: nginx-daemonset
template:
metadata:
labels:
name: nginx-daemonset
spec:
tolerations:
# this toleration is to have the daemonset runnable on master nodes
# remove it if your masters can't run pods
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: nginx
image: nginx
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
volumeMounts:
- name: nginx2-conf
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
volumes:
- name: nginx2-conf
configMap:
name: nginx2.conf
I deployed this Daemonset using kubectl apply -f nginx-daemonset-deployment.yml but my newly created Pod is crashing with the following error:
Error: failed to start container "nginx": Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:59: mounting "/var/lib/kubelet/pods/cd9f6f7b-31db-4ab3-bbc0-189e1d392979/volume-subpaths/nginx2-conf/nginx/0" to rootfs at "/var/lib/docker/overlay2/b21ccba23347a445fa40eca943a543c1103d9faeaaa0218f97f8e33bacdd4bb3/merged/etc/nginx/nginx.conf" caused: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
I did another Deployment with different nginx config before and everything worked fine, so the problem is probably somehow related to DaemonSet.
Please, how do I get over this error and mount the config file properly?
first create your config file as a ConfigMap like nginx-conf:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-conf
data:
envfile: |
events {
worker_connections 1024;
}
http {
server {
listen 80;
location / {
proxy_pass http://my-nginx;
}
}
}
then create your DaemonSet, volumes, configMap and finial y mount volumeMounts with subPath:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- name: http
containerPort: 80
volumeMounts:
- mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
readOnly: true
name: nginx-vol
volumes:
- name: nginx-vol
configMap:
name: nginx-conf
items:
- key: envfile
path: nginx.conf
note that for file mounting instead of directory mounting you must use "path in configMap" and "subPath in volumeMounts".

Resources