Proxy Passing to Kubernetes Cluster IP - nginx

I have a Kubernetes cluster that's running a API Server deployment and a Nginx deployment. I exposed the Server with a ClusterIP service so the Nginx pods can route to it with a proxy pass then I externalize the nginx pods with a LoadBalancer service. But it's not working. What am I misunderstanding? I get a 404 not found for LOAD_BALANCER_IP/api
Server Deployment:
apiVersion: v1
kind: Service
metadata:
name: league-pool-production-server-service
spec:
type: ClusterIP
selector:
app: league-pool-server
ports:
- protocol: TCP
port: 80
targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: league-pool-server-deployment
spec:
replicas: 3
selector:
matchLabels:
app: league-pool-server
template:
metadata:
labels:
app: league-pool-server
spec:
containers:
- name: league-pool-server-container
image: helloitsian/league-pool-server
ports:
- containerPort: 3000
env:
- name: MONGO_URL
value: mongodb://mongo-0.mongo,mongo-1.mongo,mongo-2.mongo:27017/player-pools
imagePullPolicy: Always
Nginx Deployment:
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: league-pool-nginx-deployment
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: league-pool-nginx-deployment
minReplicas: 2
maxReplicas: 5
metrics:
- type: Resource
resource:
name: cpu
targetAverageUtilization: 80
---
apiVersion: v1
kind: Service
metadata:
name: league-pool-nginx-service
spec:
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app: league-pool-nginx-production
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
- name: https
protocol: TCP
port: 443
targetPort: 443
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: league-pool-nginx-deployment
labels:
app: league-pool-nginx-production
spec:
replicas: 1
selector:
matchLabels:
app: league-pool-nginx-production
template:
metadata:
labels:
app: league-pool-nginx-production
spec:
containers:
- name: league-pool-nginx-production
image: helloitsian/league-pool-nginx
imagePullPolicy: Always
resources:
limits:
cpu: 300m
requests:
cpu: 100m
memory: 200Mi
ports:
- containerPort: 443
- containerPort: 80
Nginx site conf:
server {
listen 80;
root /usr/share/nginx/html;
location / {
try_files $uri $uri/ =404;
}
location /api {
proxy_pass http://league-pool-server;
}
}

Related

How can I define the `limit_req_zone`?

I am working with nginx-ingress-controller (this is not the same that ingress-nginx )
I have this ingress file
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-test"
acme.cert-manager.io/http01-edit-in-place: "true"
nginx.org/location-snippets: |
limit_req zone=by_web;
spec:
ingressClassName: nginx
rules:
- host: my.domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service
port:
number: 80
tls:
- hosts:
- my.domain.com
secretName: quickstart-example-tls
I was able to define a limit_req using nginx.org/location-snippets.
How can I define the limit_req_zone?
limit_req_zone $request_uri zone=by_web:10m rate=60r/m;
Regards.
According to this article from official documentation, you can define limit_req_zone by adding following ConfigMap keys to location-snippets and server-snippets annotations:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-test"
acme.cert-manager.io/http01-edit-in-place: "true"
nginx.org/location-snippets: |
geo $limit {
default 1;
10.0.0.0/8 0;
192.168.0.0/24 0;
}
map $limit $request_uri {
default '';
'1' $binary_remote_addr;
}
limit_req_zone $request_uri zone=by_web:10m rate=1r/s;
nginx.org/server-snippets: |
location / {
limit_req zone=by_web burst=10 nodelay;
}
spec:
ingressClassName: nginx
rules:
- host: my.domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service
port:
number: 80
tls:
- hosts:
- my.domain.com
secretName: quickstart-example-tls
So this will let you rate limit to be defined on requests from anyone who is not on an “allowlist”

Kubernetes qBitTorrentWebUI showing only on path '/'

I am trying to host a qBitTorrent server with Kubernetes. I have composed a YAML for the https://hub.docker.com/r/linuxserver/qbittorrent docker container.
The problem is that it is accessible only from path /. As soon as I move it to /torrent it does not find it anymore: 404 Not Found.
Steps to replicate:
apply following yamls
helm install nginx ingress-nginx/ingress-nginx
go to service_ip:8080, settings, WebUI, uncheck "Enable Host header validation"
go to localhost:nginx_port/torrent
Result:
page not loading
Expected Result:
qBitTorrent WebUi appears and works
What I tried:
adding nginx.ingress.kubernetes.io/rewrite-target: / to annotations
server.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: torrent-deployment
labels:
app: torrent
spec:
replicas: 1
selector:
matchLabels:
pod-label: torrent-pod
template:
metadata:
labels:
pod-label: torrent-pod
spec:
containers:
- name: linuxserver
image: linuxserver/qbittorrent:amd64-latest
---
apiVersion: v1
kind: Service
metadata:
name: torrent-service
labels:
app: torrent
spec:
selector:
pod-label: torrent-pod
ports:
- port: 8080
name: torrent-deployment
ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: torrent-ingress
annotations:
kubernetes.io/ingress.class: nginx
labels:
app: torrent
spec:
rules:
- http:
paths:
- path: /torrent
pathType: Prefix
backend:
service:
name: torrent-service
port:
number: 8080
Thanks to #matt_j I have found a workaround. I wrote and YAML for nginx myself and added the configurations from the post mentioned by matt ( https://github.com/qbittorrent/qBittorrent/wiki/NGINX-Reverse-Proxy-for-Web-UI ) and it worked.
These are the YAMLs I came up with:
server.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
namespace: nginx
spec:
selector:
matchLabels:
pod-label: nginx
template:
metadata:
labels:
pod-label: nginx
spec:
containers:
- name: nginx
image: nginx:latest
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/
volumes:
- name: nginx-conf
configMap:
name: nginx-conf
items:
- key: nginx.conf
path: nginx.conf
replicas: 1
# status:
---
apiVersion: v1
kind: Service
metadata:
namespace: nginx
name: nginx
labels:
app: nginx
spec:
selector:
pod-label: nginx
ports:
- port: 80
name: nginx
config.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-conf
namespace: nginx
data:
nginx.conf: |
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
http {
server {
server_name 10.152.183.95;
listen 80;
location /torrent/ {
proxy_pass http://torrent-service.qbittorrent:8080/;
#proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
#proxy_cookie_path / "/; Secure";
}
}
}
events {
worker_connections 1024;
}

Kubernetes nginx ingress controller Redirect

I have a website deployed in Kubernetes and ingress controller working.
I need to redirect my old subdomain to the new subdomain(old.example.com -> new.example.com).
I made some research and found that I have to use the annotation:
nginx.ingress.kubernetes.io/rewrite-target
my ingress file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rancher-ing
namespace: test
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/tls-acme: "true"
ingress.kubernetes.io/secure-backends: "true"
ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
tls:
- hosts:
- new.example.com
secretName: new.example.com
rules:
- host: new.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rancher-logo-service
port:
number: 80
- host: old.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rancher-logo-service
port:
number: 80
i created an ingress controller config for the new url.
and updated the old config into this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rancher-ing-redirect
namespace: test
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: letsencrypt-prod
kubernetes.io/tls-acme: "true"
ingress.kubernetes.io/secure-backends: "true"
ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: https://new.example.com/
spec:
rules:
- host: old.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rancher-logo-service
port:
number: 80

How can i configure ingress and nginx ingress controller to send http traffic to port 80 and https traffic to 443 port, with the same host and path

I have a service for my app:
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
labels:
io.kompose.service: app
name: app
spec:
type: ClusterIP
ports:
- name: "443"
port: 443
targetPort: 443
- name: "80"
port: 80
targetPort: 80
selector:
io.kompose.service: app
and ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: app-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/proxy-body-size: 200m
spec:
rules:
- host: test.app.com
http:
paths:
- backend:
serviceName: app
servicePort: 443
and ingress controller:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
serviceAccountName: nginx-ingress-serviceaccount
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.25.0
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
- --enable-ssl-passthrough
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 33
runAsUser: 33
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
The problem is that the ingress controller send's both http and https traffic to port 443.
i want to configure it to be able to send http traffic to port 80 and https traffic to port 443. how can i do that? or there is a something i don't understand correctly?.
Can you update a question to a bit more precise?
I can deduct in 2 cases.
1) If you want to send the same traffic to port 80 and 443?
2) You want traffic received on port 443 in ingress to send to port 80 to pod.
Still, I will give in short.
Case 1
It is not possible. you have to opt for istio etc which provide traffic mirroring.
Case 2
It is valid and used a lot called SSL offloading. for that, you have to remove below annotation and update ingress to send traffic on port 80 than 443.
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
annotations:
ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
rules:
- host: my.url.com
http:
paths:
- path: /
backend:
serviceName: servicea
servicePort: 80
- path: /
backend:
serviceName: servicea
servicePort: 443

Is there a point of having pod-level nginx when using nginx ingress?

I was wondering if I should have the pod level nginx in the implementations below:
I was previously using a normal ingress and kube-lego after migrating from VMs and now I am using cert-manager and GKE.
My Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: myapp-static-ip
kubernetes.io/ingress.class: nginx
kubernetes.io/ingress.allow-http: "false"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/rewrite-target: /
certmanager.k8s.io/cluster-issuer: letsencrypt
namespace: default
spec:
tls:
- hosts:
- myapp.com
secretName: myapp-crt
rules:
- host:
http:
paths:
- path: /
backend:
serviceName: myapp
servicePort: http
My service:
apiVersion: v1
kind: Service
metadata:
name: myapp
labels:
app: myapp
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 32111
protocol: "TCP"
name: http
selector:
app: myapp
My Deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: myapp
labels:
app: myapp
spec:
replicas: 3
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: gcr.io/myapp-1/myapp:latest
imagePullPolicy: Always
env:
- name: DB_USER
valueFrom:
secretKeyRef:
name: cloudsql
key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: cloudsql
key: password
-
name: STATIC_ROOT
value: https://storage.googleapis.com/myapp-api/static/
-
name: STATIC_URL
value: https://storage.googleapis.com/myapp-api/static/
-
name: MEDIA_ROOT
value: /myapp/media
-
name: MEDIA_URL
value: http://myapp.com/media/
-
name: nginx
image: nginx
command: [nginx, -g,'daemon off;']
imagePullPolicy: Always
volumeMounts:
-
name: api-nginx-config
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
-
name: myapp-media
mountPath: /myapp/media/
ports:
- containerPort: 80
- image: b.gcr.io/cloudsql-docker/gce-proxy:1.05
name: cloudsql-proxy
command: ["/cloud_sql_proxy", "--dir=/cloudsql",
"-instances=myapp-1:europe-west1:myapp-api=tcp:5432",
"-credential_file=/secrets/cloudsql/credentials.json"]
volumeMounts:
- name: cloudsql-oauth-credentials
mountPath: /secrets/cloudsql
readOnly: true
- name: ssl-certs
mountPath: /etc/ssl/certs
- name: cloudsql
mountPath: /cloudsql
- name: myapp-media
mountPath: /myapp/media
volumes:
- name: cloudsql-oauth-credentials
secret:
secretName: cloudsql-oauth-credentials
- name: cloudsql
emptyDir:
- name: api-nginx-config
configMap:
name: api-nginx-config
-
name: myapp-media
persistentVolumeClaim:
claimName: myapp-media
my nginx conf:
apiVersion: v1
kind: ConfigMap
metadata:
name: api-nginx-config
data:
nginx.conf: |
events {
worker_connections 1024;
}
http {
upstream api {
server 127.0.0.1:8080 fail_timeout=0;
}
server {
access_log /var/log/nginx/http-access.log;
error_log /var/log/nginx/http-error.log;
listen 80;
listen [::]:80;
server_name myapp.com;
location /media/ {
alias /myapp/media;
}
location = /favicon.ico {
access_log off;
log_not_found off;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://127.0.0.1:8080/;
}
}
}
Is it serving any major purpose given I could directly map the myapp/media directly to /media in the volume mount and my tls is handled by ingress. My major concern is with the pod-level nginx as I highlighted earlier, is it useless in this case? Is it just a baggage I am carrying over from previous implementations?
Generally, there is not really a point to have an extra nginx pod. If you do, you would have something of a double ingress. An nginx ingress controller pod already has nginx in it and you can scale that up/down.
One reason you would want to keep it is for backward compatibility, if for example, you want to use an ingress but want to gradually roll it out in this sort of fashion: create new nginx ingress -> flip traffic from your own nginx only through the new nginx ingress and your own nginx, until you have flipped all your pods -> remove your own nginx gradually until you have removed them all.
Another reason is to support a very specific nginx configuration that is not supported by the nginx ingress controller yet.
You may need to run your own nginx as deployment , for the reasons listed in the above answer , plus , you may need to scale the nginx deployment , let say 10 replicas. you cant scale ingress like that. But in any case , you just need one of them.

Resources