NGINX Container Not Loading Static Files using Traefik / Kubernetes - nginx

I am running the Traefik Ingress Controller on Kubernetes (AKS). I've successfully deployed my Django application using the Traefik Ingress but it's not currently loading any static files (and therefore the styling isn't working).
Static files are served from a custom NGINX container with /static. So if my domain name is xyz.com, static is served from xyz.com/static.
apiVersion: v1
kind: Service
metadata:
name: nginxstaticservice
labels:
app: nginxstatic
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
selector:
app: nginxstatic
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginxstatic-ingress
annotations:
kubernetes.io/ingress.class: traefik
traefik.frontend.rule.type: PathPrefixStrip
# traefik.ingress.kubernetes.io/frontend-entry-points: http,https
spec:
rules:
- host: xyz.com
http:
paths:
- path: /static
backend:
serviceName: nginxstaticservice
servicePort: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginxstatic-deployment
labels:
app: nginxstatic
spec:
replicas: 1
selector:
matchLabels:
app: nginxstatic
template:
metadata:
labels:
app: nginxstatic
spec:
containers:
- name: nginxstatic
image: nginxstatic:latest
ports:
- containerPort: 80
imagePullSecrets:
This is the default.conf running on the NGINX container (this was previously working in a Website configuration.
server {
listen 80;
server_name _;
client_max_body_size 200M;
set $cache_uri $request_uri;
location = /favicon.ico { log_not_found off; access_log off; }
location = /robots.txt { log_not_found off; access_log off; }
ignore_invalid_headers on;
add_header Access-Control-Allow_Origin *;
location /static {
autoindex on;
alias /static;
}
location /media {
autoindex on;
alias /media;
}
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
}

Resolved in comments, PathPrefixStrip was used incorrectly which caused Nginx to see different paths than expected.

Related

Nginx 503 ingress controller k8s

I have an application running on nginx port 9000, and a service attached to it. If I put service as a LoadBalancer, I can open IP:PORT/app/pages in my browser (ClusterIP and NodePort with nginx doesn't work). Created Ingress controller with ALB, and A record to point to app.mydomain.com, but I keep getting 503 or 404 errors, even 400 sometimes (tried couple of ports/paths etc). Can someone point me to what should I look at? I want to be able to open https://app.mydomain.com/app/pages. cert-manager is complaining with 400 errors also when retrieving the certificate.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-app-dev
namespace: app-dev
annotations:
cert-manager.io/issuer: letsencrypt-nginx
ingress.kubernetes.io/rewrite-target: /
# ingressclass.kubernetes.io/is-default-class: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- app.mydomain.com
secretName: letsencrypt-nginx
rules:
- host: app.mydomain.com
http:
paths:
- pathType: Prefix
backend:
service:
name: app-service
port:
number: 8080
path: /
apiVersion: v1
kind: Service
metadata:
name: app-service
namespace: app-dev
spec:
type: LoadBalancer
ports:
- port: 9090
protocol: TCP
targetPort: 8080
selector:
app: app
IP of Ingress is added as a A record to DNS (app.mydomain.com). This is my nginx conf in the docker image
bash-5.1# cat /etc/nginx/conf.d/default.conf
server {
listen 8080 ssl;
ssl_certificate /ssl/cert;
ssl_certificate_key /ssl/key;
location / {
root /www;
autoindex off;
add_header 'Access-Control-Allow-Origin' '*';
}
location /healthz {
return 200 'ok';
}
}

nginx Reverseproxy for Kibana gives empty dashboard on kubernetes

We have Deployed Kibana & Elastic Search on our Kubernetes Cluster. I want to access Kibana via my Nginx by configuring reverse proxy . I am able to configure the nginx.conf with kibana url but when I go to nginx_url/Kibana it gives me a White Dashboard ,no content on the web . It is Strange because i got it worked on my Local machine & it worked but same configuration on kubernetes giving me Empty response .
Below is my Kibana deployment file
apiVersion: apps/v1
kind: Deployment
metadata:
#namespace: vpd-cluster-elk
name: kibana
labels:
app: kibana
spec:
replicas: 1
selector:
matchLabels:
app: kibana
template:
metadata:
labels:
app: kibana
spec:
containers:
- name: kibana
image: docker.elastic.co/kibana/kibana:7.10.1
resources:
limits:
cpu: 1000m
requests:
cpu: 100m
env:
- name: ELASTICSEARCH_URL
value: "http://ES_Url:9200"
- name: SERVER_BASEPATH
value: "/app/home"
- name: SERVER_REWRITEBASEPATH
value: "false"
ports:
- containerPort: 5601
volumes:
- name: "config"
configMap:
name: kibana-config
---
apiVersion: v1
kind: Service
metadata:
#namespace: vpd-cluster-elk
name: kibana
labels:
app: kibana
spec:
ports:
- port: 5601
targetPort: 5601
selector:
app: kibana
type: LoadBalancer
---
apiVersion: v1
kind: ConfigMap
metadata:
name: kibana-config
data:
kibana.yml: |
---
server.name: kibana
elasticsearch.url: http://ES_url:9200
# server.basePath: /app/home
# server.rewriteBasePath: true
Below is my Nginx.conf file
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-conf
data:
nginx.conf: |
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# To allow special characters in headers
ignore_invalid_headers off;
client_max_body_size 0;
proxy_buffering off;
server {
listen 80;
server_name _;
location /app/home{
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_pass http://Kibana_Url:5601;
}
}
}
Kindly let me know what I am wrong or do i need any additional configurations

Nginx Giving 403

I have usecase to have 2 micro services 1st Frontend 2nd Backend
where frontend pod is exposed to outer world and rendering page from backend pod
While access the frontend service i am getting 403 error
below are the yaml for respective pods and config map
frontend pod
apiVersion: v1
kind: Pod
metadata:
name: frontend
labels:
run: frontend
spec:
volumes:
- name: webpage
hostPath:
path: /home/vagrant/html
type: Directory
- name: nginx-config-volume
configMap:
name: nginx-config
containers:
- image: nginx
name: frontend
volumeMounts:
- name: nginx-config-volume
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
- name: webpage
mountPath: /var/www/html
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
backend pod
apiVersion: v1
kind: Pod
metadata:
name: backend
labels:
run: backend
spec:
volumes:
- name: webpage
hostPath:
path: /home/vagrant/html
type: Directory
containers:
- image: php:7.2-fpm
name: backend
volumeMounts:
- name: webpage
mountPath: /app
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
Config Map for Nginx
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-config
data:
nginx.conf: |
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
server {
listen 80 default_server;
listen [::]:80 default_server;
# Set nginx to serve files from the shared volume!
root /var/www/html;
server_name _;
location / {
try_files $uri $uri/ =404;
}
location ~ \.php$ {
include fastcgi_params;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass backend:9000;
}
}
}
Services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
backend ClusterIP 10.102.90.23 <none> 9000/TCP 60m
frontend NodePort 10.111.88.172 <none> 80:30054/TCP 9h

Is there a point of having pod-level nginx when using nginx ingress?

I was wondering if I should have the pod level nginx in the implementations below:
I was previously using a normal ingress and kube-lego after migrating from VMs and now I am using cert-manager and GKE.
My Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: myapp-static-ip
kubernetes.io/ingress.class: nginx
kubernetes.io/ingress.allow-http: "false"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/rewrite-target: /
certmanager.k8s.io/cluster-issuer: letsencrypt
namespace: default
spec:
tls:
- hosts:
- myapp.com
secretName: myapp-crt
rules:
- host:
http:
paths:
- path: /
backend:
serviceName: myapp
servicePort: http
My service:
apiVersion: v1
kind: Service
metadata:
name: myapp
labels:
app: myapp
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 32111
protocol: "TCP"
name: http
selector:
app: myapp
My Deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: myapp
labels:
app: myapp
spec:
replicas: 3
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: gcr.io/myapp-1/myapp:latest
imagePullPolicy: Always
env:
- name: DB_USER
valueFrom:
secretKeyRef:
name: cloudsql
key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: cloudsql
key: password
-
name: STATIC_ROOT
value: https://storage.googleapis.com/myapp-api/static/
-
name: STATIC_URL
value: https://storage.googleapis.com/myapp-api/static/
-
name: MEDIA_ROOT
value: /myapp/media
-
name: MEDIA_URL
value: http://myapp.com/media/
-
name: nginx
image: nginx
command: [nginx, -g,'daemon off;']
imagePullPolicy: Always
volumeMounts:
-
name: api-nginx-config
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
-
name: myapp-media
mountPath: /myapp/media/
ports:
- containerPort: 80
- image: b.gcr.io/cloudsql-docker/gce-proxy:1.05
name: cloudsql-proxy
command: ["/cloud_sql_proxy", "--dir=/cloudsql",
"-instances=myapp-1:europe-west1:myapp-api=tcp:5432",
"-credential_file=/secrets/cloudsql/credentials.json"]
volumeMounts:
- name: cloudsql-oauth-credentials
mountPath: /secrets/cloudsql
readOnly: true
- name: ssl-certs
mountPath: /etc/ssl/certs
- name: cloudsql
mountPath: /cloudsql
- name: myapp-media
mountPath: /myapp/media
volumes:
- name: cloudsql-oauth-credentials
secret:
secretName: cloudsql-oauth-credentials
- name: cloudsql
emptyDir:
- name: api-nginx-config
configMap:
name: api-nginx-config
-
name: myapp-media
persistentVolumeClaim:
claimName: myapp-media
my nginx conf:
apiVersion: v1
kind: ConfigMap
metadata:
name: api-nginx-config
data:
nginx.conf: |
events {
worker_connections 1024;
}
http {
upstream api {
server 127.0.0.1:8080 fail_timeout=0;
}
server {
access_log /var/log/nginx/http-access.log;
error_log /var/log/nginx/http-error.log;
listen 80;
listen [::]:80;
server_name myapp.com;
location /media/ {
alias /myapp/media;
}
location = /favicon.ico {
access_log off;
log_not_found off;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://127.0.0.1:8080/;
}
}
}
Is it serving any major purpose given I could directly map the myapp/media directly to /media in the volume mount and my tls is handled by ingress. My major concern is with the pod-level nginx as I highlighted earlier, is it useless in this case? Is it just a baggage I am carrying over from previous implementations?
Generally, there is not really a point to have an extra nginx pod. If you do, you would have something of a double ingress. An nginx ingress controller pod already has nginx in it and you can scale that up/down.
One reason you would want to keep it is for backward compatibility, if for example, you want to use an ingress but want to gradually roll it out in this sort of fashion: create new nginx ingress -> flip traffic from your own nginx only through the new nginx ingress and your own nginx, until you have flipped all your pods -> remove your own nginx gradually until you have removed them all.
Another reason is to support a very specific nginx configuration that is not supported by the nginx ingress controller yet.
You may need to run your own nginx as deployment , for the reasons listed in the above answer , plus , you may need to scale the nginx deployment , let say 10 replicas. you cant scale ingress like that. But in any case , you just need one of them.

Configure Kubernetes Traefik Ingress with different path rewrites for each service

I'm in the process of migration of our application from single instance Docker-compose configuration to Kubernetes. I currently have the following example NGINX configuration, running as a reverse proxy of my application:
server {
server_name example.com;
ssl_certificate /etc/nginx/certs/${CERT_NAME};
ssl_certificate_key /etc/nginx/certs/${KEY_NAME};
listen 443 ssl;
keepalive_timeout 70;
access_log /var/log/nginx/access.log mtail;
ssl_protocols xxxxxx
ssl_ciphers xxxxxx
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
rewrite_log on;
resolver 127.0.0.11 ipv6=off;
location /push/ {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/htpasswd;
rewrite /push/(.*) /index.php/$1 break;
proxy_pass pushinterface:3080;
}
location /flights/ {
rewrite /flights/(.*) /$1 break;
proxy_pass flightstats:3090;
}
location /api/ {
proxy_pass $api;
}
location /grafana/ {
access_log off;
log_not_found off;
proxy_pass http://grafana:3000;
rewrite ^/grafana/(.*) /$1 break;
}
}
My initial plans for the reverse proxy part was implementing an ingress with NGINX ingress controller, but I saw that my configuration can be created as Ingress only with NGINX Plus. That's why I decided to try with Traefik, but I'm not sure if it's still possible to have different rewrites of the path for each service.
I tried the following Ingress configuration, but it seems it's not working:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-traefik
annotations:
kubernetes.io/ingress.class: traefik
traefik.frontend.rule.type: ReplacePathRegex
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: pushinterface
servicePort: 80
path: /push/(.*) /index/$1
- backend:
serviceName: flights
servicePort: 80
path: /flights/(.*) /$1
- backend:
serviceName: api
servicePort: 80
path: /api
- backend:
serviceName: grafana
servicePort: 80
path: /grafana/(.*) /$1
I will appreciate any help for solving this task
After several hours of unsuccessful attempts to solve my issue, I did it with Nginx ingress controller and it works great! Here's the ingress configuration:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite /push/(.*) /index/$1 break;
rewrite /flights/(.*) /$1 break;
rewrite /grafana/(.*) /$1 break;
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: pushinterface
servicePort: 80
path: /push
- backend:
serviceName: flights
servicePort: 80
path: /flights
- backend:
serviceName: api
servicePort: 80
path: /api
- backend:
serviceName: grafana
servicePort: 80
path: /grafana
Thanks to everyone for the answers! :)
Using ReplacePathRegex rule type in your example does not guarantee that incoming requests will be forwarded to the target backends, as mentioned in Traefik Documentation.
In order to route requests to the endpoints, use Matcher instead of Modifiers rules, as they are designed for that purpose.
Find a separate discussion on the similar issue here.

Resources