I am running celery flower on port inside Kubernetes with nginx-ingress controller
I want to do a rewrite where requests to /flower/(.*) request goes to /$1 according to their documentation:
https://flower.readthedocs.io/en/latest/config.html?highlight=nginx#url-prefix
server {
listen 80;
server_name example.com;
location /flower/ {
rewrite ^/flower/(.*)$ /$1 break;
proxy_pass http://example.com:5555;
proxy_set_header Host $host;
}
}
I have come up with the following ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: backend-airflow-ingress
namespace: edna
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/use-regex: "true"
ingress.kubernetes.io/rewrite-target: /$2
# nginx.ingress.kubernetes.io/app-root: /flower
spec:
rules:
- host:
http:
paths:
- path: /flower(/|$)(.*)
backend:
serviceName: airflow-flower-service
servicePort: 5555
Inside POD running flower, I successfully get
curl localhost:5555/dashboard
However if get into the POD running Nginx controller, then it fails.
curl localhost/flower/dashboard
I get response by the flower:
<div class="span12">
<p>
Error, page not found
</p>
</div>
this is what I see inside nginx.conf in nginx-controller pod
server {
server_name _ ;
listen 80 default_server reuseport backlog=511 ;
listen 443 default_server reuseport backlog=511 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location ~* "^/flower(/|$)(.*)" {
set $namespace "edna";
set $ingress_name "backend-airflow-ingress";
set $service_name "";
set $service_port "";
set $location_path "/flower(/|${literal_dollar})(.*)";
rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
use_port_in_redirects = false,
})
balancer.rewrite()
plugins.run()
}
# be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
# will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
# other authentication method such as basic auth or external auth useless - all requests will be allowed.
#access_by_lua_block {
#}
header_filter_by_lua_block {
lua_ingress.header()
plugins.run()
}
body_filter_by_lua_block {
}
Ok figured this out
ingress.kubernetes.io/rewrite-target: /$2
should be in my case a different annotation
nginx.ingress.kubernetes.io/rewrite-target: /$2
Related
I have the kubernetes ingress file with some micro services are exposed fully over 'abc.example.com'
How i can achieve to allow access for only one specific API from one micro service (empl-service) while rest all other microservices are exposed fully.
Specific Api to be allowed: https://abc.example.com/empl-service/api/v1/employees/capacityReport
i am trying to append in location directive as below, but i am able to access all the API's from empl-service.
location ~* /gateway/(api|(?!(empl)).+service)/api/v1/employees/capacityReport
{
deny all;
return 404 "resource not found";
}
it will be great, if someone help me in figuring out for specific API access.
below is nginx config in kubernetes ingress file.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: xyz-platform-ingress
namespace: di
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/configuration-snippet: |
server_tokens off;
location ~* /.+\.html$ {
deny all;
return 404 "resource not found";
}
location ~* (.*\/actuator.*) {
deny all;
return 404 "resource not found";
}
location ~* /gateway/(api|(?!(empl)).+service)/api/v1/employees/capacityReport
{
deny all;
return 404 "resource not found";
}
location ~* /gateway/(api|(?!(market|jdw|asd|xyz|sale)).+service).* {
deny all;
return 404 "resource not found";
}
spec:
tls:
- hosts:
- abc.example.com
secretName: tls-secret
rules:
- host: abc.example.com
http:
paths:
- path: /gateway/?(.*)
backend:
serviceName: example-service
servicePort: 8080
I have following ingress.yaml file
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-configuration-snippet
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/configuration-snippet: |
location /base/path/v1/api/update {
deny all;
return 404;
}
spec:
rules:
- http:
paths:
- path: /base/path(/|$)(.*)
backend:
serviceName: myApi
servicePort: 8080
But when I send a request to https:///base/path/v1/api/update it succeeds and I got following error in nginx ingress controller
Error: exit status 1
2020/08/06 18:35:07 [emerg] 1734#1734: location "/base/path/v1/api/update" is outside location "^/base/path(/|$)(.*)" in /tmp/nginx-cfg008325631:2445
nginx: [emerg] location "/base/path/v1/api/update" is outside location "^/base/path(/|$)(.*)" in /tmp/nginx-cfg008325631:2445
nginx: configuration file /tmp/nginx-cfg008325631 test failed
Can somebody help?
The configuration-snippet is to add configs to locations.
If you want to add a custom location to the server context, you should use the server-snippet instead:
Using the annotation nginx.ingress.kubernetes.io/server-snippet it is
possible to add custom configuration in the server configuration
block.
You also need to use some modifiers and regex to make it work (~* and ^).
The following config should work:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-configuration-snippet
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/server-snippet: |
location ~* "^/base/path/v1/api/update" {
deny all;
return 403;
}
spec:
rules:
- http:
paths:
- path: /base/path(/|$)(.*)
backend:
serviceName: myApi
servicePort: 8080
The final nginx.config should end like this:
$ kubectl exec -n kube-system nginx-ingress-controller-6fc5bcc8c9-chkxf -- cat /etc/nginx/nginx.conf
[...]
location ~* "^/base/path/v1/api/update" {
deny all;
return 403;
}
location ~* "^/base/path(/|$)(.*)" {
[...]
}
I have Nginx Open Source on AKS service. Every thing was good but unable to serve static content like index.html or favicon.ico.
When I open http:// it is not serving the index.html by default[i get 404] and if I try to open any static content I get 404 error.
nginx configuration was passed as ConfigMap and below is the config file that talks about serving static content.
server {
listen 80;
server_name localhost;
root /opt/abc/html; #also tried root /opt/abc/html/
location / {
root /opt/abc/html; #also tried root /opt/abc/html/
index index.html;
try_files $uri $uri/ /index.html?$args;
...
...
..
proxy_pass http://tomcat;
}
}
Setup:
Kubernetes on AKS
Nginx Open Source [no ingress]
configMaps to mount config.d
the static content (/opt/abc/html) was passed into pod with kubernetes cp command. [will this work?]
ref: https://github.com/RammusXu/toolkit/tree/master/k8s/echoserver
Here's a example to mount nginx.conf from ConfigMap
And make sure you kubectl rollout restart deployment echoserver after update ConfigMap. Pod only clone ConfigMap when it created. It don't sync or auto-updated.
apiVersion: apps/v1
kind: Deployment
metadata:
name: echoserver
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: echoserver
template:
metadata:
labels:
app: echoserver
spec:
volumes:
- name: config
configMap:
name: nginx-config
containers:
- name: echoserver
# image: gcr.io/google_containers/echoserver:1.10
image: openresty/openresty:1.15.8.2-1-alpine
ports:
- containerPort: 8080
name: http
# nginx.conf override
volumeMounts:
- name: config
subPath: nginx.conf
# mountPath: /etc/nginx/nginx.conf
mountPath: /usr/local/openresty/nginx/conf/nginx.conf
readOnly: true
---
apiVersion: v1
kind: Service
metadata:
name: echoserver
namespace: default
spec:
type: NodePort
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
selector:
app: echoserver
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config
namespace: default
data:
nginx.conf: |-
events {
worker_connections 1024;
}
env HOSTNAME;
env NODE_NAME;
env POD_NAME;
env POD_NAMESPACE;
env POD_IP;
http {
default_type 'text/plain';
# maximum allowed size of the client request body. By default this is 1m.
# Request with bigger bodies nginx will return error code 413.
# http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size
client_max_body_size 10m;
# https://blog.percy.io/tuning-nginx-behind-google-cloud-platform-http-s-load-balancer-305982ddb340
keepalive_timeout 650;
keepalive_requests 10000;
# GZIP support
gzip on;
gzip_min_length 128;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/css
text/plain
text/javascript
application/javascript
application/json
application/x-javascript
application/xml
application/xml+rss
application/xhtml+xml
application/x-font-ttf
application/x-font-opentype
application/vnd.ms-fontobject
image/svg+xml
image/x-icon
application/rss+xml
application/atom_xml
application/vnd.apple.mpegURL
application/x-mpegurl
vnd.apple.mpegURL
application/dash+xml;
init_by_lua_block {
local template = require("template")
-- template syntax documented here:
-- https://github.com/bungle/lua-resty-template/blob/master/README.md
tmpl = template.compile([[
Hostname: {{os.getenv("HOSTNAME") or "N/A"}}
Pod Information:
{% if os.getenv("POD_NAME") then %}
node name: {{os.getenv("NODE_NAME") or "N/A"}}
pod name: {{os.getenv("POD_NAME") or "N/A"}}
pod namespace: {{os.getenv("POD_NAMESPACE") or "N/A"}}
pod IP: {{os.getenv("POD_IP") or "N/A"}}
{% else %}
-no pod information available-
{% end %}
Server values:
server_version=nginx: {{ngx.var.nginx_version}} - lua: {{ngx.config.ngx_lua_version}}
Request Information:
client_address={{ngx.var.remote_addr}}
method={{ngx.req.get_method()}}
real path={{ngx.var.request_uri}}
query={{ngx.var.query_string or ""}}
request_version={{ngx.req.http_version()}}
request_scheme={{ngx.var.scheme}}
request_uri={{ngx.var.scheme.."://"..ngx.var.host..":"..ngx.var.server_port..ngx.var.request_uri}}
Request Headers:
{% for i, key in ipairs(keys) do %}
{{key}}={{headers[key]}}
{% end %}
Request Body:
{{ngx.var.request_body or " -no body in request-"}}
]])
}
server {
# please check the benefits of reuseport https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1
# basically instructs to create an individual listening socket for each worker process (using the SO_REUSEPORT
# socket option), allowing a kernel to distribute incoming connections between worker processes.
listen 8080 default_server reuseport;
listen 8443 default_server ssl http2 reuseport;
ssl_certificate /certs/certificate.crt;
ssl_certificate_key /certs/privateKey.key;
# Replace '_' with your hostname.
server_name _;
location / {
lua_need_request_body on;
content_by_lua_block {
ngx.header["Server"] = "echoserver"
local headers = ngx.req.get_headers()
local keys = {}
for key, val in pairs(headers) do
table.insert(keys, key)
end
table.sort(keys)
ngx.say(tmpl({os=os, ngx=ngx, keys=keys, headers=headers}))
}
}
}
}
i am trying to make a dynamic proxy_pass with nginx, doing something like that:
Requests to foo.mywebsite.com are forwarded to service with foo name
Requests to bar.mywebsite.com are forwarded to service with bar name
my nginx.conf works with static values, but with regex, works for about 5 minutes and then errors start
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config-dns-file
data:
nginx.conf: |
server {
listen 80;
server_name ~^(?<subdomain>.*?)\.;
resolver kube-dns.kube-system.svc.cluster.local valid=5s;
location /healthz {
return 200;
}
location / {
proxy_pass http://$subdomain.default.svc.cluster.local;
}
}
my pod gets the service ip instead of the name, here are the logs
2019/11/11 22:30:40 [error] 6#6: 163 10.default.svc.cluster.local
could not be resolved (3: Host not found), client: 10.142.0.34,
server: ~^(?.?)., request: "GET / HTTP/1.1", host:
"10.142.0.34"
10. is the beginning of ip to the point.
I don't know what is going wrong, can anyone help me with this, thank you!!
FIX WITH THIS
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config-dns-file
data:
nginx.conf: |
server {
listen 80;
server_name ~^(?<subdomain>.*?)\.;
resolver kube-dns.kube-system.svc.cluster.local valid=5s;
location /healthz {
return 200;
}
location / {
proxy_set_header Host $host
proxy_pass http://$subdomain.default.svc.cluster.local;
}
}
I'm in the process of migration of our application from single instance Docker-compose configuration to Kubernetes. I currently have the following example NGINX configuration, running as a reverse proxy of my application:
server {
server_name example.com;
ssl_certificate /etc/nginx/certs/${CERT_NAME};
ssl_certificate_key /etc/nginx/certs/${KEY_NAME};
listen 443 ssl;
keepalive_timeout 70;
access_log /var/log/nginx/access.log mtail;
ssl_protocols xxxxxx
ssl_ciphers xxxxxx
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
rewrite_log on;
resolver 127.0.0.11 ipv6=off;
location /push/ {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/htpasswd;
rewrite /push/(.*) /index.php/$1 break;
proxy_pass pushinterface:3080;
}
location /flights/ {
rewrite /flights/(.*) /$1 break;
proxy_pass flightstats:3090;
}
location /api/ {
proxy_pass $api;
}
location /grafana/ {
access_log off;
log_not_found off;
proxy_pass http://grafana:3000;
rewrite ^/grafana/(.*) /$1 break;
}
}
My initial plans for the reverse proxy part was implementing an ingress with NGINX ingress controller, but I saw that my configuration can be created as Ingress only with NGINX Plus. That's why I decided to try with Traefik, but I'm not sure if it's still possible to have different rewrites of the path for each service.
I tried the following Ingress configuration, but it seems it's not working:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-traefik
annotations:
kubernetes.io/ingress.class: traefik
traefik.frontend.rule.type: ReplacePathRegex
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: pushinterface
servicePort: 80
path: /push/(.*) /index/$1
- backend:
serviceName: flights
servicePort: 80
path: /flights/(.*) /$1
- backend:
serviceName: api
servicePort: 80
path: /api
- backend:
serviceName: grafana
servicePort: 80
path: /grafana/(.*) /$1
I will appreciate any help for solving this task
After several hours of unsuccessful attempts to solve my issue, I did it with Nginx ingress controller and it works great! Here's the ingress configuration:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite /push/(.*) /index/$1 break;
rewrite /flights/(.*) /$1 break;
rewrite /grafana/(.*) /$1 break;
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: pushinterface
servicePort: 80
path: /push
- backend:
serviceName: flights
servicePort: 80
path: /flights
- backend:
serviceName: api
servicePort: 80
path: /api
- backend:
serviceName: grafana
servicePort: 80
path: /grafana
Thanks to everyone for the answers! :)
Using ReplacePathRegex rule type in your example does not guarantee that incoming requests will be forwarded to the target backends, as mentioned in Traefik Documentation.
In order to route requests to the endpoints, use Matcher instead of Modifiers rules, as they are designed for that purpose.
Find a separate discussion on the similar issue here.