I have a pod that responds to requests to /api/
I want to do a rewrite where requests to /auth/api/ go to /api/.
Using an Ingress (nginx), I thought that with the ingress.kubernetes.io/rewrite-target: annotation I could do it something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapi-ing
annotations:
ingress.kubernetes.io/rewrite-target: /api
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: api.myapp.com
http:
paths:
- path: /auth/api
backend:
serviceName: myapi
servicePort: myapi-port
What's happening however is that /auth/ is being passed to the service/pod and a 404 is rightfully being thrown. I must be misunderstanding the rewrite annotation.
Is there a way to do this via k8s & ingresses?
I don't know if this is still an issue, but since version 0.22 it seems you need to use capture groups to pass values to the rewrite-target value
From the nginx example available here
Starting in Version 0.22.0, ingress definitions using the annotation nginx.ingress.kubernetes.io/rewrite-target are not backwards compatible with previous versions. In Version 0.22.0 and beyond, any substrings within the request URI that need to be passed to the rewritten path must explicitly be defined in a capture group.
For your specific needs, something like this should do the trick
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapi-ing
annotations:
ingress.kubernetes.io/rewrite-target: /api/$2
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: api.myapp.com
http:
paths:
- path: /auth/api(/|$)(.*)
backend:
serviceName: myapi
servicePort: myapi-port
I have created the following example that works and which I will explain. To run this minimal example, run these commands:
$ minikube start
$ minikube addons enable ingress # might take a while for ingress pod to bootstrap
$ kubectl apply -f kubernetes.yaml
$ curl https://$(minikube ip)/auth/api/ --insecure
success - path: /api/
$ curl https://$(minikube ip)/auth/api --insecure
failure - path: /auth/api
$ curl https://$(minikube ip)/auth/api/blah/whatever --insecure
success - path: /api/blah/whatever
As you'll notice, the ingress rewrite annotation appears to be very particular about trailing slashes. If a trailing slash is not present, the request will not be rewritten. However, if a trailing slash is provided, the request uri will be rewritten and your proxy will function as expected.
After inspecting the generated nginx.conf file from inside the ingress controller, the line of code responsible for this behavior is:
rewrite /auth/api/(.*) api/$1 break;
This line tells us that only requests matching the first argument will be rewritten with the path specified by the second argument.
I believe this is bug worthy.
kubernetes.yaml
---
apiVersion: v1
kind: Service
metadata:
name: ingress-rewite-example
spec:
selector:
app: ingress-rewite-example
ports:
- name: nginx
port: 80
protocol: TCP
targetPort: 80
type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ingress-rewite-example
spec:
template:
metadata:
labels:
app: ingress-rewite-example
spec:
containers:
- name: ingress-rewite-example
image: fbgrecojr/office-hours:so-47837087
imagePullPolicy: Always
ports:
- containerPort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rewite-example
annotations:
ingress.kubernetes.io/rewrite-target: /api
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- http:
paths:
- path: /auth/api
backend:
serviceName: ingress-rewite-example
servicePort: 80
main.go
package main
import (
"fmt"
"strings"
"net/http"
)
func httpHandler(w http.ResponseWriter, r *http.Request) {
var response string
if strings.HasPrefix(r.URL.Path, "/api") {
response = "success"
} else {
response = "failure"
}
fmt.Fprintf(w, response + " - path: " + r.URL.Path + "\n")
}
func main() {
http.HandleFunc("/", httpHandler)
panic(http.ListenAndServe(":80", nil))
}
Related
We would like to use annotation for redirecting requests to a different backend service based on url args (query)
Example:
https://example.com/foo?differentQueryString=0 -> service-a
https://example.com/foo/bar?queryString=0 - service-b
Notes: path does not matter, this can be either /foo/bar or /foo or /bar/foo
We followed up on this
Kubernetes NGINX Ingress controller - different route if query string exists
and this
Kubernetes ingress routes with url parameter
But we don't want to setup ConfigMap just for this and also we don't want to duplicate requests to the ingress but rewriting
This is what we tried
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
backend.service.name = service-b
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
We were expecting to get the response but we got 502 from the Ingress Nginx
We managed to find a nice solution without rewriting and ConfigMap
Works great and also include Nginx Ingress metrics so we can do HPA accordingly
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($args ~ queryString=0){
set $proxy_upstream_name "default-service-b-80";
set $proxy_host $proxy_upstream_name;
set $service_name "service-b";
}
spec:
ingressClassName: nginx
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-a
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: service-b
port:
number: 80
$proxy_upsteam_name convention is NAMESPACE-SERVICE_NAME-PORT
I have a configuration file like below. I deploy it to EKS cluster successfully. However, when I change the nginx-conf ConfigMap and run kubectl apply command, it doesn't seem to update the nginx.config in the pod. I tried to login to the pod and look at the file /etc/nginx/nginx.config but its content is still the old one.
I have tried to run kubectl rollout status deployment sidecar-app but it doesn't help.
And it shows the updated config map when I run kubectl describe configmap nginx-conf.
It seems the container doesn't take the config map change. How can I apply the changes without deleting the pod?
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-conf
data:
nginx.conf: |
user nginx;
worker_processes 1;
events {
worker_connections 10240;
}
http {
server {
listen 8080;
server_name localhost;
location / {
proxy_pass http://localhost:9200/;
}
location /health {
proxy_pass http://localhost:9200/_cluster/health;
}
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sidecar-app
namespace: default
spec:
replicas: 1
selector:
matchLabels:
name: sidecar-app
template:
metadata:
labels:
name: sidecar-app
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- name: http
containerPort: 8080
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
readOnly: true
volumes:
- name: nginx-conf
configMap:
name: nginx-conf
items:
- key: nginx.conf
path: nginx.conf
---
apiVersion: v1
kind: Service
metadata:
name: sidecar-entrypoint
spec:
selector:
name: sidecar-app
ports:
- port: 8080
targetPort: 8080
protocol: TCP
type: NodePort
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: sidecar-ingress-1
namespace: default
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/group.name: sidecar-ingress
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/group.order: '2'
alb.ingress.kubernetes.io/healthcheck-path: /health
# alb.ingress.kubernetes.io/healthcheck-port: '8080'
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: sidecar-entrypoint
servicePort: 8080
Kubernetes treats POD and Configmap as a different/separate object and pods don't automatically restart on specific Configmap version.
There are few alternatives to achieve this.
1 ) Reloader: https://github.com/stakater/Reloader
kind: Deployment
metadata:
annotations:
reloader.stakater.com/auto: "true"
spec:
template: metadata:
configHash annotation.
https://blog.questionable.services/article/kubernetes-deployments-configmap-change/
Use wave.
https://github.com/wave-k8s/wave
You can use kubectl rollout restart deploy/sidecar-app but this will restart the pods with zero downtime. Rolling updates allow Deployments' update to take place with zero downtime by incrementally updating Pods instances with new ones.
I'm very new to Kubernetes and still learning how to use LB, ingress, etc. Currently, I'm trying to set pod-specific value(config) for each host. Looks like in ingress yaml spec, it can read config from values. But I would like to read ingress spec, e.g. host, in Values.yaml.
For example, I have two hosts
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: service
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: service-A.com
http:
paths:
- path: /
backend:
serviceName: myservicea
servicePort: 80
- host: service-B.com
http:
paths:
- path: /
backend:
serviceName: myserviceb
servicePort: 80
And I have two variables in values.yaml:
var1: aaa
var2: bbb
I want to pass
var1 to service-A.com/myservicea
var2 to service-B.com/myserviceb
or pass both, but the application must be able to identify what host it is, then it can use the right variable.
Is there any configuration/apis available to use for this purpose?
This is how you can create a secret.
kubectl create secret generic CUSTOM_VAR\
--from-literal=VAR_A=aaa\
--from-literal=VAR_B=bbb
This is how you can access the secrets in you deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myservicea-depl
spec:
replicas: 1
selector:
matchLabels:
app: myservice
template:
metadata:
labels:
app: myservice
spec:
containers:
- name: myservice
image: user/myservicea
env:
- name: VAR_A
value: aaa // this way you directly pass values here
- name: VAR_A // this way you can store this as secret in k8s
valueFrom:
secretKeyRef:
name: CUSTOM_VAR
key: VAR_A
I have this base url api.example.com
So, ingress-nginx will get the request for api.example.com and it should do follow things.
Forward api.example.com/customer to customer-srv
It doesn't work, it forwards whole mtach to customer-srv i.e. /customer/requested_url
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: api.example.in
http:
paths:
- path: /customer/?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
I tried using rewrite annotation but that doesn't work either however this worked but this is not I want to achieve.
paths:
- path: /?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
For example,
api.example.com/customer should go to http://localhost:3000 not http://localhost:3000/customer
Here is the yaml I used:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: api.example.in
http:
paths:
- path: /customer/?(.*)
pathType: Prefix
backend:
service:
name: customer-srv
port:
number: 3000
For test purpouses I created an echo server:
kubectl run --image mendhak/http-https-echo customer
And then a service:
kubectl expose po customer --name customer-srv --port 3000 --target-port 80
I checked igress ip:
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-service <none> api.example.in 192.168.39.254 80 3m43s
And I did a curl to check it:
curl 192.168.39.254/customer/asd -H "Host: api.example.in"
{
"path": "/asd",
"headers": {
"host": "api.example.in",
...
}
Notice that the echo server echoed back a path that it received, and sice it received a path that got rewritten from /customer/asd to /asd it shows this exectly path (/asd).
So as you see this does work.
I am trying to setup a nginx ingress controller; here is the yaml of the ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rules
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
spec:
rules:
- host:
http:
paths:
- path: /discover-service
backend:
serviceName: discover-service
servicePort: discover-port
When I hit http://IP/discover-service it shows an HTML without CSS and JS. As I can see, they are looked under http://IP/eureka/css/file.css instead of http://IP/discover-service/css/file.css.
How can I preserve the original url in this case?
UPDATE #1
Now I can serve the static files by creating two ingresses like:
ingress-rules-discover-root.yaml
[..]
metadata:
name: ingress-rules-discover-root
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
[...]
spec:
rules:
- host:
http:
paths:
- path: /discover-service
backend:
serviceName: discover-service
servicePort: discover-port
and another one
[...]
metadata:
name: ingress-rules-discover-path
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
[...]
spec:
rules:
- host:
http:
paths:
- path: /eureka
backend:
serviceName: discover-service
servicePort: discover-port
I believe in this case you need to remove the annotation:
nginx.ingress.kubernetes.io/rewrite-target: /
This annotation makes the ingress rewrite http://IP/discover-service/css/file.css to http://IP/eureka/css/file.css